Difference: MOC11RFC (1 vs. 25)

Revision 252019-10-09 - AdaNebot

 
META TOPICPARENT name="MocInfo"

MOC 1.1 Proposed Recommendation: Request for Comments


Summary

Latest Draft: MOC 1.1 (Proposed Recommendation)

The substantive differences between version 1.1 of MOC and the preceding version 1.0 are:

  • The String MOC serialization was moved from an informative section (suggested syntax) to the normative section.
  • A MOCORDER convention for String MOC and JSON MOC was added.

Reference Interoperable Implementations

(Indicate here the links to at least two Reference Interoperable Implementations)

  • MOCPy >=0.5.6
  • Aladin (>v10.126): read/write ASCII MOC (select ASCII format when exporting MOC - see snapshot below) + new script command dedicated to ASCII MOC (ex: draw MOC 3/1,2,3 4/56-67)
  • The relational registry at http://dc.g-vo.org/tap parses the ASCII MOCs in Registry records into the rr.stc_spatial table (consumer)
  • DaCHS since version 1.1 produces ASCII MOCs when serialising MOC-values database columns.
  • The Moc class within the AST/PyAST WCS library now supports reading and writing ASCII and JSON MOCs in addition to FITS.
To see non-trivial examples for ASCII MOCs coming out of a database, run something like

select top 100 * from rr.stc_spatial
where 1=contains(point(10, 10), coverage)
and 0=contains(point(20, 20), coverage)

on http://dc.g-vo.org/tap.

Implementations Validators

Comments Prior to Official RFC Period

As I was considering semantic validation cases, I noticed this sentence in section 2.3.2:

"Warning: In this basic simple ASCII string format the values may be not sorted, and the MOC may be not well-formed."

which seems to contradict section 2.2.3:

" An encoded MOC must also be “well-formed”. To keep the canonical property of MOC, the redundant cells are not allowed and the hierarchical encoding principle must be respected. Thus it is not allowed to encode 4 sibling cells instead of their parent (only at level 0 for the 12 first diamonds). Using this simple constraint, there is one and only one way to describe a specific region and this makes the interoperability principle easy to implement."

Maybe the former was just a legacy from MOC 1.0 where ASCII MOC was less formal, perhaps intended for hand-crafted MOCs not intended for serious use. With ASCII MOC now being formalized, can we remove that sentence and require that ASCII MOCs must be just as well-formed as their FITS counterparts?

-- TomDonaldson - 2019-06-14

As an ASCII MOC is often written by hand (in a Web form for instance, or as an Aladin script command), it was difficult to insure that it is always well formed at this level. Is it an ASCII MOC, yes, but potentially not well formed (ex: 3/1-210) . So the tools which have to manipulate such ASCII MOCs must check before used. At the opposite, FITS MOC is always generated by a code. So there is a real interest to force it to be stored well-formed for accelerating the future reloads.
That said, you right that it is a little bit inhomogeneous, and the ASCII MOCs will be used in other contexts that just for forms and scripts. So we can considere that an ASCII MOC written by hand is not - strickly speaking an ASCII MOC (not serialized in a file) - and in this consideration - extend the well-formed constraint to all serialized MOC formats (FITS & ASCII) by just removing the sentence.

-- PierreFernique - 2019-06-18

Tom answer 2019-06-21
That would be great. Thank you for doing that.




Comments from the IVOA Community during RFC/TCG review period: 2019-06-14 - 2019-07-29

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

  • Comment from Carlo Zwölf (CarloMariaZwoelf) : It is not clear for me in this proposed recommendation what comes natively from the Healpix definition and what is the IVOA-specific layer. Is there a way for highlighting this difference in the text?

    This new document is a very minor modification of the MOC 1.0 REC to standardize the ASCII MOC so that it can be used as quickly as possible in other IVOA standards, including VOregistry. The original text was not modified voluntarily in order to quickly reach a consensus. As your suggestion may imply a significant refactoring of the document, I propose to keep this point in mind for the next major release. It is clear that the introduction of the temporal MOC and the space-time MOC will involve a deep change in the document, especially in the direction you have suggested.
    -- PierreFernique - 2019-09-03



Comments from TCG member during the RFC/TCG Review Period: 2019-06-14 - 2019-07-29

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

MOC specification changes to bring String serialization to the normative part looks a good addition to this standard. The document itself is fine in general. I have two small things I'd like to point out.

  • letting [space,CR,LF,comma], multiples/mixes of them, as the elements separator has a human readability effect (see email), not so alerting, but also requires some code in MOC usage. Isn't it really possible to force a bit this serialization to be more smooth with, at least, single and homogeneous separators throughout a single ASCII MOC? Could it be at least warmly suggested?

    The flexibility of the ASCII MOC has been driven by three main pragmatic reasons:
    1- this syntax was exactly the suggested syntax in MOC1.0.
    => consequently there are already existing ASCII MOCs following this syntax,
    => It was easier to keep the syntax as it was suggested in MOC 1.0
    2- an ASCII MOC is often written by hand (in a Web form for instance, or as an Aladin script command),
    => a certain level of tolerance may be useful to avoid rejecting MOC with double space separators...
    3- an ASCII MOC can be a very very long string line
    => As for BASE64 ASCII streaming, it may be really problematic to read/edit an ASCII MOC in a regular ASCII editor without the CR/LF separator and multiple space for possible indentation. Also, we will have serious potential problems for reading and writting a so long line due to the classical flush mecanism of ASCII libs.

    The two first points are managable even if we really want to reduce the flexibility of the ASCII MOC syntax. Concretely the MOC clients/libs will have to be smart enough to support proto ASCII MOCs and possible separator typos (same argument that Tom's point). Quite reasonnable in fact. But I'm strongly in favor to take into account the 3nd point and keep the possibility to split an ASCII MOC if required

    My conclusion would be to remove the "comma", and let space,CR,LF possibly multiple
    -- PierreFernique - 2019-09-03

  • MOCORDER is a MUST and clearly described in the FITS serialization, while in the String (and JSON) one(s) it is less clear how to grab that piece of information. This because it is only said to be a must "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order" (in which case it would be the last token of the String, followed by a slash and no npix). I agree that if it's not so it's implicit the MOCORDER is the last/highest order available, but I'd prefer it to be explicit.

    You are right. It should be a "MUST" without ambiguity.
    The current sentence for ASCII MOC (section 2.3.2) is : "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order, the MOCORDER must be provided, followed by a slash ("/") without any associated npix value."

    I suggest this new sentence: "The best HEALPix resolution of the MOC (MOCORDER) MUST be always provided, even if there is no explicit npix value at this best resolution (due to the MOC hierarchization process). In the latter case, the MOCORDER is specified followed by a slash ("/") without any associated npix value."
    Concerning JSON serialization (suggested syntax in a non normative section). Here the proposed new sentence : "In the same way as for ASCII MOC serialization, if the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order, the MOCORDER should be provided with an empty npix list."
    -- PierreFernique - 2019-09-03
-- MarcoMolinaro - 2019-07-11

Data Model Working Group

  • I support the Marco's comment about MOCORDER in string.

    See below -- PierreFernique - 2019-09-03

  • It would be nice if figures had a caption and a number used to reference them.

    Good point. Concretely the few figures in this document are already fully described in the main text and the insertion of captions introduces not very well come repetitions (see section 1.3 for instance) without modifying deeply the text. As presented, this document would like to be as light as possible modification of the MOC 1.0 standard. So, I would propose to keep in mind this suggestion for the next major release of MOC. -- PierreFernique - 2019-09-03
-- LaurentMichel - 2019-07-25

Grid & Web Services Working Group

I support Marco suggestions and at this stage I have no further comments. Please add caption to figures and tables and reference them in the text.

-- GiulianoTaffoni - 2019-09-11

Registry Working Group

I would like to echo Marco's comments above, regarding simplifying the separators in serialization, and perhaps go a bit further. At best, there should be only one way to do this. I am willing to settle for consistency within a MOC using any one of those, no mix and match, with a preferred character separator suggested. -- TheresaDower - 2019-07-24

See below -- PierreFernique - 2019-09-03

Semantics Working Group

The Semantics WG doesn't see anything concerning us in the proposed changes (that otherwise sound reasonable and useful to us). The spurious indent of the rule for moc in the EBNF grammar could be removed, though.

-- MarkusDemleitner - 2019-07-16

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Operations

Mostly OK, but some comments:

  • Table of contents: the page numbers are mostly wrong.

    Right -> and fixed -- PierreFernique - 2019-09-03

  • Sec 1.4: The existence of a FITS HEALPix serialization format on which the MOC serialization format is based is claimed, but no reference is given. Since this text (written by Eric Hivon) does now just about exist in a public form https://healpix.sourceforge.io/data/examples/healpix_fits_specs.pdf, can it be referenced here?
  • Sec 2.3.1: The HEALPix FITS encoding standard is mentioned here as well: "As the FITS MOC is derived from a more generic standard dedicated to HEALPix maps, the FITS header must contain the following keywords/value." That's a bit misleading, since the MOC serialization is not a special case of the HEALPix encoding (it doesn't contain some of the keywords required by that standard, such as INDEXSCHM and NSIDE) and some of the keywords have values which are not allowed in the HEALPix encoding (ORDERING='NUNIQ'). Given the difference between the two encodings, I'm not sure there's much point making the comparison here, but if so I'd suggest something more like "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."

    I agree. The FITS HEALPix map serialization was and is unfortunately not a standard in the sense of IVOA REC. HEALPix authors have deployed libs and tools which follow an evolving convention. The recent Eric Hivon document has clarified this situation. Some FITS keywords which was required in previous conventions have not been kept, and some other have been introduced. So I totally agree Mark's point and I will take the suggestion "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."
    -- PierreFernique - 2019-09-03

  • Sec 2.3.1: The MOCID and ORIGIN keys both "suggest using IVOA identification like ..." where the examples are of the form ivo://... . If these are intended to be IVOA Identifiers, it should say something more like "suggest using an IVOA Identifier ..." with a reference to the IVOA Identifiers document, and the examples given should be valid IVOIDs (i.e. should resolve in the registry - I don't think that "CADC" is an authority in the IVOID sense). If not, the ivo:// prefix seems a bit misleading. Reg WG may want to comment on this.
    Right -> Modified following the suggestion => MOCID: Specify an identifier for the original data used to build the MOC. If the original data set has been declared in the VO registry [11], we strongly encourage to use the IVOA identifier (ex: “ivo://wfau.roe.ac.uk/twomass-dsa”')" and "ORIGIN: This keyword could be use to store the name of the MOC origin. If the authority of the origin has been declared in the VO registry, we strongly encourage to use the IVOA identifier (ex: “ivo://cadc.nrc.ca”)." -- PierreFernique - 2019-09-03

  • Sec 2.3.2: "hightnpix" -> "highnpix" ?
    ok -> PierreFernique - 2019-09-03

  • Sec 2.3.2: I'm inclined to agree with Tom's and Marco's comments about the form of the ASCII MOC serialization - it's not clear to me why FITS MOCs have to be well-formed and ASCII ones don't, and why so many separator choices are allowed. Also, the BNF doesn't constrain where the optional MOCORDER marker has to go, or how many of these appear. Consider constraining this format a bit further, e.g. require the MOCORDER marker to be present and/or at the end (or start?) of the string?

    See above -- PierreFernique - 2019-09-03

  • Appendix B: most of the decimal separators are "." characters, but one is a "," ("1,17").
    Ok -> PierreFernique - 2019-09-03
-- MarkTaylor - 2019-07-13

Standards and Processes Committee


TCG Vote: 2019-06-14 - 2019-07-29

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG *      
Apps *      
DAL *     -- MarcoMolinaro / JamesDempsey 2019-07-17
DM *     -- LaurentMichel - 2019-09-26
GWS *      
Registry *      
Semantics *      
DCP        
KDIG        
SSIG *      
Theory        
Changed:
<
<
TD        
>
>
TDIG *     -- AdaNebot - 2019-10-09
 
Ops *      
StdProc        



<--

-->
  • MOC_ASCII_in_Aladin.png:
    MOC_ASCII_in_Aladin.png

META FILEATTACHMENT attachment="MOC_ASCII_in_Aladin.png" attr="" comment="" date="1558081955" name="MOC_ASCII_in_Aladin.png" path="MOC_ASCII_in_Aladin.png" size="9507" user="PierreFernique" version="1"

Revision 242019-10-09 - BaptisteCecconi

 
META TOPICPARENT name="MocInfo"

MOC 1.1 Proposed Recommendation: Request for Comments


Summary

Latest Draft: MOC 1.1 (Proposed Recommendation)

The substantive differences between version 1.1 of MOC and the preceding version 1.0 are:

  • The String MOC serialization was moved from an informative section (suggested syntax) to the normative section.
  • A MOCORDER convention for String MOC and JSON MOC was added.

Reference Interoperable Implementations

(Indicate here the links to at least two Reference Interoperable Implementations)

  • MOCPy >=0.5.6
  • Aladin (>v10.126): read/write ASCII MOC (select ASCII format when exporting MOC - see snapshot below) + new script command dedicated to ASCII MOC (ex: draw MOC 3/1,2,3 4/56-67)
  • The relational registry at http://dc.g-vo.org/tap parses the ASCII MOCs in Registry records into the rr.stc_spatial table (consumer)
  • DaCHS since version 1.1 produces ASCII MOCs when serialising MOC-values database columns.
  • The Moc class within the AST/PyAST WCS library now supports reading and writing ASCII and JSON MOCs in addition to FITS.
To see non-trivial examples for ASCII MOCs coming out of a database, run something like

select top 100 * from rr.stc_spatial
where 1=contains(point(10, 10), coverage)
and 0=contains(point(20, 20), coverage)

on http://dc.g-vo.org/tap.

Implementations Validators

Comments Prior to Official RFC Period

As I was considering semantic validation cases, I noticed this sentence in section 2.3.2:

"Warning: In this basic simple ASCII string format the values may be not sorted, and the MOC may be not well-formed."

which seems to contradict section 2.2.3:

" An encoded MOC must also be “well-formed”. To keep the canonical property of MOC, the redundant cells are not allowed and the hierarchical encoding principle must be respected. Thus it is not allowed to encode 4 sibling cells instead of their parent (only at level 0 for the 12 first diamonds). Using this simple constraint, there is one and only one way to describe a specific region and this makes the interoperability principle easy to implement."

Maybe the former was just a legacy from MOC 1.0 where ASCII MOC was less formal, perhaps intended for hand-crafted MOCs not intended for serious use. With ASCII MOC now being formalized, can we remove that sentence and require that ASCII MOCs must be just as well-formed as their FITS counterparts?

-- TomDonaldson - 2019-06-14

As an ASCII MOC is often written by hand (in a Web form for instance, or as an Aladin script command), it was difficult to insure that it is always well formed at this level. Is it an ASCII MOC, yes, but potentially not well formed (ex: 3/1-210) . So the tools which have to manipulate such ASCII MOCs must check before used. At the opposite, FITS MOC is always generated by a code. So there is a real interest to force it to be stored well-formed for accelerating the future reloads.
That said, you right that it is a little bit inhomogeneous, and the ASCII MOCs will be used in other contexts that just for forms and scripts. So we can considere that an ASCII MOC written by hand is not - strickly speaking an ASCII MOC (not serialized in a file) - and in this consideration - extend the well-formed constraint to all serialized MOC formats (FITS & ASCII) by just removing the sentence.

-- PierreFernique - 2019-06-18

Tom answer 2019-06-21
That would be great. Thank you for doing that.




Comments from the IVOA Community during RFC/TCG review period: 2019-06-14 - 2019-07-29

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

  • Comment from Carlo Zwölf (CarloMariaZwoelf) : It is not clear for me in this proposed recommendation what comes natively from the Healpix definition and what is the IVOA-specific layer. Is there a way for highlighting this difference in the text?

    This new document is a very minor modification of the MOC 1.0 REC to standardize the ASCII MOC so that it can be used as quickly as possible in other IVOA standards, including VOregistry. The original text was not modified voluntarily in order to quickly reach a consensus. As your suggestion may imply a significant refactoring of the document, I propose to keep this point in mind for the next major release. It is clear that the introduction of the temporal MOC and the space-time MOC will involve a deep change in the document, especially in the direction you have suggested.
    -- PierreFernique - 2019-09-03



Comments from TCG member during the RFC/TCG Review Period: 2019-06-14 - 2019-07-29

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

MOC specification changes to bring String serialization to the normative part looks a good addition to this standard. The document itself is fine in general. I have two small things I'd like to point out.

  • letting [space,CR,LF,comma], multiples/mixes of them, as the elements separator has a human readability effect (see email), not so alerting, but also requires some code in MOC usage. Isn't it really possible to force a bit this serialization to be more smooth with, at least, single and homogeneous separators throughout a single ASCII MOC? Could it be at least warmly suggested?

    The flexibility of the ASCII MOC has been driven by three main pragmatic reasons:
    1- this syntax was exactly the suggested syntax in MOC1.0.
    => consequently there are already existing ASCII MOCs following this syntax,
    => It was easier to keep the syntax as it was suggested in MOC 1.0
    2- an ASCII MOC is often written by hand (in a Web form for instance, or as an Aladin script command),
    => a certain level of tolerance may be useful to avoid rejecting MOC with double space separators...
    3- an ASCII MOC can be a very very long string line
    => As for BASE64 ASCII streaming, it may be really problematic to read/edit an ASCII MOC in a regular ASCII editor without the CR/LF separator and multiple space for possible indentation. Also, we will have serious potential problems for reading and writting a so long line due to the classical flush mecanism of ASCII libs.

    The two first points are managable even if we really want to reduce the flexibility of the ASCII MOC syntax. Concretely the MOC clients/libs will have to be smart enough to support proto ASCII MOCs and possible separator typos (same argument that Tom's point). Quite reasonnable in fact. But I'm strongly in favor to take into account the 3nd point and keep the possibility to split an ASCII MOC if required

    My conclusion would be to remove the "comma", and let space,CR,LF possibly multiple
    -- PierreFernique - 2019-09-03

  • MOCORDER is a MUST and clearly described in the FITS serialization, while in the String (and JSON) one(s) it is less clear how to grab that piece of information. This because it is only said to be a must "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order" (in which case it would be the last token of the String, followed by a slash and no npix). I agree that if it's not so it's implicit the MOCORDER is the last/highest order available, but I'd prefer it to be explicit.

    You are right. It should be a "MUST" without ambiguity.
    The current sentence for ASCII MOC (section 2.3.2) is : "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order, the MOCORDER must be provided, followed by a slash ("/") without any associated npix value."

    I suggest this new sentence: "The best HEALPix resolution of the MOC (MOCORDER) MUST be always provided, even if there is no explicit npix value at this best resolution (due to the MOC hierarchization process). In the latter case, the MOCORDER is specified followed by a slash ("/") without any associated npix value."
    Concerning JSON serialization (suggested syntax in a non normative section). Here the proposed new sentence : "In the same way as for ASCII MOC serialization, if the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order, the MOCORDER should be provided with an empty npix list."
    -- PierreFernique - 2019-09-03
-- MarcoMolinaro - 2019-07-11

Data Model Working Group

  • I support the Marco's comment about MOCORDER in string.

    See below -- PierreFernique - 2019-09-03

  • It would be nice if figures had a caption and a number used to reference them.

    Good point. Concretely the few figures in this document are already fully described in the main text and the insertion of captions introduces not very well come repetitions (see section 1.3 for instance) without modifying deeply the text. As presented, this document would like to be as light as possible modification of the MOC 1.0 standard. So, I would propose to keep in mind this suggestion for the next major release of MOC. -- PierreFernique - 2019-09-03
-- LaurentMichel - 2019-07-25

Grid & Web Services Working Group

I support Marco suggestions and at this stage I have no further comments. Please add caption to figures and tables and reference them in the text.

-- GiulianoTaffoni - 2019-09-11

Registry Working Group

I would like to echo Marco's comments above, regarding simplifying the separators in serialization, and perhaps go a bit further. At best, there should be only one way to do this. I am willing to settle for consistency within a MOC using any one of those, no mix and match, with a preferred character separator suggested. -- TheresaDower - 2019-07-24

See below -- PierreFernique - 2019-09-03

Semantics Working Group

The Semantics WG doesn't see anything concerning us in the proposed changes (that otherwise sound reasonable and useful to us). The spurious indent of the rule for moc in the EBNF grammar could be removed, though.

-- MarkusDemleitner - 2019-07-16

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Operations

Mostly OK, but some comments:

  • Table of contents: the page numbers are mostly wrong.

    Right -> and fixed -- PierreFernique - 2019-09-03

  • Sec 1.4: The existence of a FITS HEALPix serialization format on which the MOC serialization format is based is claimed, but no reference is given. Since this text (written by Eric Hivon) does now just about exist in a public form https://healpix.sourceforge.io/data/examples/healpix_fits_specs.pdf, can it be referenced here?
  • Sec 2.3.1: The HEALPix FITS encoding standard is mentioned here as well: "As the FITS MOC is derived from a more generic standard dedicated to HEALPix maps, the FITS header must contain the following keywords/value." That's a bit misleading, since the MOC serialization is not a special case of the HEALPix encoding (it doesn't contain some of the keywords required by that standard, such as INDEXSCHM and NSIDE) and some of the keywords have values which are not allowed in the HEALPix encoding (ORDERING='NUNIQ'). Given the difference between the two encodings, I'm not sure there's much point making the comparison here, but if so I'd suggest something more like "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."

    I agree. The FITS HEALPix map serialization was and is unfortunately not a standard in the sense of IVOA REC. HEALPix authors have deployed libs and tools which follow an evolving convention. The recent Eric Hivon document has clarified this situation. Some FITS keywords which was required in previous conventions have not been kept, and some other have been introduced. So I totally agree Mark's point and I will take the suggestion "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."
    -- PierreFernique - 2019-09-03

  • Sec 2.3.1: The MOCID and ORIGIN keys both "suggest using IVOA identification like ..." where the examples are of the form ivo://... . If these are intended to be IVOA Identifiers, it should say something more like "suggest using an IVOA Identifier ..." with a reference to the IVOA Identifiers document, and the examples given should be valid IVOIDs (i.e. should resolve in the registry - I don't think that "CADC" is an authority in the IVOID sense). If not, the ivo:// prefix seems a bit misleading. Reg WG may want to comment on this.
    Right -> Modified following the suggestion => MOCID: Specify an identifier for the original data used to build the MOC. If the original data set has been declared in the VO registry [11], we strongly encourage to use the IVOA identifier (ex: “ivo://wfau.roe.ac.uk/twomass-dsa”')" and "ORIGIN: This keyword could be use to store the name of the MOC origin. If the authority of the origin has been declared in the VO registry, we strongly encourage to use the IVOA identifier (ex: “ivo://cadc.nrc.ca”)." -- PierreFernique - 2019-09-03

  • Sec 2.3.2: "hightnpix" -> "highnpix" ?
    ok -> PierreFernique - 2019-09-03

  • Sec 2.3.2: I'm inclined to agree with Tom's and Marco's comments about the form of the ASCII MOC serialization - it's not clear to me why FITS MOCs have to be well-formed and ASCII ones don't, and why so many separator choices are allowed. Also, the BNF doesn't constrain where the optional MOCORDER marker has to go, or how many of these appear. Consider constraining this format a bit further, e.g. require the MOCORDER marker to be present and/or at the end (or start?) of the string?

    See above -- PierreFernique - 2019-09-03

  • Appendix B: most of the decimal separators are "." characters, but one is a "," ("1,17").
    Ok -> PierreFernique - 2019-09-03
-- MarkTaylor - 2019-07-13

Standards and Processes Committee


TCG Vote: 2019-06-14 - 2019-07-29

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG *      
Apps *      
DAL *     -- MarcoMolinaro / JamesDempsey 2019-07-17
DM *     -- LaurentMichel - 2019-09-26
GWS *      
Registry *      
Semantics *      
DCP        
KDIG        
Changed:
<
<
SSIG        
>
>
SSIG *      
 
Theory        
TD        
Ops *      
StdProc        



<--

-->
  • MOC_ASCII_in_Aladin.png:
    MOC_ASCII_in_Aladin.png

META FILEATTACHMENT attachment="MOC_ASCII_in_Aladin.png" attr="" comment="" date="1558081955" name="MOC_ASCII_in_Aladin.png" path="MOC_ASCII_in_Aladin.png" size="9507" user="PierreFernique" version="1"

Revision 232019-09-27 - PatrickDowler

 
META TOPICPARENT name="MocInfo"

MOC 1.1 Proposed Recommendation: Request for Comments


Summary

Latest Draft: MOC 1.1 (Proposed Recommendation)

The substantive differences between version 1.1 of MOC and the preceding version 1.0 are:

  • The String MOC serialization was moved from an informative section (suggested syntax) to the normative section.
  • A MOCORDER convention for String MOC and JSON MOC was added.

Reference Interoperable Implementations

(Indicate here the links to at least two Reference Interoperable Implementations)

  • MOCPy >=0.5.6
  • Aladin (>v10.126): read/write ASCII MOC (select ASCII format when exporting MOC - see snapshot below) + new script command dedicated to ASCII MOC (ex: draw MOC 3/1,2,3 4/56-67)
  • The relational registry at http://dc.g-vo.org/tap parses the ASCII MOCs in Registry records into the rr.stc_spatial table (consumer)
  • DaCHS since version 1.1 produces ASCII MOCs when serialising MOC-values database columns.
  • The Moc class within the AST/PyAST WCS library now supports reading and writing ASCII and JSON MOCs in addition to FITS.
To see non-trivial examples for ASCII MOCs coming out of a database, run something like

select top 100 * from rr.stc_spatial
where 1=contains(point(10, 10), coverage)
and 0=contains(point(20, 20), coverage)

on http://dc.g-vo.org/tap.

Implementations Validators

Comments Prior to Official RFC Period

As I was considering semantic validation cases, I noticed this sentence in section 2.3.2:

"Warning: In this basic simple ASCII string format the values may be not sorted, and the MOC may be not well-formed."

which seems to contradict section 2.2.3:

" An encoded MOC must also be “well-formed”. To keep the canonical property of MOC, the redundant cells are not allowed and the hierarchical encoding principle must be respected. Thus it is not allowed to encode 4 sibling cells instead of their parent (only at level 0 for the 12 first diamonds). Using this simple constraint, there is one and only one way to describe a specific region and this makes the interoperability principle easy to implement."

Maybe the former was just a legacy from MOC 1.0 where ASCII MOC was less formal, perhaps intended for hand-crafted MOCs not intended for serious use. With ASCII MOC now being formalized, can we remove that sentence and require that ASCII MOCs must be just as well-formed as their FITS counterparts?

-- TomDonaldson - 2019-06-14

As an ASCII MOC is often written by hand (in a Web form for instance, or as an Aladin script command), it was difficult to insure that it is always well formed at this level. Is it an ASCII MOC, yes, but potentially not well formed (ex: 3/1-210) . So the tools which have to manipulate such ASCII MOCs must check before used. At the opposite, FITS MOC is always generated by a code. So there is a real interest to force it to be stored well-formed for accelerating the future reloads.
That said, you right that it is a little bit inhomogeneous, and the ASCII MOCs will be used in other contexts that just for forms and scripts. So we can considere that an ASCII MOC written by hand is not - strickly speaking an ASCII MOC (not serialized in a file) - and in this consideration - extend the well-formed constraint to all serialized MOC formats (FITS & ASCII) by just removing the sentence.

-- PierreFernique - 2019-06-18

Tom answer 2019-06-21
That would be great. Thank you for doing that.




Comments from the IVOA Community during RFC/TCG review period: 2019-06-14 - 2019-07-29

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

  • Comment from Carlo Zwölf (CarloMariaZwoelf) : It is not clear for me in this proposed recommendation what comes natively from the Healpix definition and what is the IVOA-specific layer. Is there a way for highlighting this difference in the text?

    This new document is a very minor modification of the MOC 1.0 REC to standardize the ASCII MOC so that it can be used as quickly as possible in other IVOA standards, including VOregistry. The original text was not modified voluntarily in order to quickly reach a consensus. As your suggestion may imply a significant refactoring of the document, I propose to keep this point in mind for the next major release. It is clear that the introduction of the temporal MOC and the space-time MOC will involve a deep change in the document, especially in the direction you have suggested.
    -- PierreFernique - 2019-09-03



Comments from TCG member during the RFC/TCG Review Period: 2019-06-14 - 2019-07-29

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

MOC specification changes to bring String serialization to the normative part looks a good addition to this standard. The document itself is fine in general. I have two small things I'd like to point out.

  • letting [space,CR,LF,comma], multiples/mixes of them, as the elements separator has a human readability effect (see email), not so alerting, but also requires some code in MOC usage. Isn't it really possible to force a bit this serialization to be more smooth with, at least, single and homogeneous separators throughout a single ASCII MOC? Could it be at least warmly suggested?

    The flexibility of the ASCII MOC has been driven by three main pragmatic reasons:
    1- this syntax was exactly the suggested syntax in MOC1.0.
    => consequently there are already existing ASCII MOCs following this syntax,
    => It was easier to keep the syntax as it was suggested in MOC 1.0
    2- an ASCII MOC is often written by hand (in a Web form for instance, or as an Aladin script command),
    => a certain level of tolerance may be useful to avoid rejecting MOC with double space separators...
    3- an ASCII MOC can be a very very long string line
    => As for BASE64 ASCII streaming, it may be really problematic to read/edit an ASCII MOC in a regular ASCII editor without the CR/LF separator and multiple space for possible indentation. Also, we will have serious potential problems for reading and writting a so long line due to the classical flush mecanism of ASCII libs.

    The two first points are managable even if we really want to reduce the flexibility of the ASCII MOC syntax. Concretely the MOC clients/libs will have to be smart enough to support proto ASCII MOCs and possible separator typos (same argument that Tom's point). Quite reasonnable in fact. But I'm strongly in favor to take into account the 3nd point and keep the possibility to split an ASCII MOC if required

    My conclusion would be to remove the "comma", and let space,CR,LF possibly multiple
    -- PierreFernique - 2019-09-03

  • MOCORDER is a MUST and clearly described in the FITS serialization, while in the String (and JSON) one(s) it is less clear how to grab that piece of information. This because it is only said to be a must "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order" (in which case it would be the last token of the String, followed by a slash and no npix). I agree that if it's not so it's implicit the MOCORDER is the last/highest order available, but I'd prefer it to be explicit.

    You are right. It should be a "MUST" without ambiguity.
    The current sentence for ASCII MOC (section 2.3.2) is : "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order, the MOCORDER must be provided, followed by a slash ("/") without any associated npix value."

    I suggest this new sentence: "The best HEALPix resolution of the MOC (MOCORDER) MUST be always provided, even if there is no explicit npix value at this best resolution (due to the MOC hierarchization process). In the latter case, the MOCORDER is specified followed by a slash ("/") without any associated npix value."
    Concerning JSON serialization (suggested syntax in a non normative section). Here the proposed new sentence : "In the same way as for ASCII MOC serialization, if the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order, the MOCORDER should be provided with an empty npix list."
    -- PierreFernique - 2019-09-03
-- MarcoMolinaro - 2019-07-11

Data Model Working Group

  • I support the Marco's comment about MOCORDER in string.

    See below -- PierreFernique - 2019-09-03

  • It would be nice if figures had a caption and a number used to reference them.

    Good point. Concretely the few figures in this document are already fully described in the main text and the insertion of captions introduces not very well come repetitions (see section 1.3 for instance) without modifying deeply the text. As presented, this document would like to be as light as possible modification of the MOC 1.0 standard. So, I would propose to keep in mind this suggestion for the next major release of MOC. -- PierreFernique - 2019-09-03
-- LaurentMichel - 2019-07-25

Grid & Web Services Working Group

I support Marco suggestions and at this stage I have no further comments. Please add caption to figures and tables and reference them in the text.

-- GiulianoTaffoni - 2019-09-11

Registry Working Group

I would like to echo Marco's comments above, regarding simplifying the separators in serialization, and perhaps go a bit further. At best, there should be only one way to do this. I am willing to settle for consistency within a MOC using any one of those, no mix and match, with a preferred character separator suggested. -- TheresaDower - 2019-07-24

See below -- PierreFernique - 2019-09-03

Semantics Working Group

The Semantics WG doesn't see anything concerning us in the proposed changes (that otherwise sound reasonable and useful to us). The spurious indent of the rule for moc in the EBNF grammar could be removed, though.

-- MarkusDemleitner - 2019-07-16

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Operations

Mostly OK, but some comments:

  • Table of contents: the page numbers are mostly wrong.

    Right -> and fixed -- PierreFernique - 2019-09-03

  • Sec 1.4: The existence of a FITS HEALPix serialization format on which the MOC serialization format is based is claimed, but no reference is given. Since this text (written by Eric Hivon) does now just about exist in a public form https://healpix.sourceforge.io/data/examples/healpix_fits_specs.pdf, can it be referenced here?
  • Sec 2.3.1: The HEALPix FITS encoding standard is mentioned here as well: "As the FITS MOC is derived from a more generic standard dedicated to HEALPix maps, the FITS header must contain the following keywords/value." That's a bit misleading, since the MOC serialization is not a special case of the HEALPix encoding (it doesn't contain some of the keywords required by that standard, such as INDEXSCHM and NSIDE) and some of the keywords have values which are not allowed in the HEALPix encoding (ORDERING='NUNIQ'). Given the difference between the two encodings, I'm not sure there's much point making the comparison here, but if so I'd suggest something more like "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."

    I agree. The FITS HEALPix map serialization was and is unfortunately not a standard in the sense of IVOA REC. HEALPix authors have deployed libs and tools which follow an evolving convention. The recent Eric Hivon document has clarified this situation. Some FITS keywords which was required in previous conventions have not been kept, and some other have been introduced. So I totally agree Mark's point and I will take the suggestion "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."
    -- PierreFernique - 2019-09-03

  • Sec 2.3.1: The MOCID and ORIGIN keys both "suggest using IVOA identification like ..." where the examples are of the form ivo://... . If these are intended to be IVOA Identifiers, it should say something more like "suggest using an IVOA Identifier ..." with a reference to the IVOA Identifiers document, and the examples given should be valid IVOIDs (i.e. should resolve in the registry - I don't think that "CADC" is an authority in the IVOID sense). If not, the ivo:// prefix seems a bit misleading. Reg WG may want to comment on this.
    Right -> Modified following the suggestion => MOCID: Specify an identifier for the original data used to build the MOC. If the original data set has been declared in the VO registry [11], we strongly encourage to use the IVOA identifier (ex: “ivo://wfau.roe.ac.uk/twomass-dsa”')" and "ORIGIN: This keyword could be use to store the name of the MOC origin. If the authority of the origin has been declared in the VO registry, we strongly encourage to use the IVOA identifier (ex: “ivo://cadc.nrc.ca”)." -- PierreFernique - 2019-09-03

  • Sec 2.3.2: "hightnpix" -> "highnpix" ?
    ok -> PierreFernique - 2019-09-03

  • Sec 2.3.2: I'm inclined to agree with Tom's and Marco's comments about the form of the ASCII MOC serialization - it's not clear to me why FITS MOCs have to be well-formed and ASCII ones don't, and why so many separator choices are allowed. Also, the BNF doesn't constrain where the optional MOCORDER marker has to go, or how many of these appear. Consider constraining this format a bit further, e.g. require the MOCORDER marker to be present and/or at the end (or start?) of the string?

    See above -- PierreFernique - 2019-09-03

  • Appendix B: most of the decimal separators are "." characters, but one is a "," ("1,17").
    Ok -> PierreFernique - 2019-09-03
-- MarkTaylor - 2019-07-13

Standards and Processes Committee


TCG Vote: 2019-06-14 - 2019-07-29

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
Changed:
<
<
TCG        
>
>
TCG *      
 
Apps *      
DAL *     -- MarcoMolinaro / JamesDempsey 2019-07-17
DM *     -- LaurentMichel - 2019-09-26
GWS *      
Registry *      
Semantics *      
DCP        
KDIG        
SSIG        
Theory        
TD        
Ops *      
StdProc        



<--

-->
  • MOC_ASCII_in_Aladin.png:
    MOC_ASCII_in_Aladin.png

META FILEATTACHMENT attachment="MOC_ASCII_in_Aladin.png" attr="" comment="" date="1558081955" name="MOC_ASCII_in_Aladin.png" path="MOC_ASCII_in_Aladin.png" size="9507" user="PierreFernique" version="1"

Revision 222019-09-26 - TheresaDower

 
META TOPICPARENT name="MocInfo"

MOC 1.1 Proposed Recommendation: Request for Comments


Summary

Latest Draft: MOC 1.1 (Proposed Recommendation)

The substantive differences between version 1.1 of MOC and the preceding version 1.0 are:

  • The String MOC serialization was moved from an informative section (suggested syntax) to the normative section.
  • A MOCORDER convention for String MOC and JSON MOC was added.

Reference Interoperable Implementations

(Indicate here the links to at least two Reference Interoperable Implementations)

  • MOCPy >=0.5.6
  • Aladin (>v10.126): read/write ASCII MOC (select ASCII format when exporting MOC - see snapshot below) + new script command dedicated to ASCII MOC (ex: draw MOC 3/1,2,3 4/56-67)
  • The relational registry at http://dc.g-vo.org/tap parses the ASCII MOCs in Registry records into the rr.stc_spatial table (consumer)
  • DaCHS since version 1.1 produces ASCII MOCs when serialising MOC-values database columns.
  • The Moc class within the AST/PyAST WCS library now supports reading and writing ASCII and JSON MOCs in addition to FITS.
To see non-trivial examples for ASCII MOCs coming out of a database, run something like

select top 100 * from rr.stc_spatial
where 1=contains(point(10, 10), coverage)
and 0=contains(point(20, 20), coverage)

on http://dc.g-vo.org/tap.

Implementations Validators

Comments Prior to Official RFC Period

As I was considering semantic validation cases, I noticed this sentence in section 2.3.2:

"Warning: In this basic simple ASCII string format the values may be not sorted, and the MOC may be not well-formed."

which seems to contradict section 2.2.3:

" An encoded MOC must also be “well-formed”. To keep the canonical property of MOC, the redundant cells are not allowed and the hierarchical encoding principle must be respected. Thus it is not allowed to encode 4 sibling cells instead of their parent (only at level 0 for the 12 first diamonds). Using this simple constraint, there is one and only one way to describe a specific region and this makes the interoperability principle easy to implement."

Maybe the former was just a legacy from MOC 1.0 where ASCII MOC was less formal, perhaps intended for hand-crafted MOCs not intended for serious use. With ASCII MOC now being formalized, can we remove that sentence and require that ASCII MOCs must be just as well-formed as their FITS counterparts?

-- TomDonaldson - 2019-06-14

As an ASCII MOC is often written by hand (in a Web form for instance, or as an Aladin script command), it was difficult to insure that it is always well formed at this level. Is it an ASCII MOC, yes, but potentially not well formed (ex: 3/1-210) . So the tools which have to manipulate such ASCII MOCs must check before used. At the opposite, FITS MOC is always generated by a code. So there is a real interest to force it to be stored well-formed for accelerating the future reloads.
That said, you right that it is a little bit inhomogeneous, and the ASCII MOCs will be used in other contexts that just for forms and scripts. So we can considere that an ASCII MOC written by hand is not - strickly speaking an ASCII MOC (not serialized in a file) - and in this consideration - extend the well-formed constraint to all serialized MOC formats (FITS & ASCII) by just removing the sentence.

-- PierreFernique - 2019-06-18

Tom answer 2019-06-21
That would be great. Thank you for doing that.




Comments from the IVOA Community during RFC/TCG review period: 2019-06-14 - 2019-07-29

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

  • Comment from Carlo Zwölf (CarloMariaZwoelf) : It is not clear for me in this proposed recommendation what comes natively from the Healpix definition and what is the IVOA-specific layer. Is there a way for highlighting this difference in the text?

    This new document is a very minor modification of the MOC 1.0 REC to standardize the ASCII MOC so that it can be used as quickly as possible in other IVOA standards, including VOregistry. The original text was not modified voluntarily in order to quickly reach a consensus. As your suggestion may imply a significant refactoring of the document, I propose to keep this point in mind for the next major release. It is clear that the introduction of the temporal MOC and the space-time MOC will involve a deep change in the document, especially in the direction you have suggested.
    -- PierreFernique - 2019-09-03



Comments from TCG member during the RFC/TCG Review Period: 2019-06-14 - 2019-07-29

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

MOC specification changes to bring String serialization to the normative part looks a good addition to this standard. The document itself is fine in general. I have two small things I'd like to point out.

  • letting [space,CR,LF,comma], multiples/mixes of them, as the elements separator has a human readability effect (see email), not so alerting, but also requires some code in MOC usage. Isn't it really possible to force a bit this serialization to be more smooth with, at least, single and homogeneous separators throughout a single ASCII MOC? Could it be at least warmly suggested?

    The flexibility of the ASCII MOC has been driven by three main pragmatic reasons:
    1- this syntax was exactly the suggested syntax in MOC1.0.
    => consequently there are already existing ASCII MOCs following this syntax,
    => It was easier to keep the syntax as it was suggested in MOC 1.0
    2- an ASCII MOC is often written by hand (in a Web form for instance, or as an Aladin script command),
    => a certain level of tolerance may be useful to avoid rejecting MOC with double space separators...
    3- an ASCII MOC can be a very very long string line
    => As for BASE64 ASCII streaming, it may be really problematic to read/edit an ASCII MOC in a regular ASCII editor without the CR/LF separator and multiple space for possible indentation. Also, we will have serious potential problems for reading and writting a so long line due to the classical flush mecanism of ASCII libs.

    The two first points are managable even if we really want to reduce the flexibility of the ASCII MOC syntax. Concretely the MOC clients/libs will have to be smart enough to support proto ASCII MOCs and possible separator typos (same argument that Tom's point). Quite reasonnable in fact. But I'm strongly in favor to take into account the 3nd point and keep the possibility to split an ASCII MOC if required

    My conclusion would be to remove the "comma", and let space,CR,LF possibly multiple
    -- PierreFernique - 2019-09-03

  • MOCORDER is a MUST and clearly described in the FITS serialization, while in the String (and JSON) one(s) it is less clear how to grab that piece of information. This because it is only said to be a must "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order" (in which case it would be the last token of the String, followed by a slash and no npix). I agree that if it's not so it's implicit the MOCORDER is the last/highest order available, but I'd prefer it to be explicit.

    You are right. It should be a "MUST" without ambiguity.
    The current sentence for ASCII MOC (section 2.3.2) is : "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order, the MOCORDER must be provided, followed by a slash ("/") without any associated npix value."

    I suggest this new sentence: "The best HEALPix resolution of the MOC (MOCORDER) MUST be always provided, even if there is no explicit npix value at this best resolution (due to the MOC hierarchization process). In the latter case, the MOCORDER is specified followed by a slash ("/") without any associated npix value."
    Concerning JSON serialization (suggested syntax in a non normative section). Here the proposed new sentence : "In the same way as for ASCII MOC serialization, if the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order, the MOCORDER should be provided with an empty npix list."
    -- PierreFernique - 2019-09-03
-- MarcoMolinaro - 2019-07-11

Data Model Working Group

  • I support the Marco's comment about MOCORDER in string.

    See below -- PierreFernique - 2019-09-03

  • It would be nice if figures had a caption and a number used to reference them.

    Good point. Concretely the few figures in this document are already fully described in the main text and the insertion of captions introduces not very well come repetitions (see section 1.3 for instance) without modifying deeply the text. As presented, this document would like to be as light as possible modification of the MOC 1.0 standard. So, I would propose to keep in mind this suggestion for the next major release of MOC. -- PierreFernique - 2019-09-03
-- LaurentMichel - 2019-07-25

Grid & Web Services Working Group

I support Marco suggestions and at this stage I have no further comments. Please add caption to figures and tables and reference them in the text.

-- GiulianoTaffoni - 2019-09-11

Registry Working Group

I would like to echo Marco's comments above, regarding simplifying the separators in serialization, and perhaps go a bit further. At best, there should be only one way to do this. I am willing to settle for consistency within a MOC using any one of those, no mix and match, with a preferred character separator suggested. -- TheresaDower - 2019-07-24

See below -- PierreFernique - 2019-09-03

Semantics Working Group

The Semantics WG doesn't see anything concerning us in the proposed changes (that otherwise sound reasonable and useful to us). The spurious indent of the rule for moc in the EBNF grammar could be removed, though.

-- MarkusDemleitner - 2019-07-16

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Operations

Mostly OK, but some comments:

  • Table of contents: the page numbers are mostly wrong.

    Right -> and fixed -- PierreFernique - 2019-09-03

  • Sec 1.4: The existence of a FITS HEALPix serialization format on which the MOC serialization format is based is claimed, but no reference is given. Since this text (written by Eric Hivon) does now just about exist in a public form https://healpix.sourceforge.io/data/examples/healpix_fits_specs.pdf, can it be referenced here?
  • Sec 2.3.1: The HEALPix FITS encoding standard is mentioned here as well: "As the FITS MOC is derived from a more generic standard dedicated to HEALPix maps, the FITS header must contain the following keywords/value." That's a bit misleading, since the MOC serialization is not a special case of the HEALPix encoding (it doesn't contain some of the keywords required by that standard, such as INDEXSCHM and NSIDE) and some of the keywords have values which are not allowed in the HEALPix encoding (ORDERING='NUNIQ'). Given the difference between the two encodings, I'm not sure there's much point making the comparison here, but if so I'd suggest something more like "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."

    I agree. The FITS HEALPix map serialization was and is unfortunately not a standard in the sense of IVOA REC. HEALPix authors have deployed libs and tools which follow an evolving convention. The recent Eric Hivon document has clarified this situation. Some FITS keywords which was required in previous conventions have not been kept, and some other have been introduced. So I totally agree Mark's point and I will take the suggestion "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."
    -- PierreFernique - 2019-09-03

  • Sec 2.3.1: The MOCID and ORIGIN keys both "suggest using IVOA identification like ..." where the examples are of the form ivo://... . If these are intended to be IVOA Identifiers, it should say something more like "suggest using an IVOA Identifier ..." with a reference to the IVOA Identifiers document, and the examples given should be valid IVOIDs (i.e. should resolve in the registry - I don't think that "CADC" is an authority in the IVOID sense). If not, the ivo:// prefix seems a bit misleading. Reg WG may want to comment on this.
    Right -> Modified following the suggestion => MOCID: Specify an identifier for the original data used to build the MOC. If the original data set has been declared in the VO registry [11], we strongly encourage to use the IVOA identifier (ex: “ivo://wfau.roe.ac.uk/twomass-dsa”')" and "ORIGIN: This keyword could be use to store the name of the MOC origin. If the authority of the origin has been declared in the VO registry, we strongly encourage to use the IVOA identifier (ex: “ivo://cadc.nrc.ca”)." -- PierreFernique - 2019-09-03

  • Sec 2.3.2: "hightnpix" -> "highnpix" ?
    ok -> PierreFernique - 2019-09-03

  • Sec 2.3.2: I'm inclined to agree with Tom's and Marco's comments about the form of the ASCII MOC serialization - it's not clear to me why FITS MOCs have to be well-formed and ASCII ones don't, and why so many separator choices are allowed. Also, the BNF doesn't constrain where the optional MOCORDER marker has to go, or how many of these appear. Consider constraining this format a bit further, e.g. require the MOCORDER marker to be present and/or at the end (or start?) of the string?

    See above -- PierreFernique - 2019-09-03

  • Appendix B: most of the decimal separators are "." characters, but one is a "," ("1,17").
    Ok -> PierreFernique - 2019-09-03
-- MarkTaylor - 2019-07-13

Standards and Processes Committee


TCG Vote: 2019-06-14 - 2019-07-29

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps *      
DAL *     -- MarcoMolinaro / JamesDempsey 2019-07-17
DM *     -- LaurentMichel - 2019-09-26
GWS *      
Changed:
<
<
Registry        
>
>
Registry *      
 
Semantics *      
DCP        
KDIG        
SSIG        
Theory        
TD        
Ops *      
StdProc        



<--

-->
  • MOC_ASCII_in_Aladin.png:
    MOC_ASCII_in_Aladin.png

META FILEATTACHMENT attachment="MOC_ASCII_in_Aladin.png" attr="" comment="" date="1558081955" name="MOC_ASCII_in_Aladin.png" path="MOC_ASCII_in_Aladin.png" size="9507" user="PierreFernique" version="1"

Revision 212019-09-26 - LaurentMichel

 
META TOPICPARENT name="MocInfo"

MOC 1.1 Proposed Recommendation: Request for Comments


Summary

Latest Draft: MOC 1.1 (Proposed Recommendation)

The substantive differences between version 1.1 of MOC and the preceding version 1.0 are:

  • The String MOC serialization was moved from an informative section (suggested syntax) to the normative section.
  • A MOCORDER convention for String MOC and JSON MOC was added.

Reference Interoperable Implementations

(Indicate here the links to at least two Reference Interoperable Implementations)

  • MOCPy >=0.5.6
  • Aladin (>v10.126): read/write ASCII MOC (select ASCII format when exporting MOC - see snapshot below) + new script command dedicated to ASCII MOC (ex: draw MOC 3/1,2,3 4/56-67)
  • The relational registry at http://dc.g-vo.org/tap parses the ASCII MOCs in Registry records into the rr.stc_spatial table (consumer)
  • DaCHS since version 1.1 produces ASCII MOCs when serialising MOC-values database columns.
  • The Moc class within the AST/PyAST WCS library now supports reading and writing ASCII and JSON MOCs in addition to FITS.
To see non-trivial examples for ASCII MOCs coming out of a database, run something like

select top 100 * from rr.stc_spatial
where 1=contains(point(10, 10), coverage)
and 0=contains(point(20, 20), coverage)

on http://dc.g-vo.org/tap.

Implementations Validators

Comments Prior to Official RFC Period

As I was considering semantic validation cases, I noticed this sentence in section 2.3.2:

"Warning: In this basic simple ASCII string format the values may be not sorted, and the MOC may be not well-formed."

which seems to contradict section 2.2.3:

" An encoded MOC must also be “well-formed”. To keep the canonical property of MOC, the redundant cells are not allowed and the hierarchical encoding principle must be respected. Thus it is not allowed to encode 4 sibling cells instead of their parent (only at level 0 for the 12 first diamonds). Using this simple constraint, there is one and only one way to describe a specific region and this makes the interoperability principle easy to implement."

Maybe the former was just a legacy from MOC 1.0 where ASCII MOC was less formal, perhaps intended for hand-crafted MOCs not intended for serious use. With ASCII MOC now being formalized, can we remove that sentence and require that ASCII MOCs must be just as well-formed as their FITS counterparts?

-- TomDonaldson - 2019-06-14

As an ASCII MOC is often written by hand (in a Web form for instance, or as an Aladin script command), it was difficult to insure that it is always well formed at this level. Is it an ASCII MOC, yes, but potentially not well formed (ex: 3/1-210) . So the tools which have to manipulate such ASCII MOCs must check before used. At the opposite, FITS MOC is always generated by a code. So there is a real interest to force it to be stored well-formed for accelerating the future reloads.
That said, you right that it is a little bit inhomogeneous, and the ASCII MOCs will be used in other contexts that just for forms and scripts. So we can considere that an ASCII MOC written by hand is not - strickly speaking an ASCII MOC (not serialized in a file) - and in this consideration - extend the well-formed constraint to all serialized MOC formats (FITS & ASCII) by just removing the sentence.

-- PierreFernique - 2019-06-18

Tom answer 2019-06-21
That would be great. Thank you for doing that.




Comments from the IVOA Community during RFC/TCG review period: 2019-06-14 - 2019-07-29

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

  • Comment from Carlo Zwölf (CarloMariaZwoelf) : It is not clear for me in this proposed recommendation what comes natively from the Healpix definition and what is the IVOA-specific layer. Is there a way for highlighting this difference in the text?

    This new document is a very minor modification of the MOC 1.0 REC to standardize the ASCII MOC so that it can be used as quickly as possible in other IVOA standards, including VOregistry. The original text was not modified voluntarily in order to quickly reach a consensus. As your suggestion may imply a significant refactoring of the document, I propose to keep this point in mind for the next major release. It is clear that the introduction of the temporal MOC and the space-time MOC will involve a deep change in the document, especially in the direction you have suggested.
    -- PierreFernique - 2019-09-03



Comments from TCG member during the RFC/TCG Review Period: 2019-06-14 - 2019-07-29

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

MOC specification changes to bring String serialization to the normative part looks a good addition to this standard. The document itself is fine in general. I have two small things I'd like to point out.

  • letting [space,CR,LF,comma], multiples/mixes of them, as the elements separator has a human readability effect (see email), not so alerting, but also requires some code in MOC usage. Isn't it really possible to force a bit this serialization to be more smooth with, at least, single and homogeneous separators throughout a single ASCII MOC? Could it be at least warmly suggested?

    The flexibility of the ASCII MOC has been driven by three main pragmatic reasons:
    1- this syntax was exactly the suggested syntax in MOC1.0.
    => consequently there are already existing ASCII MOCs following this syntax,
    => It was easier to keep the syntax as it was suggested in MOC 1.0
    2- an ASCII MOC is often written by hand (in a Web form for instance, or as an Aladin script command),
    => a certain level of tolerance may be useful to avoid rejecting MOC with double space separators...
    3- an ASCII MOC can be a very very long string line
    => As for BASE64 ASCII streaming, it may be really problematic to read/edit an ASCII MOC in a regular ASCII editor without the CR/LF separator and multiple space for possible indentation. Also, we will have serious potential problems for reading and writting a so long line due to the classical flush mecanism of ASCII libs.

    The two first points are managable even if we really want to reduce the flexibility of the ASCII MOC syntax. Concretely the MOC clients/libs will have to be smart enough to support proto ASCII MOCs and possible separator typos (same argument that Tom's point). Quite reasonnable in fact. But I'm strongly in favor to take into account the 3nd point and keep the possibility to split an ASCII MOC if required

    My conclusion would be to remove the "comma", and let space,CR,LF possibly multiple
    -- PierreFernique - 2019-09-03

  • MOCORDER is a MUST and clearly described in the FITS serialization, while in the String (and JSON) one(s) it is less clear how to grab that piece of information. This because it is only said to be a must "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order" (in which case it would be the last token of the String, followed by a slash and no npix). I agree that if it's not so it's implicit the MOCORDER is the last/highest order available, but I'd prefer it to be explicit.

    You are right. It should be a "MUST" without ambiguity.
    The current sentence for ASCII MOC (section 2.3.2) is : "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order, the MOCORDER must be provided, followed by a slash ("/") without any associated npix value."

    I suggest this new sentence: "The best HEALPix resolution of the MOC (MOCORDER) MUST be always provided, even if there is no explicit npix value at this best resolution (due to the MOC hierarchization process). In the latter case, the MOCORDER is specified followed by a slash ("/") without any associated npix value."
    Concerning JSON serialization (suggested syntax in a non normative section). Here the proposed new sentence : "In the same way as for ASCII MOC serialization, if the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order, the MOCORDER should be provided with an empty npix list."
    -- PierreFernique - 2019-09-03
-- MarcoMolinaro - 2019-07-11

Data Model Working Group

  • I support the Marco's comment about MOCORDER in string.

    See below -- PierreFernique - 2019-09-03

  • It would be nice if figures had a caption and a number used to reference them.

    Good point. Concretely the few figures in this document are already fully described in the main text and the insertion of captions introduces not very well come repetitions (see section 1.3 for instance) without modifying deeply the text. As presented, this document would like to be as light as possible modification of the MOC 1.0 standard. So, I would propose to keep in mind this suggestion for the next major release of MOC. -- PierreFernique - 2019-09-03
-- LaurentMichel - 2019-07-25

Grid & Web Services Working Group

I support Marco suggestions and at this stage I have no further comments. Please add caption to figures and tables and reference them in the text.

-- GiulianoTaffoni - 2019-09-11

Registry Working Group

I would like to echo Marco's comments above, regarding simplifying the separators in serialization, and perhaps go a bit further. At best, there should be only one way to do this. I am willing to settle for consistency within a MOC using any one of those, no mix and match, with a preferred character separator suggested. -- TheresaDower - 2019-07-24

See below -- PierreFernique - 2019-09-03

Semantics Working Group

The Semantics WG doesn't see anything concerning us in the proposed changes (that otherwise sound reasonable and useful to us). The spurious indent of the rule for moc in the EBNF grammar could be removed, though.

-- MarkusDemleitner - 2019-07-16

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Operations

Mostly OK, but some comments:

  • Table of contents: the page numbers are mostly wrong.

    Right -> and fixed -- PierreFernique - 2019-09-03

  • Sec 1.4: The existence of a FITS HEALPix serialization format on which the MOC serialization format is based is claimed, but no reference is given. Since this text (written by Eric Hivon) does now just about exist in a public form https://healpix.sourceforge.io/data/examples/healpix_fits_specs.pdf, can it be referenced here?
  • Sec 2.3.1: The HEALPix FITS encoding standard is mentioned here as well: "As the FITS MOC is derived from a more generic standard dedicated to HEALPix maps, the FITS header must contain the following keywords/value." That's a bit misleading, since the MOC serialization is not a special case of the HEALPix encoding (it doesn't contain some of the keywords required by that standard, such as INDEXSCHM and NSIDE) and some of the keywords have values which are not allowed in the HEALPix encoding (ORDERING='NUNIQ'). Given the difference between the two encodings, I'm not sure there's much point making the comparison here, but if so I'd suggest something more like "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."

    I agree. The FITS HEALPix map serialization was and is unfortunately not a standard in the sense of IVOA REC. HEALPix authors have deployed libs and tools which follow an evolving convention. The recent Eric Hivon document has clarified this situation. Some FITS keywords which was required in previous conventions have not been kept, and some other have been introduced. So I totally agree Mark's point and I will take the suggestion "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."
    -- PierreFernique - 2019-09-03

  • Sec 2.3.1: The MOCID and ORIGIN keys both "suggest using IVOA identification like ..." where the examples are of the form ivo://... . If these are intended to be IVOA Identifiers, it should say something more like "suggest using an IVOA Identifier ..." with a reference to the IVOA Identifiers document, and the examples given should be valid IVOIDs (i.e. should resolve in the registry - I don't think that "CADC" is an authority in the IVOID sense). If not, the ivo:// prefix seems a bit misleading. Reg WG may want to comment on this.
    Right -> Modified following the suggestion => MOCID: Specify an identifier for the original data used to build the MOC. If the original data set has been declared in the VO registry [11], we strongly encourage to use the IVOA identifier (ex: “ivo://wfau.roe.ac.uk/twomass-dsa”')" and "ORIGIN: This keyword could be use to store the name of the MOC origin. If the authority of the origin has been declared in the VO registry, we strongly encourage to use the IVOA identifier (ex: “ivo://cadc.nrc.ca”)." -- PierreFernique - 2019-09-03

  • Sec 2.3.2: "hightnpix" -> "highnpix" ?
    ok -> PierreFernique - 2019-09-03

  • Sec 2.3.2: I'm inclined to agree with Tom's and Marco's comments about the form of the ASCII MOC serialization - it's not clear to me why FITS MOCs have to be well-formed and ASCII ones don't, and why so many separator choices are allowed. Also, the BNF doesn't constrain where the optional MOCORDER marker has to go, or how many of these appear. Consider constraining this format a bit further, e.g. require the MOCORDER marker to be present and/or at the end (or start?) of the string?

    See above -- PierreFernique - 2019-09-03

  • Appendix B: most of the decimal separators are "." characters, but one is a "," ("1,17").
    Ok -> PierreFernique - 2019-09-03
-- MarkTaylor - 2019-07-13

Standards and Processes Committee


TCG Vote: 2019-06-14 - 2019-07-29

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps *      
DAL *     -- MarcoMolinaro / JamesDempsey 2019-07-17
Changed:
<
<
DM        
>
>
DM *     -- LaurentMichel - 2019-09-26
 
GWS *      
Registry        
Semantics *      
DCP        
KDIG        
SSIG        
Theory        
TD        
Ops *      
StdProc        



<--

-->
  • MOC_ASCII_in_Aladin.png:
    MOC_ASCII_in_Aladin.png

META FILEATTACHMENT attachment="MOC_ASCII_in_Aladin.png" attr="" comment="" date="1558081955" name="MOC_ASCII_in_Aladin.png" path="MOC_ASCII_in_Aladin.png" size="9507" user="PierreFernique" version="1"

Revision 202019-09-26 - TomDonaldson

 
META TOPICPARENT name="MocInfo"

MOC 1.1 Proposed Recommendation: Request for Comments


Summary

Latest Draft: MOC 1.1 (Proposed Recommendation)

The substantive differences between version 1.1 of MOC and the preceding version 1.0 are:

  • The String MOC serialization was moved from an informative section (suggested syntax) to the normative section.
  • A MOCORDER convention for String MOC and JSON MOC was added.

Reference Interoperable Implementations

(Indicate here the links to at least two Reference Interoperable Implementations)

  • MOCPy >=0.5.6
  • Aladin (>v10.126): read/write ASCII MOC (select ASCII format when exporting MOC - see snapshot below) + new script command dedicated to ASCII MOC (ex: draw MOC 3/1,2,3 4/56-67)
  • The relational registry at http://dc.g-vo.org/tap parses the ASCII MOCs in Registry records into the rr.stc_spatial table (consumer)
  • DaCHS since version 1.1 produces ASCII MOCs when serialising MOC-values database columns.
  • The Moc class within the AST/PyAST WCS library now supports reading and writing ASCII and JSON MOCs in addition to FITS.
To see non-trivial examples for ASCII MOCs coming out of a database, run something like

select top 100 * from rr.stc_spatial
where 1=contains(point(10, 10), coverage)
and 0=contains(point(20, 20), coverage)

on http://dc.g-vo.org/tap.

Implementations Validators

Comments Prior to Official RFC Period

As I was considering semantic validation cases, I noticed this sentence in section 2.3.2:

"Warning: In this basic simple ASCII string format the values may be not sorted, and the MOC may be not well-formed."

which seems to contradict section 2.2.3:

" An encoded MOC must also be “well-formed”. To keep the canonical property of MOC, the redundant cells are not allowed and the hierarchical encoding principle must be respected. Thus it is not allowed to encode 4 sibling cells instead of their parent (only at level 0 for the 12 first diamonds). Using this simple constraint, there is one and only one way to describe a specific region and this makes the interoperability principle easy to implement."

Maybe the former was just a legacy from MOC 1.0 where ASCII MOC was less formal, perhaps intended for hand-crafted MOCs not intended for serious use. With ASCII MOC now being formalized, can we remove that sentence and require that ASCII MOCs must be just as well-formed as their FITS counterparts?

-- TomDonaldson - 2019-06-14

As an ASCII MOC is often written by hand (in a Web form for instance, or as an Aladin script command), it was difficult to insure that it is always well formed at this level. Is it an ASCII MOC, yes, but potentially not well formed (ex: 3/1-210) . So the tools which have to manipulate such ASCII MOCs must check before used. At the opposite, FITS MOC is always generated by a code. So there is a real interest to force it to be stored well-formed for accelerating the future reloads.
That said, you right that it is a little bit inhomogeneous, and the ASCII MOCs will be used in other contexts that just for forms and scripts. So we can considere that an ASCII MOC written by hand is not - strickly speaking an ASCII MOC (not serialized in a file) - and in this consideration - extend the well-formed constraint to all serialized MOC formats (FITS & ASCII) by just removing the sentence.

-- PierreFernique - 2019-06-18

Tom answer 2019-06-21
That would be great. Thank you for doing that.




Comments from the IVOA Community during RFC/TCG review period: 2019-06-14 - 2019-07-29

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

  • Comment from Carlo Zwölf (CarloMariaZwoelf) : It is not clear for me in this proposed recommendation what comes natively from the Healpix definition and what is the IVOA-specific layer. Is there a way for highlighting this difference in the text?

    This new document is a very minor modification of the MOC 1.0 REC to standardize the ASCII MOC so that it can be used as quickly as possible in other IVOA standards, including VOregistry. The original text was not modified voluntarily in order to quickly reach a consensus. As your suggestion may imply a significant refactoring of the document, I propose to keep this point in mind for the next major release. It is clear that the introduction of the temporal MOC and the space-time MOC will involve a deep change in the document, especially in the direction you have suggested.
    -- PierreFernique - 2019-09-03



Comments from TCG member during the RFC/TCG Review Period: 2019-06-14 - 2019-07-29

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

MOC specification changes to bring String serialization to the normative part looks a good addition to this standard. The document itself is fine in general. I have two small things I'd like to point out.

  • letting [space,CR,LF,comma], multiples/mixes of them, as the elements separator has a human readability effect (see email), not so alerting, but also requires some code in MOC usage. Isn't it really possible to force a bit this serialization to be more smooth with, at least, single and homogeneous separators throughout a single ASCII MOC? Could it be at least warmly suggested?

    The flexibility of the ASCII MOC has been driven by three main pragmatic reasons:
    1- this syntax was exactly the suggested syntax in MOC1.0.
    => consequently there are already existing ASCII MOCs following this syntax,
    => It was easier to keep the syntax as it was suggested in MOC 1.0
    2- an ASCII MOC is often written by hand (in a Web form for instance, or as an Aladin script command),
    => a certain level of tolerance may be useful to avoid rejecting MOC with double space separators...
    3- an ASCII MOC can be a very very long string line
    => As for BASE64 ASCII streaming, it may be really problematic to read/edit an ASCII MOC in a regular ASCII editor without the CR/LF separator and multiple space for possible indentation. Also, we will have serious potential problems for reading and writting a so long line due to the classical flush mecanism of ASCII libs.

    The two first points are managable even if we really want to reduce the flexibility of the ASCII MOC syntax. Concretely the MOC clients/libs will have to be smart enough to support proto ASCII MOCs and possible separator typos (same argument that Tom's point). Quite reasonnable in fact. But I'm strongly in favor to take into account the 3nd point and keep the possibility to split an ASCII MOC if required

    My conclusion would be to remove the "comma", and let space,CR,LF possibly multiple
    -- PierreFernique - 2019-09-03

  • MOCORDER is a MUST and clearly described in the FITS serialization, while in the String (and JSON) one(s) it is less clear how to grab that piece of information. This because it is only said to be a must "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order" (in which case it would be the last token of the String, followed by a slash and no npix). I agree that if it's not so it's implicit the MOCORDER is the last/highest order available, but I'd prefer it to be explicit.

    You are right. It should be a "MUST" without ambiguity.
    The current sentence for ASCII MOC (section 2.3.2) is : "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order, the MOCORDER must be provided, followed by a slash ("/") without any associated npix value."

    I suggest this new sentence: "The best HEALPix resolution of the MOC (MOCORDER) MUST be always provided, even if there is no explicit npix value at this best resolution (due to the MOC hierarchization process). In the latter case, the MOCORDER is specified followed by a slash ("/") without any associated npix value."
    Concerning JSON serialization (suggested syntax in a non normative section). Here the proposed new sentence : "In the same way as for ASCII MOC serialization, if the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order, the MOCORDER should be provided with an empty npix list."
    -- PierreFernique - 2019-09-03
-- MarcoMolinaro - 2019-07-11

Data Model Working Group

  • I support the Marco's comment about MOCORDER in string.

    See below -- PierreFernique - 2019-09-03

  • It would be nice if figures had a caption and a number used to reference them.

    Good point. Concretely the few figures in this document are already fully described in the main text and the insertion of captions introduces not very well come repetitions (see section 1.3 for instance) without modifying deeply the text. As presented, this document would like to be as light as possible modification of the MOC 1.0 standard. So, I would propose to keep in mind this suggestion for the next major release of MOC. -- PierreFernique - 2019-09-03
-- LaurentMichel - 2019-07-25

Grid & Web Services Working Group

I support Marco suggestions and at this stage I have no further comments. Please add caption to figures and tables and reference them in the text.

-- GiulianoTaffoni - 2019-09-11

Registry Working Group

I would like to echo Marco's comments above, regarding simplifying the separators in serialization, and perhaps go a bit further. At best, there should be only one way to do this. I am willing to settle for consistency within a MOC using any one of those, no mix and match, with a preferred character separator suggested. -- TheresaDower - 2019-07-24

See below -- PierreFernique - 2019-09-03

Semantics Working Group

The Semantics WG doesn't see anything concerning us in the proposed changes (that otherwise sound reasonable and useful to us). The spurious indent of the rule for moc in the EBNF grammar could be removed, though.

-- MarkusDemleitner - 2019-07-16

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Operations

Mostly OK, but some comments:

  • Table of contents: the page numbers are mostly wrong.

    Right -> and fixed -- PierreFernique - 2019-09-03

  • Sec 1.4: The existence of a FITS HEALPix serialization format on which the MOC serialization format is based is claimed, but no reference is given. Since this text (written by Eric Hivon) does now just about exist in a public form https://healpix.sourceforge.io/data/examples/healpix_fits_specs.pdf, can it be referenced here?
  • Sec 2.3.1: The HEALPix FITS encoding standard is mentioned here as well: "As the FITS MOC is derived from a more generic standard dedicated to HEALPix maps, the FITS header must contain the following keywords/value." That's a bit misleading, since the MOC serialization is not a special case of the HEALPix encoding (it doesn't contain some of the keywords required by that standard, such as INDEXSCHM and NSIDE) and some of the keywords have values which are not allowed in the HEALPix encoding (ORDERING='NUNIQ'). Given the difference between the two encodings, I'm not sure there's much point making the comparison here, but if so I'd suggest something more like "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."

    I agree. The FITS HEALPix map serialization was and is unfortunately not a standard in the sense of IVOA REC. HEALPix authors have deployed libs and tools which follow an evolving convention. The recent Eric Hivon document has clarified this situation. Some FITS keywords which was required in previous conventions have not been kept, and some other have been introduced. So I totally agree Mark's point and I will take the suggestion "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."
    -- PierreFernique - 2019-09-03

  • Sec 2.3.1: The MOCID and ORIGIN keys both "suggest using IVOA identification like ..." where the examples are of the form ivo://... . If these are intended to be IVOA Identifiers, it should say something more like "suggest using an IVOA Identifier ..." with a reference to the IVOA Identifiers document, and the examples given should be valid IVOIDs (i.e. should resolve in the registry - I don't think that "CADC" is an authority in the IVOID sense). If not, the ivo:// prefix seems a bit misleading. Reg WG may want to comment on this.
    Right -> Modified following the suggestion => MOCID: Specify an identifier for the original data used to build the MOC. If the original data set has been declared in the VO registry [11], we strongly encourage to use the IVOA identifier (ex: “ivo://wfau.roe.ac.uk/twomass-dsa”')" and "ORIGIN: This keyword could be use to store the name of the MOC origin. If the authority of the origin has been declared in the VO registry, we strongly encourage to use the IVOA identifier (ex: “ivo://cadc.nrc.ca”)." -- PierreFernique - 2019-09-03

  • Sec 2.3.2: "hightnpix" -> "highnpix" ?
    ok -> PierreFernique - 2019-09-03

  • Sec 2.3.2: I'm inclined to agree with Tom's and Marco's comments about the form of the ASCII MOC serialization - it's not clear to me why FITS MOCs have to be well-formed and ASCII ones don't, and why so many separator choices are allowed. Also, the BNF doesn't constrain where the optional MOCORDER marker has to go, or how many of these appear. Consider constraining this format a bit further, e.g. require the MOCORDER marker to be present and/or at the end (or start?) of the string?

    See above -- PierreFernique - 2019-09-03

  • Appendix B: most of the decimal separators are "." characters, but one is a "," ("1,17").
    Ok -> PierreFernique - 2019-09-03
-- MarkTaylor - 2019-07-13

Standards and Processes Committee


TCG Vote: 2019-06-14 - 2019-07-29

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Changed:
<
<
Apps        
>
>
Apps *      
 
DAL *     -- MarcoMolinaro / JamesDempsey 2019-07-17
DM        
GWS *      
Registry        
Semantics *      
DCP        
KDIG        
SSIG        
Theory        
TD        
Ops *      
StdProc        



<--

-->
  • MOC_ASCII_in_Aladin.png:
    MOC_ASCII_in_Aladin.png

META FILEATTACHMENT attachment="MOC_ASCII_in_Aladin.png" attr="" comment="" date="1558081955" name="MOC_ASCII_in_Aladin.png" path="MOC_ASCII_in_Aladin.png" size="9507" user="PierreFernique" version="1"

Revision 192019-09-11 - MarkTaylor

 
META TOPICPARENT name="MocInfo"

MOC 1.1 Proposed Recommendation: Request for Comments


Summary

Latest Draft: MOC 1.1 (Proposed Recommendation)

The substantive differences between version 1.1 of MOC and the preceding version 1.0 are:

  • The String MOC serialization was moved from an informative section (suggested syntax) to the normative section.
  • A MOCORDER convention for String MOC and JSON MOC was added.

Reference Interoperable Implementations

(Indicate here the links to at least two Reference Interoperable Implementations)

  • MOCPy >=0.5.6
  • Aladin (>v10.126): read/write ASCII MOC (select ASCII format when exporting MOC - see snapshot below) + new script command dedicated to ASCII MOC (ex: draw MOC 3/1,2,3 4/56-67)
  • The relational registry at http://dc.g-vo.org/tap parses the ASCII MOCs in Registry records into the rr.stc_spatial table (consumer)
  • DaCHS since version 1.1 produces ASCII MOCs when serialising MOC-values database columns.
  • The Moc class within the AST/PyAST WCS library now supports reading and writing ASCII and JSON MOCs in addition to FITS.
To see non-trivial examples for ASCII MOCs coming out of a database, run something like

select top 100 * from rr.stc_spatial
where 1=contains(point(10, 10), coverage)
and 0=contains(point(20, 20), coverage)

on http://dc.g-vo.org/tap.

Implementations Validators

Comments Prior to Official RFC Period

As I was considering semantic validation cases, I noticed this sentence in section 2.3.2:

"Warning: In this basic simple ASCII string format the values may be not sorted, and the MOC may be not well-formed."

which seems to contradict section 2.2.3:

" An encoded MOC must also be “well-formed”. To keep the canonical property of MOC, the redundant cells are not allowed and the hierarchical encoding principle must be respected. Thus it is not allowed to encode 4 sibling cells instead of their parent (only at level 0 for the 12 first diamonds). Using this simple constraint, there is one and only one way to describe a specific region and this makes the interoperability principle easy to implement."

Maybe the former was just a legacy from MOC 1.0 where ASCII MOC was less formal, perhaps intended for hand-crafted MOCs not intended for serious use. With ASCII MOC now being formalized, can we remove that sentence and require that ASCII MOCs must be just as well-formed as their FITS counterparts?

-- TomDonaldson - 2019-06-14

As an ASCII MOC is often written by hand (in a Web form for instance, or as an Aladin script command), it was difficult to insure that it is always well formed at this level. Is it an ASCII MOC, yes, but potentially not well formed (ex: 3/1-210) . So the tools which have to manipulate such ASCII MOCs must check before used. At the opposite, FITS MOC is always generated by a code. So there is a real interest to force it to be stored well-formed for accelerating the future reloads.
That said, you right that it is a little bit inhomogeneous, and the ASCII MOCs will be used in other contexts that just for forms and scripts. So we can considere that an ASCII MOC written by hand is not - strickly speaking an ASCII MOC (not serialized in a file) - and in this consideration - extend the well-formed constraint to all serialized MOC formats (FITS & ASCII) by just removing the sentence.

-- PierreFernique - 2019-06-18

Tom answer 2019-06-21
That would be great. Thank you for doing that.




Comments from the IVOA Community during RFC/TCG review period: 2019-06-14 - 2019-07-29

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

  • Comment from Carlo Zwölf (CarloMariaZwoelf) : It is not clear for me in this proposed recommendation what comes natively from the Healpix definition and what is the IVOA-specific layer. Is there a way for highlighting this difference in the text?

    This new document is a very minor modification of the MOC 1.0 REC to standardize the ASCII MOC so that it can be used as quickly as possible in other IVOA standards, including VOregistry. The original text was not modified voluntarily in order to quickly reach a consensus. As your suggestion may imply a significant refactoring of the document, I propose to keep this point in mind for the next major release. It is clear that the introduction of the temporal MOC and the space-time MOC will involve a deep change in the document, especially in the direction you have suggested.
    -- PierreFernique - 2019-09-03



Comments from TCG member during the RFC/TCG Review Period: 2019-06-14 - 2019-07-29

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

MOC specification changes to bring String serialization to the normative part looks a good addition to this standard. The document itself is fine in general. I have two small things I'd like to point out.

  • letting [space,CR,LF,comma], multiples/mixes of them, as the elements separator has a human readability effect (see email), not so alerting, but also requires some code in MOC usage. Isn't it really possible to force a bit this serialization to be more smooth with, at least, single and homogeneous separators throughout a single ASCII MOC? Could it be at least warmly suggested?

    The flexibility of the ASCII MOC has been driven by three main pragmatic reasons:
    1- this syntax was exactly the suggested syntax in MOC1.0.
    => consequently there are already existing ASCII MOCs following this syntax,
    => It was easier to keep the syntax as it was suggested in MOC 1.0
    2- an ASCII MOC is often written by hand (in a Web form for instance, or as an Aladin script command),
    => a certain level of tolerance may be useful to avoid rejecting MOC with double space separators...
    3- an ASCII MOC can be a very very long string line
    => As for BASE64 ASCII streaming, it may be really problematic to read/edit an ASCII MOC in a regular ASCII editor without the CR/LF separator and multiple space for possible indentation. Also, we will have serious potential problems for reading and writting a so long line due to the classical flush mecanism of ASCII libs.

    The two first points are managable even if we really want to reduce the flexibility of the ASCII MOC syntax. Concretely the MOC clients/libs will have to be smart enough to support proto ASCII MOCs and possible separator typos (same argument that Tom's point). Quite reasonnable in fact. But I'm strongly in favor to take into account the 3nd point and keep the possibility to split an ASCII MOC if required

    My conclusion would be to remove the "comma", and let space,CR,LF possibly multiple
    -- PierreFernique - 2019-09-03

  • MOCORDER is a MUST and clearly described in the FITS serialization, while in the String (and JSON) one(s) it is less clear how to grab that piece of information. This because it is only said to be a must "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order" (in which case it would be the last token of the String, followed by a slash and no npix). I agree that if it's not so it's implicit the MOCORDER is the last/highest order available, but I'd prefer it to be explicit.

    You are right. It should be a "MUST" without ambiguity.
    The current sentence for ASCII MOC (section 2.3.2) is : "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order, the MOCORDER must be provided, followed by a slash ("/") without any associated npix value."

    I suggest this new sentence: "The best HEALPix resolution of the MOC (MOCORDER) MUST be always provided, even if there is no explicit npix value at this best resolution (due to the MOC hierarchization process). In the latter case, the MOCORDER is specified followed by a slash ("/") without any associated npix value."
    Concerning JSON serialization (suggested syntax in a non normative section). Here the proposed new sentence : "In the same way as for ASCII MOC serialization, if the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order, the MOCORDER should be provided with an empty npix list."
    -- PierreFernique - 2019-09-03
-- MarcoMolinaro - 2019-07-11

Data Model Working Group

  • I support the Marco's comment about MOCORDER in string.

    See below -- PierreFernique - 2019-09-03

  • It would be nice if figures had a caption and a number used to reference them.

    Good point. Concretely the few figures in this document are already fully described in the main text and the insertion of captions introduces not very well come repetitions (see section 1.3 for instance) without modifying deeply the text. As presented, this document would like to be as light as possible modification of the MOC 1.0 standard. So, I would propose to keep in mind this suggestion for the next major release of MOC. -- PierreFernique - 2019-09-03
-- LaurentMichel - 2019-07-25

Grid & Web Services Working Group

I support Marco suggestions and at this stage I have no further comments. Please add caption to figures and tables and reference them in the text.

-- GiulianoTaffoni - 2019-09-11

Registry Working Group

I would like to echo Marco's comments above, regarding simplifying the separators in serialization, and perhaps go a bit further. At best, there should be only one way to do this. I am willing to settle for consistency within a MOC using any one of those, no mix and match, with a preferred character separator suggested. -- TheresaDower - 2019-07-24

See below -- PierreFernique - 2019-09-03

Semantics Working Group

The Semantics WG doesn't see anything concerning us in the proposed changes (that otherwise sound reasonable and useful to us). The spurious indent of the rule for moc in the EBNF grammar could be removed, though.

-- MarkusDemleitner - 2019-07-16

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Operations

Mostly OK, but some comments:

  • Table of contents: the page numbers are mostly wrong.

    Right -> and fixed -- PierreFernique - 2019-09-03

  • Sec 1.4: The existence of a FITS HEALPix serialization format on which the MOC serialization format is based is claimed, but no reference is given. Since this text (written by Eric Hivon) does now just about exist in a public form https://healpix.sourceforge.io/data/examples/healpix_fits_specs.pdf, can it be referenced here?
  • Sec 2.3.1: The HEALPix FITS encoding standard is mentioned here as well: "As the FITS MOC is derived from a more generic standard dedicated to HEALPix maps, the FITS header must contain the following keywords/value." That's a bit misleading, since the MOC serialization is not a special case of the HEALPix encoding (it doesn't contain some of the keywords required by that standard, such as INDEXSCHM and NSIDE) and some of the keywords have values which are not allowed in the HEALPix encoding (ORDERING='NUNIQ'). Given the difference between the two encodings, I'm not sure there's much point making the comparison here, but if so I'd suggest something more like "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."

    I agree. The FITS HEALPix map serialization was and is unfortunately not a standard in the sense of IVOA REC. HEALPix authors have deployed libs and tools which follow an evolving convention. The recent Eric Hivon document has clarified this situation. Some FITS keywords which was required in previous conventions have not been kept, and some other have been introduced. So I totally agree Mark's point and I will take the suggestion "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."
    -- PierreFernique - 2019-09-03

  • Sec 2.3.1: The MOCID and ORIGIN keys both "suggest using IVOA identification like ..." where the examples are of the form ivo://... . If these are intended to be IVOA Identifiers, it should say something more like "suggest using an IVOA Identifier ..." with a reference to the IVOA Identifiers document, and the examples given should be valid IVOIDs (i.e. should resolve in the registry - I don't think that "CADC" is an authority in the IVOID sense). If not, the ivo:// prefix seems a bit misleading. Reg WG may want to comment on this.
    Right -> Modified following the suggestion => MOCID: Specify an identifier for the original data used to build the MOC. If the original data set has been declared in the VO registry [11], we strongly encourage to use the IVOA identifier (ex: “ivo://wfau.roe.ac.uk/twomass-dsa”')" and "ORIGIN: This keyword could be use to store the name of the MOC origin. If the authority of the origin has been declared in the VO registry, we strongly encourage to use the IVOA identifier (ex: “ivo://cadc.nrc.ca”)." -- PierreFernique - 2019-09-03

  • Sec 2.3.2: "hightnpix" -> "highnpix" ?
    ok -> PierreFernique - 2019-09-03

  • Sec 2.3.2: I'm inclined to agree with Tom's and Marco's comments about the form of the ASCII MOC serialization - it's not clear to me why FITS MOCs have to be well-formed and ASCII ones don't, and why so many separator choices are allowed. Also, the BNF doesn't constrain where the optional MOCORDER marker has to go, or how many of these appear. Consider constraining this format a bit further, e.g. require the MOCORDER marker to be present and/or at the end (or start?) of the string?

    See above -- PierreFernique - 2019-09-03

  • Appendix B: most of the decimal separators are "." characters, but one is a "," ("1,17").
    Ok -> PierreFernique - 2019-09-03
-- MarkTaylor - 2019-07-13

Standards and Processes Committee


TCG Vote: 2019-06-14 - 2019-07-29

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL *     -- MarcoMolinaro / JamesDempsey 2019-07-17
DM        
GWS *      
Registry        
Semantics *      
DCP        
KDIG        
SSIG        
Theory        
TD        
Changed:
<
<
Ops        
>
>
Ops *      
 
StdProc        



<--

-->
  • MOC_ASCII_in_Aladin.png:
    MOC_ASCII_in_Aladin.png

META FILEATTACHMENT attachment="MOC_ASCII_in_Aladin.png" attr="" comment="" date="1558081955" name="MOC_ASCII_in_Aladin.png" path="MOC_ASCII_in_Aladin.png" size="9507" user="PierreFernique" version="1"

Revision 182019-09-11 - GiulianoTaffoni

 
META TOPICPARENT name="MocInfo"

MOC 1.1 Proposed Recommendation: Request for Comments


Summary

Latest Draft: MOC 1.1 (Proposed Recommendation)

The substantive differences between version 1.1 of MOC and the preceding version 1.0 are:

  • The String MOC serialization was moved from an informative section (suggested syntax) to the normative section.
  • A MOCORDER convention for String MOC and JSON MOC was added.

Reference Interoperable Implementations

(Indicate here the links to at least two Reference Interoperable Implementations)

  • MOCPy >=0.5.6
  • Aladin (>v10.126): read/write ASCII MOC (select ASCII format when exporting MOC - see snapshot below) + new script command dedicated to ASCII MOC (ex: draw MOC 3/1,2,3 4/56-67)
  • The relational registry at http://dc.g-vo.org/tap parses the ASCII MOCs in Registry records into the rr.stc_spatial table (consumer)
  • DaCHS since version 1.1 produces ASCII MOCs when serialising MOC-values database columns.
  • The Moc class within the AST/PyAST WCS library now supports reading and writing ASCII and JSON MOCs in addition to FITS.
To see non-trivial examples for ASCII MOCs coming out of a database, run something like

select top 100 * from rr.stc_spatial
where 1=contains(point(10, 10), coverage)
and 0=contains(point(20, 20), coverage)

on http://dc.g-vo.org/tap.

Implementations Validators

Comments Prior to Official RFC Period

As I was considering semantic validation cases, I noticed this sentence in section 2.3.2:

"Warning: In this basic simple ASCII string format the values may be not sorted, and the MOC may be not well-formed."

which seems to contradict section 2.2.3:

" An encoded MOC must also be “well-formed”. To keep the canonical property of MOC, the redundant cells are not allowed and the hierarchical encoding principle must be respected. Thus it is not allowed to encode 4 sibling cells instead of their parent (only at level 0 for the 12 first diamonds). Using this simple constraint, there is one and only one way to describe a specific region and this makes the interoperability principle easy to implement."

Maybe the former was just a legacy from MOC 1.0 where ASCII MOC was less formal, perhaps intended for hand-crafted MOCs not intended for serious use. With ASCII MOC now being formalized, can we remove that sentence and require that ASCII MOCs must be just as well-formed as their FITS counterparts?

-- TomDonaldson - 2019-06-14

As an ASCII MOC is often written by hand (in a Web form for instance, or as an Aladin script command), it was difficult to insure that it is always well formed at this level. Is it an ASCII MOC, yes, but potentially not well formed (ex: 3/1-210) . So the tools which have to manipulate such ASCII MOCs must check before used. At the opposite, FITS MOC is always generated by a code. So there is a real interest to force it to be stored well-formed for accelerating the future reloads.
That said, you right that it is a little bit inhomogeneous, and the ASCII MOCs will be used in other contexts that just for forms and scripts. So we can considere that an ASCII MOC written by hand is not - strickly speaking an ASCII MOC (not serialized in a file) - and in this consideration - extend the well-formed constraint to all serialized MOC formats (FITS & ASCII) by just removing the sentence.

-- PierreFernique - 2019-06-18

Tom answer 2019-06-21
That would be great. Thank you for doing that.




Comments from the IVOA Community during RFC/TCG review period: 2019-06-14 - 2019-07-29

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

  • Comment from Carlo Zwölf (CarloMariaZwoelf) : It is not clear for me in this proposed recommendation what comes natively from the Healpix definition and what is the IVOA-specific layer. Is there a way for highlighting this difference in the text?

    This new document is a very minor modification of the MOC 1.0 REC to standardize the ASCII MOC so that it can be used as quickly as possible in other IVOA standards, including VOregistry. The original text was not modified voluntarily in order to quickly reach a consensus. As your suggestion may imply a significant refactoring of the document, I propose to keep this point in mind for the next major release. It is clear that the introduction of the temporal MOC and the space-time MOC will involve a deep change in the document, especially in the direction you have suggested.
    -- PierreFernique - 2019-09-03



Comments from TCG member during the RFC/TCG Review Period: 2019-06-14 - 2019-07-29

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

MOC specification changes to bring String serialization to the normative part looks a good addition to this standard. The document itself is fine in general. I have two small things I'd like to point out.

  • letting [space,CR,LF,comma], multiples/mixes of them, as the elements separator has a human readability effect (see email), not so alerting, but also requires some code in MOC usage. Isn't it really possible to force a bit this serialization to be more smooth with, at least, single and homogeneous separators throughout a single ASCII MOC? Could it be at least warmly suggested?

    The flexibility of the ASCII MOC has been driven by three main pragmatic reasons:
    1- this syntax was exactly the suggested syntax in MOC1.0.
    => consequently there are already existing ASCII MOCs following this syntax,
    => It was easier to keep the syntax as it was suggested in MOC 1.0
    2- an ASCII MOC is often written by hand (in a Web form for instance, or as an Aladin script command),
    => a certain level of tolerance may be useful to avoid rejecting MOC with double space separators...
    3- an ASCII MOC can be a very very long string line
    => As for BASE64 ASCII streaming, it may be really problematic to read/edit an ASCII MOC in a regular ASCII editor without the CR/LF separator and multiple space for possible indentation. Also, we will have serious potential problems for reading and writting a so long line due to the classical flush mecanism of ASCII libs.

    The two first points are managable even if we really want to reduce the flexibility of the ASCII MOC syntax. Concretely the MOC clients/libs will have to be smart enough to support proto ASCII MOCs and possible separator typos (same argument that Tom's point). Quite reasonnable in fact. But I'm strongly in favor to take into account the 3nd point and keep the possibility to split an ASCII MOC if required

    My conclusion would be to remove the "comma", and let space,CR,LF possibly multiple
    -- PierreFernique - 2019-09-03

  • MOCORDER is a MUST and clearly described in the FITS serialization, while in the String (and JSON) one(s) it is less clear how to grab that piece of information. This because it is only said to be a must "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order" (in which case it would be the last token of the String, followed by a slash and no npix). I agree that if it's not so it's implicit the MOCORDER is the last/highest order available, but I'd prefer it to be explicit.

    You are right. It should be a "MUST" without ambiguity.
    The current sentence for ASCII MOC (section 2.3.2) is : "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order, the MOCORDER must be provided, followed by a slash ("/") without any associated npix value."

    I suggest this new sentence: "The best HEALPix resolution of the MOC (MOCORDER) MUST be always provided, even if there is no explicit npix value at this best resolution (due to the MOC hierarchization process). In the latter case, the MOCORDER is specified followed by a slash ("/") without any associated npix value."
    Concerning JSON serialization (suggested syntax in a non normative section). Here the proposed new sentence : "In the same way as for ASCII MOC serialization, if the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order, the MOCORDER should be provided with an empty npix list."
    -- PierreFernique - 2019-09-03
-- MarcoMolinaro - 2019-07-11

Data Model Working Group

  • I support the Marco's comment about MOCORDER in string.

    See below -- PierreFernique - 2019-09-03

  • It would be nice if figures had a caption and a number used to reference them.

    Good point. Concretely the few figures in this document are already fully described in the main text and the insertion of captions introduces not very well come repetitions (see section 1.3 for instance) without modifying deeply the text. As presented, this document would like to be as light as possible modification of the MOC 1.0 standard. So, I would propose to keep in mind this suggestion for the next major release of MOC. -- PierreFernique - 2019-09-03
-- LaurentMichel - 2019-07-25

Grid & Web Services Working Group

Added:
>
>
I support Marco suggestions and at this stage I have no further comments. Please add caption to figures and tables and reference them in the text.

-- GiulianoTaffoni - 2019-09-11

 

Registry Working Group

I would like to echo Marco's comments above, regarding simplifying the separators in serialization, and perhaps go a bit further. At best, there should be only one way to do this. I am willing to settle for consistency within a MOC using any one of those, no mix and match, with a preferred character separator suggested. -- TheresaDower - 2019-07-24

See below -- PierreFernique - 2019-09-03

Semantics Working Group

The Semantics WG doesn't see anything concerning us in the proposed changes (that otherwise sound reasonable and useful to us). The spurious indent of the rule for moc in the EBNF grammar could be removed, though.

-- MarkusDemleitner - 2019-07-16

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Operations

Mostly OK, but some comments:

  • Table of contents: the page numbers are mostly wrong.

    Right -> and fixed -- PierreFernique - 2019-09-03

  • Sec 1.4: The existence of a FITS HEALPix serialization format on which the MOC serialization format is based is claimed, but no reference is given. Since this text (written by Eric Hivon) does now just about exist in a public form https://healpix.sourceforge.io/data/examples/healpix_fits_specs.pdf, can it be referenced here?
  • Sec 2.3.1: The HEALPix FITS encoding standard is mentioned here as well: "As the FITS MOC is derived from a more generic standard dedicated to HEALPix maps, the FITS header must contain the following keywords/value." That's a bit misleading, since the MOC serialization is not a special case of the HEALPix encoding (it doesn't contain some of the keywords required by that standard, such as INDEXSCHM and NSIDE) and some of the keywords have values which are not allowed in the HEALPix encoding (ORDERING='NUNIQ'). Given the difference between the two encodings, I'm not sure there's much point making the comparison here, but if so I'd suggest something more like "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."

    I agree. The FITS HEALPix map serialization was and is unfortunately not a standard in the sense of IVOA REC. HEALPix authors have deployed libs and tools which follow an evolving convention. The recent Eric Hivon document has clarified this situation. Some FITS keywords which was required in previous conventions have not been kept, and some other have been introduced. So I totally agree Mark's point and I will take the suggestion "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."
    -- PierreFernique - 2019-09-03

  • Sec 2.3.1: The MOCID and ORIGIN keys both "suggest using IVOA identification like ..." where the examples are of the form ivo://... . If these are intended to be IVOA Identifiers, it should say something more like "suggest using an IVOA Identifier ..." with a reference to the IVOA Identifiers document, and the examples given should be valid IVOIDs (i.e. should resolve in the registry - I don't think that "CADC" is an authority in the IVOID sense). If not, the ivo:// prefix seems a bit misleading. Reg WG may want to comment on this.
    Right -> Modified following the suggestion => MOCID: Specify an identifier for the original data used to build the MOC. If the original data set has been declared in the VO registry [11], we strongly encourage to use the IVOA identifier (ex: “ivo://wfau.roe.ac.uk/twomass-dsa”')" and "ORIGIN: This keyword could be use to store the name of the MOC origin. If the authority of the origin has been declared in the VO registry, we strongly encourage to use the IVOA identifier (ex: “ivo://cadc.nrc.ca”)." -- PierreFernique - 2019-09-03

  • Sec 2.3.2: "hightnpix" -> "highnpix" ?
    ok -> PierreFernique - 2019-09-03

  • Sec 2.3.2: I'm inclined to agree with Tom's and Marco's comments about the form of the ASCII MOC serialization - it's not clear to me why FITS MOCs have to be well-formed and ASCII ones don't, and why so many separator choices are allowed. Also, the BNF doesn't constrain where the optional MOCORDER marker has to go, or how many of these appear. Consider constraining this format a bit further, e.g. require the MOCORDER marker to be present and/or at the end (or start?) of the string?

    See above -- PierreFernique - 2019-09-03

  • Appendix B: most of the decimal separators are "." characters, but one is a "," ("1,17").
    Ok -> PierreFernique - 2019-09-03
-- MarkTaylor - 2019-07-13

Standards and Processes Committee


TCG Vote: 2019-06-14 - 2019-07-29

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL *     -- MarcoMolinaro / JamesDempsey 2019-07-17
DM        
Changed:
<
<
GWS        
>
>
GWS *      
 
Registry        
Semantics *      
DCP        
KDIG        
SSIG        
Theory        
TD        
Ops        
StdProc        



<--

-->
  • MOC_ASCII_in_Aladin.png:
    MOC_ASCII_in_Aladin.png

META FILEATTACHMENT attachment="MOC_ASCII_in_Aladin.png" attr="" comment="" date="1558081955" name="MOC_ASCII_in_Aladin.png" path="MOC_ASCII_in_Aladin.png" size="9507" user="PierreFernique" version="1"

Revision 172019-09-03 - PierreFernique

 
META TOPICPARENT name="MocInfo"

MOC 1.1 Proposed Recommendation: Request for Comments


Summary

Latest Draft: MOC 1.1 (Proposed Recommendation)

The substantive differences between version 1.1 of MOC and the preceding version 1.0 are:

  • The String MOC serialization was moved from an informative section (suggested syntax) to the normative section.
  • A MOCORDER convention for String MOC and JSON MOC was added.

Reference Interoperable Implementations

(Indicate here the links to at least two Reference Interoperable Implementations)

  • MOCPy >=0.5.6
  • Aladin (>v10.126): read/write ASCII MOC (select ASCII format when exporting MOC - see snapshot below) + new script command dedicated to ASCII MOC (ex: draw MOC 3/1,2,3 4/56-67)
  • The relational registry at http://dc.g-vo.org/tap parses the ASCII MOCs in Registry records into the rr.stc_spatial table (consumer)
  • DaCHS since version 1.1 produces ASCII MOCs when serialising MOC-values database columns.
  • The Moc class within the AST/PyAST WCS library now supports reading and writing ASCII and JSON MOCs in addition to FITS.
To see non-trivial examples for ASCII MOCs coming out of a database, run something like

select top 100 * from rr.stc_spatial
where 1=contains(point(10, 10), coverage)
and 0=contains(point(20, 20), coverage)

on http://dc.g-vo.org/tap.

Implementations Validators

Comments Prior to Official RFC Period

As I was considering semantic validation cases, I noticed this sentence in section 2.3.2:

"Warning: In this basic simple ASCII string format the values may be not sorted, and the MOC may be not well-formed."

which seems to contradict section 2.2.3:

" An encoded MOC must also be “well-formed”. To keep the canonical property of MOC, the redundant cells are not allowed and the hierarchical encoding principle must be respected. Thus it is not allowed to encode 4 sibling cells instead of their parent (only at level 0 for the 12 first diamonds). Using this simple constraint, there is one and only one way to describe a specific region and this makes the interoperability principle easy to implement."

Maybe the former was just a legacy from MOC 1.0 where ASCII MOC was less formal, perhaps intended for hand-crafted MOCs not intended for serious use. With ASCII MOC now being formalized, can we remove that sentence and require that ASCII MOCs must be just as well-formed as their FITS counterparts?

-- TomDonaldson - 2019-06-14

As an ASCII MOC is often written by hand (in a Web form for instance, or as an Aladin script command), it was difficult to insure that it is always well formed at this level. Is it an ASCII MOC, yes, but potentially not well formed (ex: 3/1-210) . So the tools which have to manipulate such ASCII MOCs must check before used. At the opposite, FITS MOC is always generated by a code. So there is a real interest to force it to be stored well-formed for accelerating the future reloads.
That said, you right that it is a little bit inhomogeneous, and the ASCII MOCs will be used in other contexts that just for forms and scripts. So we can considere that an ASCII MOC written by hand is not - strickly speaking an ASCII MOC (not serialized in a file) - and in this consideration - extend the well-formed constraint to all serialized MOC formats (FITS & ASCII) by just removing the sentence.

-- PierreFernique - 2019-06-18

Tom answer 2019-06-21
That would be great. Thank you for doing that.




Comments from the IVOA Community during RFC/TCG review period: 2019-06-14 - 2019-07-29

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

  • Comment from Carlo Zwölf (CarloMariaZwoelf) : It is not clear for me in this proposed recommendation what comes natively from the Healpix definition and what is the IVOA-specific layer. Is there a way for highlighting this difference in the text?

    This new document is a very minor modification of the MOC 1.0 REC to standardize the ASCII MOC so that it can be used as quickly as possible in other IVOA standards, including VOregistry. The original text was not modified voluntarily in order to quickly reach a consensus. As your suggestion may imply a significant refactoring of the document, I propose to keep this point in mind for the next major release. It is clear that the introduction of the temporal MOC and the space-time MOC will involve a deep change in the document, especially in the direction you have suggested.
    -- PierreFernique - 2019-09-03



Comments from TCG member during the RFC/TCG Review Period: 2019-06-14 - 2019-07-29

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

MOC specification changes to bring String serialization to the normative part looks a good addition to this standard. The document itself is fine in general. I have two small things I'd like to point out.

Changed:
<
<
  • letting [space,CR,LF,comma], multiples/mixes of them, as the elements separator has a human readability effect (see email), not so alerting, but also requires some code in MOC usage. Isn't it really possible to force a bit this serialization to be more smooth with, at least, single and homogeneous separators throughout a single ASCII MOC? Could it be at least warmly suggested?

    The flexibility of the ASCII MOC has been driven by three main pragmatic reasons:
    1- this syntax was exactly the suggested syntax in MOC1.0.
    => consequently there are already existing ASCII MOCs following this syntax,
    => It was easier to keep the syntax as it was suggested in MOC 1.0
    2- an ASCII MOC is often written by hand (in a Web form for instance, or as an Aladin script command),
    => a certain level of tolerance may be useful to avoid rejecting MOC with double space separators...
    3- an ASCII MOC can be a very very long string line
    => As for BASE64 ASCII streaming, it may be really problematic to read/edit an ASCII MOC in a regular ASCII editor without the CR/LF separator and multiple space for possible indentation. Also, we will have serious potential problems for reading and writting a so long line due to the classical flush mecanism of ASCII libs.

    The two first points are managable even if we really want to reduce the flexibility of the ASCII MOC syntax. Concretely the MOC clients/libs will have to be smart enough to support proto ASCII MOCs and possible separator typos (same argument that Tom's point). Quite reasonnable in fact. But I'm strongly in favor to take into account the 3nd point and keep the possibility to split an ASCII MOC if required

    My conclusion would be to remove the "comma", and let space,CR,LF possibly multiple for the space
    -- PierreFernique - 2019-09-03

  • MOCORDER is a MUST and clearly described in the FITS serialization, while in the String (and JSON) one(s) it is less clear how to grab that piece of information. This because it is only said to be a must "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order" (in which case it would be the last token of the String, followed by a slash and no npix). I agree that if it's not so it's implicit the MOCORDER is the last/highest order available, but I'd prefer it to be explicit.

    You are right. It should be a "MUST" without ambiguity.
    The current sentence is : "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order, the MOCORDER must be provided, followed by a slash ("/") without any associated npix value."
    I suggest this new sentence:
    "The best HEALPix resolution of the MOC (MOCORDER) MUST be always provided, even if there is no explicit npix value at this best resolution (due to the MOC hierarchization process). In the latter case, the MOCORDER is specified followed by a slash ("/") without any associated npix value."
    -- PierreFernique - 2019-09-03
>
>
  • letting [space,CR,LF,comma], multiples/mixes of them, as the elements separator has a human readability effect (see email), not so alerting, but also requires some code in MOC usage. Isn't it really possible to force a bit this serialization to be more smooth with, at least, single and homogeneous separators throughout a single ASCII MOC? Could it be at least warmly suggested?

    The flexibility of the ASCII MOC has been driven by three main pragmatic reasons:
    1- this syntax was exactly the suggested syntax in MOC1.0.
    => consequently there are already existing ASCII MOCs following this syntax,
    => It was easier to keep the syntax as it was suggested in MOC 1.0
    2- an ASCII MOC is often written by hand (in a Web form for instance, or as an Aladin script command),
    => a certain level of tolerance may be useful to avoid rejecting MOC with double space separators...
    3- an ASCII MOC can be a very very long string line
    => As for BASE64 ASCII streaming, it may be really problematic to read/edit an ASCII MOC in a regular ASCII editor without the CR/LF separator and multiple space for possible indentation. Also, we will have serious potential problems for reading and writting a so long line due to the classical flush mecanism of ASCII libs.

    The two first points are managable even if we really want to reduce the flexibility of the ASCII MOC syntax. Concretely the MOC clients/libs will have to be smart enough to support proto ASCII MOCs and possible separator typos (same argument that Tom's point). Quite reasonnable in fact. But I'm strongly in favor to take into account the 3nd point and keep the possibility to split an ASCII MOC if required

    My conclusion would be to remove the "comma", and let space,CR,LF possibly multiple
    -- PierreFernique - 2019-09-03

  • MOCORDER is a MUST and clearly described in the FITS serialization, while in the String (and JSON) one(s) it is less clear how to grab that piece of information. This because it is only said to be a must "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order" (in which case it would be the last token of the String, followed by a slash and no npix). I agree that if it's not so it's implicit the MOCORDER is the last/highest order available, but I'd prefer it to be explicit.

    You are right. It should be a "MUST" without ambiguity.
    The current sentence for ASCII MOC (section 2.3.2) is : "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order, the MOCORDER must be provided, followed by a slash ("/") without any associated npix value."

    I suggest this new sentence: "The best HEALPix resolution of the MOC (MOCORDER) MUST be always provided, even if there is no explicit npix value at this best resolution (due to the MOC hierarchization process). In the latter case, the MOCORDER is specified followed by a slash ("/") without any associated npix value."
    Concerning JSON serialization (suggested syntax in a non normative section). Here the proposed new sentence : "In the same way as for ASCII MOC serialization, if the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order, the MOCORDER should be provided with an empty npix list."
    -- PierreFernique - 2019-09-03
 -- MarcoMolinaro - 2019-07-11

Data Model Working Group

  • I support the Marco's comment about MOCORDER in string.

    See below -- PierreFernique - 2019-09-03
Changed:
<
<
  • It would be nice if figures had a caption and a number used to reference them.

    Good point. I do -- PierreFernique - 2019-09-03
>
>
Added:
>
>
  • It would be nice if figures had a caption and a number used to reference them.

    Good point. Concretely the few figures in this document are already fully described in the main text and the insertion of captions introduces not very well come repetitions (see section 1.3 for instance) without modifying deeply the text. As presented, this document would like to be as light as possible modification of the MOC 1.0 standard. So, I would propose to keep in mind this suggestion for the next major release of MOC. -- PierreFernique - 2019-09-03
 -- LaurentMichel - 2019-07-25

Grid & Web Services Working Group

Registry Working Group

I would like to echo Marco's comments above, regarding simplifying the separators in serialization, and perhaps go a bit further. At best, there should be only one way to do this. I am willing to settle for consistency within a MOC using any one of those, no mix and match, with a preferred character separator suggested. -- TheresaDower - 2019-07-24

See below -- PierreFernique - 2019-09-03

Semantics Working Group

The Semantics WG doesn't see anything concerning us in the proposed changes (that otherwise sound reasonable and useful to us). The spurious indent of the rule for moc in the EBNF grammar could be removed, though.

-- MarkusDemleitner - 2019-07-16

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Operations

Mostly OK, but some comments:

Changed:
<
<
  • Table of contents: the page numbers are mostly wrong.

    Right -> and fixed -- PierreFernique - 2019-09-03
>
>
  • Table of contents: the page numbers are mostly wrong.

    Right -> and fixed -- PierreFernique - 2019-09-03
Added:
>
>
 
Changed:
<
<
  • Sec 2.3.1: The HEALPix FITS encoding standard is mentioned here as well: "As the FITS MOC is derived from a more generic standard dedicated to HEALPix maps, the FITS header must contain the following keywords/value." That's a bit misleading, since the MOC serialization is not a special case of the HEALPix encoding (it doesn't contain some of the keywords required by that standard, such as INDEXSCHM and NSIDE) and some of the keywords have values which are not allowed in the HEALPix encoding (ORDERING='NUNIQ'). Given the difference between the two encodings, I'm not sure there's much point making the comparison here, but if so I'd suggest something more like "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."
  • Sec 2.3.1: The MOCID and ORIGIN keys both "suggest using IVOA identification like ..." where the examples are of the form ivo://... . If these are intended to be IVOA Identifiers, it should say something more like "suggest using an IVOA Identifier ..." with a reference to the IVOA Identifiers document, and the examples given should be valid IVOIDs (i.e. should resolve in the registry - I don't think that "CADC" is an authority in the IVOID sense). If not, the ivo:// prefix seems a bit misleading. Reg WG may want to comment on this.
  • Sec 2.3.2: "hightnpix" -> "highnpix" ?
  • Sec 2.3.2: I'm inclined to agree with Tom's and Marco's comments about the form of the ASCII MOC serialization - it's not clear to me why FITS MOCs have to be well-formed and ASCII ones don't, and why so many separator choices are allowed. Also, the BNF doesn't constrain where the optional MOCORDER marker has to go, or how many of these appear. Consider constraining this format a bit further, e.g. require the MOCORDER marker to be present and/or at the end (or start?) of the string?
  • Appendix B: most of the decimal separators are "." characters, but one is a "," ("1,17").
>
>
  • Sec 2.3.1: The HEALPix FITS encoding standard is mentioned here as well: "As the FITS MOC is derived from a more generic standard dedicated to HEALPix maps, the FITS header must contain the following keywords/value." That's a bit misleading, since the MOC serialization is not a special case of the HEALPix encoding (it doesn't contain some of the keywords required by that standard, such as INDEXSCHM and NSIDE) and some of the keywords have values which are not allowed in the HEALPix encoding (ORDERING='NUNIQ'). Given the difference between the two encodings, I'm not sure there's much point making the comparison here, but if so I'd suggest something more like "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."

    I agree. The FITS HEALPix map serialization was and is unfortunately not a standard in the sense of IVOA REC. HEALPix authors have deployed libs and tools which follow an evolving convention. The recent Eric Hivon document has clarified this situation. Some FITS keywords which was required in previous conventions have not been kept, and some other have been introduced. So I totally agree Mark's point and I will take the suggestion "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."
    -- PierreFernique - 2019-09-03

  • Sec 2.3.1: The MOCID and ORIGIN keys both "suggest using IVOA identification like ..." where the examples are of the form ivo://... . If these are intended to be IVOA Identifiers, it should say something more like "suggest using an IVOA Identifier ..." with a reference to the IVOA Identifiers document, and the examples given should be valid IVOIDs (i.e. should resolve in the registry - I don't think that "CADC" is an authority in the IVOID sense). If not, the ivo:// prefix seems a bit misleading. Reg WG may want to comment on this.
    Right -> Modified following the suggestion => MOCID: Specify an identifier for the original data used to build the MOC. If the original data set has been declared in the VO registry [11], we strongly encourage to use the IVOA identifier (ex: “ivo://wfau.roe.ac.uk/twomass-dsa”')" and "ORIGIN: This keyword could be use to store the name of the MOC origin. If the authority of the origin has been declared in the VO registry, we strongly encourage to use the IVOA identifier (ex: “ivo://cadc.nrc.ca”)." -- PierreFernique - 2019-09-03

  • Sec 2.3.2: "hightnpix" -> "highnpix" ?
    ok -> PierreFernique - 2019-09-03

  • Sec 2.3.2: I'm inclined to agree with Tom's and Marco's comments about the form of the ASCII MOC serialization - it's not clear to me why FITS MOCs have to be well-formed and ASCII ones don't, and why so many separator choices are allowed. Also, the BNF doesn't constrain where the optional MOCORDER marker has to go, or how many of these appear. Consider constraining this format a bit further, e.g. require the MOCORDER marker to be present and/or at the end (or start?) of the string?

    See above -- PierreFernique - 2019-09-03

  • Appendix B: most of the decimal separators are "." characters, but one is a "," ("1,17").
    Ok -> PierreFernique - 2019-09-03
 -- MarkTaylor - 2019-07-13

Standards and Processes Committee


TCG Vote: 2019-06-14 - 2019-07-29

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL *     -- MarcoMolinaro / JamesDempsey 2019-07-17
DM        
GWS        
Registry        
Semantics *      
DCP        
KDIG        
SSIG        
Theory        
TD        
Ops        
StdProc        



<--

-->
  • MOC_ASCII_in_Aladin.png:
    MOC_ASCII_in_Aladin.png

META FILEATTACHMENT attachment="MOC_ASCII_in_Aladin.png" attr="" comment="" date="1558081955" name="MOC_ASCII_in_Aladin.png" path="MOC_ASCII_in_Aladin.png" size="9507" user="PierreFernique" version="1"

Revision 162019-09-03 - PierreFernique

 
META TOPICPARENT name="MocInfo"

MOC 1.1 Proposed Recommendation: Request for Comments


Summary

Latest Draft: MOC 1.1 (Proposed Recommendation)

The substantive differences between version 1.1 of MOC and the preceding version 1.0 are:

  • The String MOC serialization was moved from an informative section (suggested syntax) to the normative section.
  • A MOCORDER convention for String MOC and JSON MOC was added.

Reference Interoperable Implementations

(Indicate here the links to at least two Reference Interoperable Implementations)

  • MOCPy >=0.5.6
  • Aladin (>v10.126): read/write ASCII MOC (select ASCII format when exporting MOC - see snapshot below) + new script command dedicated to ASCII MOC (ex: draw MOC 3/1,2,3 4/56-67)
  • The relational registry at http://dc.g-vo.org/tap parses the ASCII MOCs in Registry records into the rr.stc_spatial table (consumer)
  • DaCHS since version 1.1 produces ASCII MOCs when serialising MOC-values database columns.
  • The Moc class within the AST/PyAST WCS library now supports reading and writing ASCII and JSON MOCs in addition to FITS.
To see non-trivial examples for ASCII MOCs coming out of a database, run something like

select top 100 * from rr.stc_spatial
where 1=contains(point(10, 10), coverage)
and 0=contains(point(20, 20), coverage)

on http://dc.g-vo.org/tap.

Implementations Validators

Comments Prior to Official RFC Period

As I was considering semantic validation cases, I noticed this sentence in section 2.3.2:

"Warning: In this basic simple ASCII string format the values may be not sorted, and the MOC may be not well-formed."

which seems to contradict section 2.2.3:

" An encoded MOC must also be “well-formed”. To keep the canonical property of MOC, the redundant cells are not allowed and the hierarchical encoding principle must be respected. Thus it is not allowed to encode 4 sibling cells instead of their parent (only at level 0 for the 12 first diamonds). Using this simple constraint, there is one and only one way to describe a specific region and this makes the interoperability principle easy to implement."

Maybe the former was just a legacy from MOC 1.0 where ASCII MOC was less formal, perhaps intended for hand-crafted MOCs not intended for serious use. With ASCII MOC now being formalized, can we remove that sentence and require that ASCII MOCs must be just as well-formed as their FITS counterparts?

-- TomDonaldson - 2019-06-14

Added:
>
>
As an ASCII MOC is often written by hand (in a Web form for instance, or as an Aladin script command), it was difficult to insure that it is always well formed at this level. Is it an ASCII MOC, yes, but potentially not well formed (ex: 3/1-210) . So the tools which have to manipulate such ASCII MOCs must check before used. At the opposite, FITS MOC is always generated by a code. So there is a real interest to force it to be stored well-formed for accelerating the future reloads.
That said, you right that it is a little bit inhomogeneous, and the ASCII MOCs will be used in other contexts that just for forms and scripts. So we can considere that an ASCII MOC written by hand is not - strickly speaking an ASCII MOC (not serialized in a file) - and in this consideration - extend the well-formed constraint to all serialized MOC formats (FITS & ASCII) by just removing the sentence.

-- PierreFernique - 2019-06-18

Tom answer 2019-06-21
That would be great. Thank you for doing that.

 


Comments from the IVOA Community during RFC/TCG review period: 2019-06-14 - 2019-07-29

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Changed:
<
<
  • Comment from Carlo Zwölf (CarloMariaZwoelf) : It is not clear for me in this proposed recommendation what comes natively from the Healpix definition and what is the IVOA-specific layer. Is there a way for highlighting this difference in the text?
>
>
  • Comment from Carlo Zwölf (CarloMariaZwoelf) : It is not clear for me in this proposed recommendation what comes natively from the Healpix definition and what is the IVOA-specific layer. Is there a way for highlighting this difference in the text?

    This new document is a very minor modification of the MOC 1.0 REC to standardize the ASCII MOC so that it can be used as quickly as possible in other IVOA standards, including VOregistry. The original text was not modified voluntarily in order to quickly reach a consensus. As your suggestion may imply a significant refactoring of the document, I propose to keep this point in mind for the next major release. It is clear that the introduction of the temporal MOC and the space-time MOC will involve a deep change in the document, especially in the direction you have suggested.
    -- PierreFernique - 2019-09-03
 



Comments from TCG member during the RFC/TCG Review Period: 2019-06-14 - 2019-07-29

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

MOC specification changes to bring String serialization to the normative part looks a good addition to this standard. The document itself is fine in general. I have two small things I'd like to point out.

Changed:
<
<
  • letting [space,CR,LF,comma], multiples/mixes of them, as the elements separator has a human readability effect (see email), not so alerting, but also requires some code in MOC usage. Isn't it really possible to force a bit this serialization to be more smooth with, at least, single and homogeneous separators throughout a single ASCII MOC? Could it be at least warmly suggested?
  • MOCORDER is a MUST and clearly described in the FITS serialization, while in the String (and JSON) one(s) it is less clear how to grab that piece of information. This because it is only said to be a must "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order" (in which case it would be the last token of the String, followed by a slash and no npix). I agree that if it's not so it's implicit the MOCORDER is the last/highest order available, but I'd prefer it to be explicit.
>
>
  • letting [space,CR,LF,comma], multiples/mixes of them, as the elements separator has a human readability effect (see email), not so alerting, but also requires some code in MOC usage. Isn't it really possible to force a bit this serialization to be more smooth with, at least, single and homogeneous separators throughout a single ASCII MOC? Could it be at least warmly suggested?

    The flexibility of the ASCII MOC has been driven by three main pragmatic reasons:
    1- this syntax was exactly the suggested syntax in MOC1.0.
    => consequently there are already existing ASCII MOCs following this syntax,
    => It was easier to keep the syntax as it was suggested in MOC 1.0
    2- an ASCII MOC is often written by hand (in a Web form for instance, or as an Aladin script command),
    => a certain level of tolerance may be useful to avoid rejecting MOC with double space separators...
    3- an ASCII MOC can be a very very long string line
    => As for BASE64 ASCII streaming, it may be really problematic to read/edit an ASCII MOC in a regular ASCII editor without the CR/LF separator and multiple space for possible indentation. Also, we will have serious potential problems for reading and writting a so long line due to the classical flush mecanism of ASCII libs.

    The two first points are managable even if we really want to reduce the flexibility of the ASCII MOC syntax. Concretely the MOC clients/libs will have to be smart enough to support proto ASCII MOCs and possible separator typos (same argument that Tom's point). Quite reasonnable in fact. But I'm strongly in favor to take into account the 3nd point and keep the possibility to split an ASCII MOC if required

    My conclusion would be to remove the "comma", and let space,CR,LF possibly multiple for the space
    -- PierreFernique - 2019-09-03

  • MOCORDER is a MUST and clearly described in the FITS serialization, while in the String (and JSON) one(s) it is less clear how to grab that piece of information. This because it is only said to be a must "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order" (in which case it would be the last token of the String, followed by a slash and no npix). I agree that if it's not so it's implicit the MOCORDER is the last/highest order available, but I'd prefer it to be explicit.

    You are right. It should be a "MUST" without ambiguity.
    The current sentence is : "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order, the MOCORDER must be provided, followed by a slash ("/") without any associated npix value."
    I suggest this new sentence:
    "The best HEALPix resolution of the MOC (MOCORDER) MUST be always provided, even if there is no explicit npix value at this best resolution (due to the MOC hierarchization process). In the latter case, the MOCORDER is specified followed by a slash ("/") without any associated npix value."
    -- PierreFernique - 2019-09-03
 -- MarcoMolinaro - 2019-07-11

Data Model Working Group

Changed:
<
<
  • I support the Marco's comment about MOCORDER in string.
  • It would be nice if figures had a caption and a number used to reference them.
>
>
  • I support the Marco's comment about MOCORDER in string.

    See below -- PierreFernique - 2019-09-03
  • It would be nice if figures had a caption and a number used to reference them.

    Good point. I do -- PierreFernique - 2019-09-03
 -- LaurentMichel - 2019-07-25

Grid & Web Services Working Group

Registry Working Group

Changed:
<
<
I would like to echo Marco's comments above, regarding simplifying the separators in serialization, and perhaps go a bit further. At best, there should be only one way to do this. I am willing to settle for consistency within a MOC using any one of those, no mix and match, with a preferred character separator suggested. -- TheresaDower - 2019-07-24
>
>
I would like to echo Marco's comments above, regarding simplifying the separators in serialization, and perhaps go a bit further. At best, there should be only one way to do this. I am willing to settle for consistency within a MOC using any one of those, no mix and match, with a preferred character separator suggested. -- TheresaDower - 2019-07-24

See below -- PierreFernique - 2019-09-03
 

Semantics Working Group

The Semantics WG doesn't see anything concerning us in the proposed changes (that otherwise sound reasonable and useful to us). The spurious indent of the rule for moc in the EBNF grammar could be removed, though.

-- MarkusDemleitner - 2019-07-16

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Operations

Mostly OK, but some comments:

Changed:
<
<
  • Table of contents: the page numbers are mostly wrong.
>
>
  • Table of contents: the page numbers are mostly wrong.

    Right -> and fixed -- PierreFernique - 2019-09-03
 
Changed:
<
<
  • Sec 2.3.1: The HEALPix FITS encoding standard is mentioned here as well: "As the FITS MOC is derived from a more generic standard dedicated to HEALPix maps, the FITS header must contain the following keywords/value." That's a bit misleading, since the MOC serialization is not a special case of the HEALPix encoding (it doesn't contain some of the keywords required by that standard, such as INDEXSCHM and NSIDE) and some of the keywords have values which are not allowed in the HEALPix encoding (ORDERING='NUNIQ'). Given the difference between the two encodings, I'm not sure there's much point making the comparison here, but if so I'd suggest something more like "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."
>
>
  • Sec 2.3.1: The HEALPix FITS encoding standard is mentioned here as well: "As the FITS MOC is derived from a more generic standard dedicated to HEALPix maps, the FITS header must contain the following keywords/value." That's a bit misleading, since the MOC serialization is not a special case of the HEALPix encoding (it doesn't contain some of the keywords required by that standard, such as INDEXSCHM and NSIDE) and some of the keywords have values which are not allowed in the HEALPix encoding (ORDERING='NUNIQ'). Given the difference between the two encodings, I'm not sure there's much point making the comparison here, but if so I'd suggest something more like "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."
 
  • Sec 2.3.1: The MOCID and ORIGIN keys both "suggest using IVOA identification like ..." where the examples are of the form ivo://... . If these are intended to be IVOA Identifiers, it should say something more like "suggest using an IVOA Identifier ..." with a reference to the IVOA Identifiers document, and the examples given should be valid IVOIDs (i.e. should resolve in the registry - I don't think that "CADC" is an authority in the IVOID sense). If not, the ivo:// prefix seems a bit misleading. Reg WG may want to comment on this.
  • Sec 2.3.2: "hightnpix" -> "highnpix" ?
  • Sec 2.3.2: I'm inclined to agree with Tom's and Marco's comments about the form of the ASCII MOC serialization - it's not clear to me why FITS MOCs have to be well-formed and ASCII ones don't, and why so many separator choices are allowed. Also, the BNF doesn't constrain where the optional MOCORDER marker has to go, or how many of these appear. Consider constraining this format a bit further, e.g. require the MOCORDER marker to be present and/or at the end (or start?) of the string?
  • Appendix B: most of the decimal separators are "." characters, but one is a "," ("1,17").
-- MarkTaylor - 2019-07-13

Standards and Processes Committee


TCG Vote: 2019-06-14 - 2019-07-29

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL *     -- MarcoMolinaro / JamesDempsey 2019-07-17
DM        
GWS        
Registry        
Semantics *      
DCP        
KDIG        
SSIG        
Theory        
TD        
Ops        
StdProc        



<--

-->
  • MOC_ASCII_in_Aladin.png:
    MOC_ASCII_in_Aladin.png

META FILEATTACHMENT attachment="MOC_ASCII_in_Aladin.png" attr="" comment="" date="1558081955" name="MOC_ASCII_in_Aladin.png" path="MOC_ASCII_in_Aladin.png" size="9507" user="PierreFernique" version="1"

Revision 152019-07-25 - LaurentMichel

 
META TOPICPARENT name="MocInfo"

MOC 1.1 Proposed Recommendation: Request for Comments


Summary

Latest Draft: MOC 1.1 (Proposed Recommendation)

The substantive differences between version 1.1 of MOC and the preceding version 1.0 are:

  • The String MOC serialization was moved from an informative section (suggested syntax) to the normative section.
  • A MOCORDER convention for String MOC and JSON MOC was added.

Reference Interoperable Implementations

(Indicate here the links to at least two Reference Interoperable Implementations)

  • MOCPy >=0.5.6
  • Aladin (>v10.126): read/write ASCII MOC (select ASCII format when exporting MOC - see snapshot below) + new script command dedicated to ASCII MOC (ex: draw MOC 3/1,2,3 4/56-67)
  • The relational registry at http://dc.g-vo.org/tap parses the ASCII MOCs in Registry records into the rr.stc_spatial table (consumer)
  • DaCHS since version 1.1 produces ASCII MOCs when serialising MOC-values database columns.
  • The Moc class within the AST/PyAST WCS library now supports reading and writing ASCII and JSON MOCs in addition to FITS.
To see non-trivial examples for ASCII MOCs coming out of a database, run something like

select top 100 * from rr.stc_spatial
where 1=contains(point(10, 10), coverage)
and 0=contains(point(20, 20), coverage)

on http://dc.g-vo.org/tap.

Implementations Validators

Comments Prior to Official RFC Period

As I was considering semantic validation cases, I noticed this sentence in section 2.3.2:

"Warning: In this basic simple ASCII string format the values may be not sorted, and the MOC may be not well-formed."

which seems to contradict section 2.2.3:

" An encoded MOC must also be “well-formed”. To keep the canonical property of MOC, the redundant cells are not allowed and the hierarchical encoding principle must be respected. Thus it is not allowed to encode 4 sibling cells instead of their parent (only at level 0 for the 12 first diamonds). Using this simple constraint, there is one and only one way to describe a specific region and this makes the interoperability principle easy to implement."

Maybe the former was just a legacy from MOC 1.0 where ASCII MOC was less formal, perhaps intended for hand-crafted MOCs not intended for serious use. With ASCII MOC now being formalized, can we remove that sentence and require that ASCII MOCs must be just as well-formed as their FITS counterparts?

-- TomDonaldson - 2019-06-14




Comments from the IVOA Community during RFC/TCG review period: 2019-06-14 - 2019-07-29

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

  • Comment from Carlo Zwölf (CarloMariaZwoelf) : It is not clear for me in this proposed recommendation what comes natively from the Healpix definition and what is the IVOA-specific layer. Is there a way for highlighting this difference in the text?



Comments from TCG member during the RFC/TCG Review Period: 2019-06-14 - 2019-07-29

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

MOC specification changes to bring String serialization to the normative part looks a good addition to this standard. The document itself is fine in general. I have two small things I'd like to point out.

  • letting [space,CR,LF,comma], multiples/mixes of them, as the elements separator has a human readability effect (see email), not so alerting, but also requires some code in MOC usage. Isn't it really possible to force a bit this serialization to be more smooth with, at least, single and homogeneous separators throughout a single ASCII MOC? Could it be at least warmly suggested?
  • MOCORDER is a MUST and clearly described in the FITS serialization, while in the String (and JSON) one(s) it is less clear how to grab that piece of information. This because it is only said to be a must "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order" (in which case it would be the last token of the String, followed by a slash and no npix). I agree that if it's not so it's implicit the MOCORDER is the last/highest order available, but I'd prefer it to be explicit.
-- MarcoMolinaro - 2019-07-11

Data Model Working Group

Changed:
<
<
>
>
  • I support the Marco's comment about MOCORDER in string.
Added:
>
>
  • It would be nice if figures had a caption and a number used to reference them.
-- LaurentMichel - 2019-07-25
 

Grid & Web Services Working Group

Registry Working Group

I would like to echo Marco's comments above, regarding simplifying the separators in serialization, and perhaps go a bit further. At best, there should be only one way to do this. I am willing to settle for consistency within a MOC using any one of those, no mix and match, with a preferred character separator suggested. -- TheresaDower - 2019-07-24

Semantics Working Group

The Semantics WG doesn't see anything concerning us in the proposed changes (that otherwise sound reasonable and useful to us). The spurious indent of the rule for moc in the EBNF grammar could be removed, though.

-- MarkusDemleitner - 2019-07-16

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Operations

Mostly OK, but some comments:

  • Table of contents: the page numbers are mostly wrong.
  • Sec 1.4: The existence of a FITS HEALPix serialization format on which the MOC serialization format is based is claimed, but no reference is given. Since this text (written by Eric Hivon) does now just about exist in a public form https://healpix.sourceforge.io/data/examples/healpix_fits_specs.pdf, can it be referenced here?
  • Sec 2.3.1: The HEALPix FITS encoding standard is mentioned here as well: "As the FITS MOC is derived from a more generic standard dedicated to HEALPix maps, the FITS header must contain the following keywords/value." That's a bit misleading, since the MOC serialization is not a special case of the HEALPix encoding (it doesn't contain some of the keywords required by that standard, such as INDEXSCHM and NSIDE) and some of the keywords have values which are not allowed in the HEALPix encoding (ORDERING='NUNIQ'). Given the difference between the two encodings, I'm not sure there's much point making the comparison here, but if so I'd suggest something more like "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."
  • Sec 2.3.1: The MOCID and ORIGIN keys both "suggest using IVOA identification like ..." where the examples are of the form ivo://... . If these are intended to be IVOA Identifiers, it should say something more like "suggest using an IVOA Identifier ..." with a reference to the IVOA Identifiers document, and the examples given should be valid IVOIDs (i.e. should resolve in the registry - I don't think that "CADC" is an authority in the IVOID sense). If not, the ivo:// prefix seems a bit misleading. Reg WG may want to comment on this.
  • Sec 2.3.2: "hightnpix" -> "highnpix" ?
  • Sec 2.3.2: I'm inclined to agree with Tom's and Marco's comments about the form of the ASCII MOC serialization - it's not clear to me why FITS MOCs have to be well-formed and ASCII ones don't, and why so many separator choices are allowed. Also, the BNF doesn't constrain where the optional MOCORDER marker has to go, or how many of these appear. Consider constraining this format a bit further, e.g. require the MOCORDER marker to be present and/or at the end (or start?) of the string?
  • Appendix B: most of the decimal separators are "." characters, but one is a "," ("1,17").
-- MarkTaylor - 2019-07-13

Standards and Processes Committee


TCG Vote: 2019-06-14 - 2019-07-29

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL *     -- MarcoMolinaro / JamesDempsey 2019-07-17
DM        
GWS        
Registry        
Semantics *      
DCP        
KDIG        
SSIG        
Theory        
TD        
Ops        
StdProc        



<--

-->
  • MOC_ASCII_in_Aladin.png:
    MOC_ASCII_in_Aladin.png

META FILEATTACHMENT attachment="MOC_ASCII_in_Aladin.png" attr="" comment="" date="1558081955" name="MOC_ASCII_in_Aladin.png" path="MOC_ASCII_in_Aladin.png" size="9507" user="PierreFernique" version="1"

Revision 142019-07-24 - TheresaDower

 
META TOPICPARENT name="MocInfo"

MOC 1.1 Proposed Recommendation: Request for Comments


Summary

Latest Draft: MOC 1.1 (Proposed Recommendation)

The substantive differences between version 1.1 of MOC and the preceding version 1.0 are:

  • The String MOC serialization was moved from an informative section (suggested syntax) to the normative section.
  • A MOCORDER convention for String MOC and JSON MOC was added.

Reference Interoperable Implementations

(Indicate here the links to at least two Reference Interoperable Implementations)

  • MOCPy >=0.5.6
  • Aladin (>v10.126): read/write ASCII MOC (select ASCII format when exporting MOC - see snapshot below) + new script command dedicated to ASCII MOC (ex: draw MOC 3/1,2,3 4/56-67)
  • The relational registry at http://dc.g-vo.org/tap parses the ASCII MOCs in Registry records into the rr.stc_spatial table (consumer)
  • DaCHS since version 1.1 produces ASCII MOCs when serialising MOC-values database columns.
  • The Moc class within the AST/PyAST WCS library now supports reading and writing ASCII and JSON MOCs in addition to FITS.
To see non-trivial examples for ASCII MOCs coming out of a database, run something like

select top 100 * from rr.stc_spatial
where 1=contains(point(10, 10), coverage)
and 0=contains(point(20, 20), coverage)

on http://dc.g-vo.org/tap.

Implementations Validators

Comments Prior to Official RFC Period

As I was considering semantic validation cases, I noticed this sentence in section 2.3.2:

"Warning: In this basic simple ASCII string format the values may be not sorted, and the MOC may be not well-formed."

which seems to contradict section 2.2.3:

" An encoded MOC must also be “well-formed”. To keep the canonical property of MOC, the redundant cells are not allowed and the hierarchical encoding principle must be respected. Thus it is not allowed to encode 4 sibling cells instead of their parent (only at level 0 for the 12 first diamonds). Using this simple constraint, there is one and only one way to describe a specific region and this makes the interoperability principle easy to implement."

Maybe the former was just a legacy from MOC 1.0 where ASCII MOC was less formal, perhaps intended for hand-crafted MOCs not intended for serious use. With ASCII MOC now being formalized, can we remove that sentence and require that ASCII MOCs must be just as well-formed as their FITS counterparts?

-- TomDonaldson - 2019-06-14




Comments from the IVOA Community during RFC/TCG review period: 2019-06-14 - 2019-07-29

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

  • Comment from Carlo Zwölf (CarloMariaZwoelf) : It is not clear for me in this proposed recommendation what comes natively from the Healpix definition and what is the IVOA-specific layer. Is there a way for highlighting this difference in the text?



Comments from TCG member during the RFC/TCG Review Period: 2019-06-14 - 2019-07-29

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

MOC specification changes to bring String serialization to the normative part looks a good addition to this standard. The document itself is fine in general. I have two small things I'd like to point out.

  • letting [space,CR,LF,comma], multiples/mixes of them, as the elements separator has a human readability effect (see email), not so alerting, but also requires some code in MOC usage. Isn't it really possible to force a bit this serialization to be more smooth with, at least, single and homogeneous separators throughout a single ASCII MOC? Could it be at least warmly suggested?
  • MOCORDER is a MUST and clearly described in the FITS serialization, while in the String (and JSON) one(s) it is less clear how to grab that piece of information. This because it is only said to be a must "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order" (in which case it would be the last token of the String, followed by a slash and no npix). I agree that if it's not so it's implicit the MOCORDER is the last/highest order available, but I'd prefer it to be explicit.
-- MarcoMolinaro - 2019-07-11

Data Model Working Group

Grid & Web Services Working Group

Registry Working Group

Added:
>
>
I would like to echo Marco's comments above, regarding simplifying the separators in serialization, and perhaps go a bit further. At best, there should be only one way to do this. I am willing to settle for consistency within a MOC using any one of those, no mix and match, with a preferred character separator suggested. -- TheresaDower - 2019-07-24
 

Semantics Working Group

Changed:
<
<
The Semantics WG doesn't see anything concerning us in the proposed
>
>
The Semantics WG doesn't see anything concerning us in the proposed changes (that otherwise sound reasonable and useful to us). The spurious indent of the rule for moc in the EBNF grammar could be removed, though.
Deleted:
<
<
changes (that otherwise sound reasonable and useful to us). The spurious indent of the rule for moc in the EBNF grammar could be removed, though.
  -- MarkusDemleitner - 2019-07-16

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Operations

Mostly OK, but some comments:

  • Table of contents: the page numbers are mostly wrong.
  • Sec 1.4: The existence of a FITS HEALPix serialization format on which the MOC serialization format is based is claimed, but no reference is given. Since this text (written by Eric Hivon) does now just about exist in a public form https://healpix.sourceforge.io/data/examples/healpix_fits_specs.pdf, can it be referenced here?
  • Sec 2.3.1: The HEALPix FITS encoding standard is mentioned here as well: "As the FITS MOC is derived from a more generic standard dedicated to HEALPix maps, the FITS header must contain the following keywords/value." That's a bit misleading, since the MOC serialization is not a special case of the HEALPix encoding (it doesn't contain some of the keywords required by that standard, such as INDEXSCHM and NSIDE) and some of the keywords have values which are not allowed in the HEALPix encoding (ORDERING='NUNIQ'). Given the difference between the two encodings, I'm not sure there's much point making the comparison here, but if so I'd suggest something more like "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."
  • Sec 2.3.1: The MOCID and ORIGIN keys both "suggest using IVOA identification like ..." where the examples are of the form ivo://... . If these are intended to be IVOA Identifiers, it should say something more like "suggest using an IVOA Identifier ..." with a reference to the IVOA Identifiers document, and the examples given should be valid IVOIDs (i.e. should resolve in the registry - I don't think that "CADC" is an authority in the IVOID sense). If not, the ivo:// prefix seems a bit misleading. Reg WG may want to comment on this.
  • Sec 2.3.2: "hightnpix" -> "highnpix" ?
  • Sec 2.3.2: I'm inclined to agree with Tom's and Marco's comments about the form of the ASCII MOC serialization - it's not clear to me why FITS MOCs have to be well-formed and ASCII ones don't, and why so many separator choices are allowed. Also, the BNF doesn't constrain where the optional MOCORDER marker has to go, or how many of these appear. Consider constraining this format a bit further, e.g. require the MOCORDER marker to be present and/or at the end (or start?) of the string?
  • Appendix B: most of the decimal separators are "." characters, but one is a "," ("1,17").
-- MarkTaylor - 2019-07-13

Standards and Processes Committee


TCG Vote: 2019-06-14 - 2019-07-29

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL *     -- MarcoMolinaro / JamesDempsey 2019-07-17
DM        
GWS        
Registry        
Semantics *      
DCP        
KDIG        
SSIG        
Theory        
TD        
Ops        
StdProc        



<--

-->
Changed:
<
<
  • MOC_ASCII_in_Aladin.png:
    MOC_ASCII_in_Aladin.png
>
>
  • MOC_ASCII_in_Aladin.png:
    MOC_ASCII_in_Aladin.png
 
META FILEATTACHMENT attachment="MOC_ASCII_in_Aladin.png" attr="" comment="" date="1558081955" name="MOC_ASCII_in_Aladin.png" path="MOC_ASCII_in_Aladin.png" size="9507" user="PierreFernique" version="1"

Revision 132019-07-17 - MarcoMolinaro

 
META TOPICPARENT name="MocInfo"

MOC 1.1 Proposed Recommendation: Request for Comments


Summary

Latest Draft: MOC 1.1 (Proposed Recommendation)

The substantive differences between version 1.1 of MOC and the preceding version 1.0 are:

  • The String MOC serialization was moved from an informative section (suggested syntax) to the normative section.
  • A MOCORDER convention for String MOC and JSON MOC was added.

Reference Interoperable Implementations

(Indicate here the links to at least two Reference Interoperable Implementations)

  • MOCPy >=0.5.6
  • Aladin (>v10.126): read/write ASCII MOC (select ASCII format when exporting MOC - see snapshot below) + new script command dedicated to ASCII MOC (ex: draw MOC 3/1,2,3 4/56-67)
  • The relational registry at http://dc.g-vo.org/tap parses the ASCII MOCs in Registry records into the rr.stc_spatial table (consumer)
  • DaCHS since version 1.1 produces ASCII MOCs when serialising MOC-values database columns.
  • The Moc class within the AST/PyAST WCS library now supports reading and writing ASCII and JSON MOCs in addition to FITS.
To see non-trivial examples for ASCII MOCs coming out of a database, run something like

select top 100 * from rr.stc_spatial
where 1=contains(point(10, 10), coverage)
and 0=contains(point(20, 20), coverage)

on http://dc.g-vo.org/tap.

Implementations Validators

Comments Prior to Official RFC Period

As I was considering semantic validation cases, I noticed this sentence in section 2.3.2:

"Warning: In this basic simple ASCII string format the values may be not sorted, and the MOC may be not well-formed."

which seems to contradict section 2.2.3:

" An encoded MOC must also be “well-formed”. To keep the canonical property of MOC, the redundant cells are not allowed and the hierarchical encoding principle must be respected. Thus it is not allowed to encode 4 sibling cells instead of their parent (only at level 0 for the 12 first diamonds). Using this simple constraint, there is one and only one way to describe a specific region and this makes the interoperability principle easy to implement."

Maybe the former was just a legacy from MOC 1.0 where ASCII MOC was less formal, perhaps intended for hand-crafted MOCs not intended for serious use. With ASCII MOC now being formalized, can we remove that sentence and require that ASCII MOCs must be just as well-formed as their FITS counterparts?

-- TomDonaldson - 2019-06-14




Comments from the IVOA Community during RFC/TCG review period: 2019-06-14 - 2019-07-29

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

  • Comment from Carlo Zwölf (CarloMariaZwoelf) : It is not clear for me in this proposed recommendation what comes natively from the Healpix definition and what is the IVOA-specific layer. Is there a way for highlighting this difference in the text?



Comments from TCG member during the RFC/TCG Review Period: 2019-06-14 - 2019-07-29

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

MOC specification changes to bring String serialization to the normative part looks a good addition to this standard. The document itself is fine in general. I have two small things I'd like to point out.

  • letting [space,CR,LF,comma], multiples/mixes of them, as the elements separator has a human readability effect (see email), not so alerting, but also requires some code in MOC usage. Isn't it really possible to force a bit this serialization to be more smooth with, at least, single and homogeneous separators throughout a single ASCII MOC? Could it be at least warmly suggested?
  • MOCORDER is a MUST and clearly described in the FITS serialization, while in the String (and JSON) one(s) it is less clear how to grab that piece of information. This because it is only said to be a must "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order" (in which case it would be the last token of the String, followed by a slash and no npix). I agree that if it's not so it's implicit the MOCORDER is the last/highest order available, but I'd prefer it to be explicit.
-- MarcoMolinaro - 2019-07-11

Data Model Working Group

Grid & Web Services Working Group

Registry Working Group

Semantics Working Group

The Semantics WG doesn't see anything concerning us in the proposed changes (that otherwise sound reasonable and useful to us). The spurious indent of the rule for moc in the EBNF grammar could be removed, though.

-- MarkusDemleitner - 2019-07-16

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Operations

Mostly OK, but some comments:

  • Table of contents: the page numbers are mostly wrong.
  • Sec 1.4: The existence of a FITS HEALPix serialization format on which the MOC serialization format is based is claimed, but no reference is given. Since this text (written by Eric Hivon) does now just about exist in a public form https://healpix.sourceforge.io/data/examples/healpix_fits_specs.pdf, can it be referenced here?
  • Sec 2.3.1: The HEALPix FITS encoding standard is mentioned here as well: "As the FITS MOC is derived from a more generic standard dedicated to HEALPix maps, the FITS header must contain the following keywords/value." That's a bit misleading, since the MOC serialization is not a special case of the HEALPix encoding (it doesn't contain some of the keywords required by that standard, such as INDEXSCHM and NSIDE) and some of the keywords have values which are not allowed in the HEALPix encoding (ORDERING='NUNIQ'). Given the difference between the two encodings, I'm not sure there's much point making the comparison here, but if so I'd suggest something more like "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."
  • Sec 2.3.1: The MOCID and ORIGIN keys both "suggest using IVOA identification like ..." where the examples are of the form ivo://... . If these are intended to be IVOA Identifiers, it should say something more like "suggest using an IVOA Identifier ..." with a reference to the IVOA Identifiers document, and the examples given should be valid IVOIDs (i.e. should resolve in the registry - I don't think that "CADC" is an authority in the IVOID sense). If not, the ivo:// prefix seems a bit misleading. Reg WG may want to comment on this.
  • Sec 2.3.2: "hightnpix" -> "highnpix" ?
  • Sec 2.3.2: I'm inclined to agree with Tom's and Marco's comments about the form of the ASCII MOC serialization - it's not clear to me why FITS MOCs have to be well-formed and ASCII ones don't, and why so many separator choices are allowed. Also, the BNF doesn't constrain where the optional MOCORDER marker has to go, or how many of these appear. Consider constraining this format a bit further, e.g. require the MOCORDER marker to be present and/or at the end (or start?) of the string?
  • Appendix B: most of the decimal separators are "." characters, but one is a "," ("1,17").
-- MarkTaylor - 2019-07-13

Standards and Processes Committee


TCG Vote: 2019-06-14 - 2019-07-29

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
Changed:
<
<
DAL        
>
>
DAL *     -- MarcoMolinaro / JamesDempsey 2019-07-17
 
DM        
GWS        
Registry        
Semantics *      
DCP        
KDIG        
SSIG        
Theory        
TD        
Ops        
StdProc        



<--

-->
  • MOC_ASCII_in_Aladin.png:
    MOC_ASCII_in_Aladin.png

META FILEATTACHMENT attachment="MOC_ASCII_in_Aladin.png" attr="" comment="" date="1558081955" name="MOC_ASCII_in_Aladin.png" path="MOC_ASCII_in_Aladin.png" size="9507" user="PierreFernique" version="1"

Revision 122019-07-16 - MarkusDemleitner

 
META TOPICPARENT name="MocInfo"

MOC 1.1 Proposed Recommendation: Request for Comments


Summary

Latest Draft: MOC 1.1 (Proposed Recommendation)

The substantive differences between version 1.1 of MOC and the preceding version 1.0 are:

  • The String MOC serialization was moved from an informative section (suggested syntax) to the normative section.
  • A MOCORDER convention for String MOC and JSON MOC was added.

Reference Interoperable Implementations

(Indicate here the links to at least two Reference Interoperable Implementations)

  • MOCPy >=0.5.6
  • Aladin (>v10.126): read/write ASCII MOC (select ASCII format when exporting MOC - see snapshot below) + new script command dedicated to ASCII MOC (ex: draw MOC 3/1,2,3 4/56-67)
  • The relational registry at http://dc.g-vo.org/tap parses the ASCII MOCs in Registry records into the rr.stc_spatial table (consumer)
  • DaCHS since version 1.1 produces ASCII MOCs when serialising MOC-values database columns.
  • The Moc class within the AST/PyAST WCS library now supports reading and writing ASCII and JSON MOCs in addition to FITS.
To see non-trivial examples for ASCII MOCs coming out of a database, run something like

select top 100 * from rr.stc_spatial
where 1=contains(point(10, 10), coverage)
and 0=contains(point(20, 20), coverage)

on http://dc.g-vo.org/tap.

Implementations Validators

Comments Prior to Official RFC Period

As I was considering semantic validation cases, I noticed this sentence in section 2.3.2:

"Warning: In this basic simple ASCII string format the values may be not sorted, and the MOC may be not well-formed."

which seems to contradict section 2.2.3:

" An encoded MOC must also be “well-formed”. To keep the canonical property of MOC, the redundant cells are not allowed and the hierarchical encoding principle must be respected. Thus it is not allowed to encode 4 sibling cells instead of their parent (only at level 0 for the 12 first diamonds). Using this simple constraint, there is one and only one way to describe a specific region and this makes the interoperability principle easy to implement."

Maybe the former was just a legacy from MOC 1.0 where ASCII MOC was less formal, perhaps intended for hand-crafted MOCs not intended for serious use. With ASCII MOC now being formalized, can we remove that sentence and require that ASCII MOCs must be just as well-formed as their FITS counterparts?

-- TomDonaldson - 2019-06-14




Comments from the IVOA Community during RFC/TCG review period: 2019-06-14 - 2019-07-29

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Changed:
<
<
  • Comment from Carlo Zwölf (CarloMariaZwoelf) : It is not clear for me in this proposed recommendation what comes natively from the Healpix definition and what is the IVOA-specific layer. Is there a way for highlighting this difference in the text?
>
>
  • Comment from Carlo Zwölf (CarloMariaZwoelf) : It is not clear for me in this proposed recommendation what comes natively from the Healpix definition and what is the IVOA-specific layer. Is there a way for highlighting this difference in the text?
 



Comments from TCG member during the RFC/TCG Review Period: 2019-06-14 - 2019-07-29

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

MOC specification changes to bring String serialization to the normative part looks a good addition to this standard. The document itself is fine in general. I have two small things I'd like to point out.

  • letting [space,CR,LF,comma], multiples/mixes of them, as the elements separator has a human readability effect (see email), not so alerting, but also requires some code in MOC usage. Isn't it really possible to force a bit this serialization to be more smooth with, at least, single and homogeneous separators throughout a single ASCII MOC? Could it be at least warmly suggested?
  • MOCORDER is a MUST and clearly described in the FITS serialization, while in the String (and JSON) one(s) it is less clear how to grab that piece of information. This because it is only said to be a must "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order" (in which case it would be the last token of the String, followed by a slash and no npix). I agree that if it's not so it's implicit the MOCORDER is the last/highest order available, but I'd prefer it to be explicit.
-- MarcoMolinaro - 2019-07-11

Data Model Working Group

Grid & Web Services Working Group

Registry Working Group

Semantics Working Group

Added:
>
>
The Semantics WG doesn't see anything concerning us in the proposed changes (that otherwise sound reasonable and useful to us). The spurious indent of the rule for moc in the EBNF grammar could be removed, though.

-- MarkusDemleitner - 2019-07-16

 

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Operations

Mostly OK, but some comments:

  • Table of contents: the page numbers are mostly wrong.
Changed:
<
<
  • Sec 1.4: The existence of a FITS HEALPix serialization format on which the MOC serialization format is based is claimed, but no reference is given. Since this text (written by Eric Hivon) does now just about exist in a public form https://healpix.sourceforge.io/data/examples/healpix_fits_specs.pdf, can it be referenced here?
  • Sec 2.3.1: The HEALPix FITS encoding standard is mentioned here as well: "As the FITS MOC is derived from a more generic standard dedicated to HEALPix maps, the FITS header must contain the following keywords/value." That's a bit misleading, since the MOC serialization is not a special case of the HEALPix encoding (it doesn't contain some of the keywords required by that standard, such as INDEXSCHM and NSIDE) and some of the keywords have values which are not allowed in the HEALPix encoding (ORDERING='NUNIQ'). Given the difference between the two encodings, I'm not sure there's much point making the comparison here, but if so I'd suggest something more like "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."
  • Sec 2.3.1: The MOCID and ORIGIN keys both "suggest using IVOA identification like ..." where the examples are of the form ivo://... . If these are intended to be IVOA Identifiers, it should say something more like "suggest using an IVOA Identifier ..." with a reference to the IVOA Identifiers document, and the examples given should be valid IVOIDs (i.e. should resolve in the registry - I don't think that "CADC" is an authority in the IVOID sense). If not, the ivo:// prefix seems a bit misleading. Reg WG may want to comment on this.
  • Sec 2.3.2: "hightnpix" -> "highnpix" ?
  • Sec 2.3.2: I'm inclined to agree with Tom's and Marco's comments about the form of the ASCII MOC serialization - it's not clear to me why FITS MOCs have to be well-formed and ASCII ones don't, and why so many separator choices are allowed. Also, the BNF doesn't constrain where the optional MOCORDER marker has to go, or how many of these appear. Consider constraining this format a bit further, e.g. require the MOCORDER marker to be present and/or at the end (or start?) of the string?
  • Appendix B: most of the decimal separators are "." characters, but one is a "," ("1,17").
>
>
  • Sec 1.4: The existence of a FITS HEALPix serialization format on which the MOC serialization format is based is claimed, but no reference is given. Since this text (written by Eric Hivon) does now just about exist in a public form https://healpix.sourceforge.io/data/examples/healpix_fits_specs.pdf, can it be referenced here?
  • Sec 2.3.1: The HEALPix FITS encoding standard is mentioned here as well: "As the FITS MOC is derived from a more generic standard dedicated to HEALPix maps, the FITS header must contain the following keywords/value." That's a bit misleading, since the MOC serialization is not a special case of the HEALPix encoding (it doesn't contain some of the keywords required by that standard, such as INDEXSCHM and NSIDE) and some of the keywords have values which are not allowed in the HEALPix encoding (ORDERING='NUNIQ'). Given the difference between the two encodings, I'm not sure there's much point making the comparison here, but if so I'd suggest something more like "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."
  • Sec 2.3.1: The MOCID and ORIGIN keys both "suggest using IVOA identification like ..." where the examples are of the form ivo://... . If these are intended to be IVOA Identifiers, it should say something more like "suggest using an IVOA Identifier ..." with a reference to the IVOA Identifiers document, and the examples given should be valid IVOIDs (i.e. should resolve in the registry - I don't think that "CADC" is an authority in the IVOID sense). If not, the ivo:// prefix seems a bit misleading. Reg WG may want to comment on this.
  • Sec 2.3.2: "hightnpix" -> "highnpix" ?
  • Sec 2.3.2: I'm inclined to agree with Tom's and Marco's comments about the form of the ASCII MOC serialization - it's not clear to me why FITS MOCs have to be well-formed and ASCII ones don't, and why so many separator choices are allowed. Also, the BNF doesn't constrain where the optional MOCORDER marker has to go, or how many of these appear. Consider constraining this format a bit further, e.g. require the MOCORDER marker to be present and/or at the end (or start?) of the string?
  • Appendix B: most of the decimal separators are "." characters, but one is a "," ("1,17").
 -- MarkTaylor - 2019-07-13

Standards and Processes Committee


TCG Vote: 2019-06-14 - 2019-07-29

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL        
DM        
GWS        
Registry        
Changed:
<
<
Semantics        
>
>
Semantics *      
 
DCP        
KDIG        
SSIG        
Theory        
TD        
Ops        
StdProc        



<--

-->
Changed:
<
<
  • MOC_ASCII_in_Aladin.png:
    MOC_ASCII_in_Aladin.png
>
>
  • MOC_ASCII_in_Aladin.png:
    MOC_ASCII_in_Aladin.png
 
META FILEATTACHMENT attachment="MOC_ASCII_in_Aladin.png" attr="" comment="" date="1558081955" name="MOC_ASCII_in_Aladin.png" path="MOC_ASCII_in_Aladin.png" size="9507" user="PierreFernique" version="1"

Revision 112019-07-13 - MarkTaylor

 
META TOPICPARENT name="MocInfo"

MOC 1.1 Proposed Recommendation: Request for Comments


Summary

Latest Draft: MOC 1.1 (Proposed Recommendation)

The substantive differences between version 1.1 of MOC and the preceding version 1.0 are:

  • The String MOC serialization was moved from an informative section (suggested syntax) to the normative section.
  • A MOCORDER convention for String MOC and JSON MOC was added.

Reference Interoperable Implementations

(Indicate here the links to at least two Reference Interoperable Implementations)

  • MOCPy >=0.5.6
  • Aladin (>v10.126): read/write ASCII MOC (select ASCII format when exporting MOC - see snapshot below) + new script command dedicated to ASCII MOC (ex: draw MOC 3/1,2,3 4/56-67)
  • The relational registry at http://dc.g-vo.org/tap parses the ASCII MOCs in Registry records into the rr.stc_spatial table (consumer)
  • DaCHS since version 1.1 produces ASCII MOCs when serialising MOC-values database columns.
  • The Moc class within the AST/PyAST WCS library now supports reading and writing ASCII and JSON MOCs in addition to FITS.
To see non-trivial examples for ASCII MOCs coming out of a database, run something like

select top 100 * from rr.stc_spatial
where 1=contains(point(10, 10), coverage)
and 0=contains(point(20, 20), coverage)

on http://dc.g-vo.org/tap.

Implementations Validators

Comments Prior to Official RFC Period

As I was considering semantic validation cases, I noticed this sentence in section 2.3.2:

"Warning: In this basic simple ASCII string format the values may be not sorted, and the MOC may be not well-formed."

which seems to contradict section 2.2.3:

" An encoded MOC must also be “well-formed”. To keep the canonical property of MOC, the redundant cells are not allowed and the hierarchical encoding principle must be respected. Thus it is not allowed to encode 4 sibling cells instead of their parent (only at level 0 for the 12 first diamonds). Using this simple constraint, there is one and only one way to describe a specific region and this makes the interoperability principle easy to implement."

Maybe the former was just a legacy from MOC 1.0 where ASCII MOC was less formal, perhaps intended for hand-crafted MOCs not intended for serious use. With ASCII MOC now being formalized, can we remove that sentence and require that ASCII MOCs must be just as well-formed as their FITS counterparts?

-- TomDonaldson - 2019-06-14




Comments from the IVOA Community during RFC/TCG review period: 2019-06-14 - 2019-07-29

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

  • Comment from Carlo Zwölf (CarloMariaZwoelf) : It is not clear for me in this proposed recommendation what comes natively from the Healpix definition and what is the IVOA-specific layer. Is there a way for highlighting this difference in the text?



Comments from TCG member during the RFC/TCG Review Period: 2019-06-14 - 2019-07-29

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

MOC specification changes to bring String serialization to the normative part looks a good addition to this standard. The document itself is fine in general. I have two small things I'd like to point out.

  • letting [space,CR,LF,comma], multiples/mixes of them, as the elements separator has a human readability effect (see email), not so alerting, but also requires some code in MOC usage. Isn't it really possible to force a bit this serialization to be more smooth with, at least, single and homogeneous separators throughout a single ASCII MOC? Could it be at least warmly suggested?
  • MOCORDER is a MUST and clearly described in the FITS serialization, while in the String (and JSON) one(s) it is less clear how to grab that piece of information. This because it is only said to be a must "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order" (in which case it would be the last token of the String, followed by a slash and no npix). I agree that if it's not so it's implicit the MOCORDER is the last/highest order available, but I'd prefer it to be explicit.
-- MarcoMolinaro - 2019-07-11

Data Model Working Group

Grid & Web Services Working Group

Registry Working Group

Semantics Working Group

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Operations

Added:
>
>
Mostly OK, but some comments:
  • Table of contents: the page numbers are mostly wrong.
  • Sec 1.4: The existence of a FITS HEALPix serialization format on which the MOC serialization format is based is claimed, but no reference is given. Since this text (written by Eric Hivon) does now just about exist in a public form https://healpix.sourceforge.io/data/examples/healpix_fits_specs.pdf, can it be referenced here?
  • Sec 2.3.1: The HEALPix FITS encoding standard is mentioned here as well: "As the FITS MOC is derived from a more generic standard dedicated to HEALPix maps, the FITS header must contain the following keywords/value." That's a bit misleading, since the MOC serialization is not a special case of the HEALPix encoding (it doesn't contain some of the keywords required by that standard, such as INDEXSCHM and NSIDE) and some of the keywords have values which are not allowed in the HEALPix encoding (ORDERING='NUNIQ'). Given the difference between the two encodings, I'm not sure there's much point making the comparison here, but if so I'd suggest something more like "The following keywords are mandatory and derive from the HEALPix MOC encoding [with a document reference]."
  • Sec 2.3.1: The MOCID and ORIGIN keys both "suggest using IVOA identification like ..." where the examples are of the form ivo://... . If these are intended to be IVOA Identifiers, it should say something more like "suggest using an IVOA Identifier ..." with a reference to the IVOA Identifiers document, and the examples given should be valid IVOIDs (i.e. should resolve in the registry - I don't think that "CADC" is an authority in the IVOID sense). If not, the ivo:// prefix seems a bit misleading. Reg WG may want to comment on this.
  • Sec 2.3.2: "hightnpix" -> "highnpix" ?
  • Sec 2.3.2: I'm inclined to agree with Tom's and Marco's comments about the form of the ASCII MOC serialization - it's not clear to me why FITS MOCs have to be well-formed and ASCII ones don't, and why so many separator choices are allowed. Also, the BNF doesn't constrain where the optional MOCORDER marker has to go, or how many of these appear. Consider constraining this format a bit further, e.g. require the MOCORDER marker to be present and/or at the end (or start?) of the string?
  • Appendix B: most of the decimal separators are "." characters, but one is a "," ("1,17").
-- MarkTaylor - 2019-07-13
 

Standards and Processes Committee


TCG Vote: 2019-06-14 - 2019-07-29

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL        
DM        
GWS        
Registry        
Semantics        
DCP        
KDIG        
SSIG        
Theory        
TD        
Ops        
StdProc        



<--

-->
  • MOC_ASCII_in_Aladin.png:
    MOC_ASCII_in_Aladin.png

META FILEATTACHMENT attachment="MOC_ASCII_in_Aladin.png" attr="" comment="" date="1558081955" name="MOC_ASCII_in_Aladin.png" path="MOC_ASCII_in_Aladin.png" size="9507" user="PierreFernique" version="1"

Revision 102019-07-11 - CarloMariaZwoelf

 
META TOPICPARENT name="MocInfo"

MOC 1.1 Proposed Recommendation: Request for Comments


Summary

Latest Draft: MOC 1.1 (Proposed Recommendation)

The substantive differences between version 1.1 of MOC and the preceding version 1.0 are:

  • The String MOC serialization was moved from an informative section (suggested syntax) to the normative section.
  • A MOCORDER convention for String MOC and JSON MOC was added.

Reference Interoperable Implementations

(Indicate here the links to at least two Reference Interoperable Implementations)

  • MOCPy >=0.5.6
  • Aladin (>v10.126): read/write ASCII MOC (select ASCII format when exporting MOC - see snapshot below) + new script command dedicated to ASCII MOC (ex: draw MOC 3/1,2,3 4/56-67)
  • The relational registry at http://dc.g-vo.org/tap parses the ASCII MOCs in Registry records into the rr.stc_spatial table (consumer)
  • DaCHS since version 1.1 produces ASCII MOCs when serialising MOC-values database columns.
  • The Moc class within the AST/PyAST WCS library now supports reading and writing ASCII and JSON MOCs in addition to FITS.
To see non-trivial examples for ASCII MOCs coming out of a database, run something like

select top 100 * from rr.stc_spatial
where 1=contains(point(10, 10), coverage)
and 0=contains(point(20, 20), coverage)

on http://dc.g-vo.org/tap.

Implementations Validators

Comments Prior to Official RFC Period

As I was considering semantic validation cases, I noticed this sentence in section 2.3.2:

"Warning: In this basic simple ASCII string format the values may be not sorted, and the MOC may be not well-formed."

which seems to contradict section 2.2.3:

" An encoded MOC must also be “well-formed”. To keep the canonical property of MOC, the redundant cells are not allowed and the hierarchical encoding principle must be respected. Thus it is not allowed to encode 4 sibling cells instead of their parent (only at level 0 for the 12 first diamonds). Using this simple constraint, there is one and only one way to describe a specific region and this makes the interoperability principle easy to implement."

Maybe the former was just a legacy from MOC 1.0 where ASCII MOC was less formal, perhaps intended for hand-crafted MOCs not intended for serious use. With ASCII MOC now being formalized, can we remove that sentence and require that ASCII MOCs must be just as well-formed as their FITS counterparts?

-- TomDonaldson - 2019-06-14




Comments from the IVOA Community during RFC/TCG review period: 2019-06-14 - 2019-07-29

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Added:
>
>
  • Comment from Carlo Zwölf (CarloMariaZwoelf) : It is not clear for me in this proposed recommendation what comes natively from the Healpix definition and what is the IVOA-specific layer. Is there a way for highlighting this difference in the text?
 



Comments from TCG member during the RFC/TCG Review Period: 2019-06-14 - 2019-07-29

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

MOC specification changes to bring String serialization to the normative part looks a good addition to this standard. The document itself is fine in general. I have two small things I'd like to point out.

  • letting [space,CR,LF,comma], multiples/mixes of them, as the elements separator has a human readability effect (see email), not so alerting, but also requires some code in MOC usage. Isn't it really possible to force a bit this serialization to be more smooth with, at least, single and homogeneous separators throughout a single ASCII MOC? Could it be at least warmly suggested?
  • MOCORDER is a MUST and clearly described in the FITS serialization, while in the String (and JSON) one(s) it is less clear how to grab that piece of information. This because it is only said to be a must "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order" (in which case it would be the last token of the String, followed by a slash and no npix). I agree that if it's not so it's implicit the MOCORDER is the last/highest order available, but I'd prefer it to be explicit.
-- MarcoMolinaro - 2019-07-11

Data Model Working Group

Grid & Web Services Working Group

Registry Working Group

Semantics Working Group

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Operations

Standards and Processes Committee


TCG Vote: 2019-06-14 - 2019-07-29

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL        
DM        
GWS        
Registry        
Semantics        
DCP        
KDIG        
SSIG        
Theory        
TD        
Ops        
StdProc        



<--

-->
  • MOC_ASCII_in_Aladin.png:
    MOC_ASCII_in_Aladin.png

META FILEATTACHMENT attachment="MOC_ASCII_in_Aladin.png" attr="" comment="" date="1558081955" name="MOC_ASCII_in_Aladin.png" path="MOC_ASCII_in_Aladin.png" size="9507" user="PierreFernique" version="1"

Revision 92019-07-11 - MarcoMolinaro

 
META TOPICPARENT name="MocInfo"

MOC 1.1 Proposed Recommendation: Request for Comments


Summary

Latest Draft: MOC 1.1 (Proposed Recommendation)

The substantive differences between version 1.1 of MOC and the preceding version 1.0 are:

  • The String MOC serialization was moved from an informative section (suggested syntax) to the normative section.
  • A MOCORDER convention for String MOC and JSON MOC was added.

Reference Interoperable Implementations

(Indicate here the links to at least two Reference Interoperable Implementations)

  • MOCPy >=0.5.6
  • Aladin (>v10.126): read/write ASCII MOC (select ASCII format when exporting MOC - see snapshot below) + new script command dedicated to ASCII MOC (ex: draw MOC 3/1,2,3 4/56-67)
  • The relational registry at http://dc.g-vo.org/tap parses the ASCII MOCs in Registry records into the rr.stc_spatial table (consumer)
  • DaCHS since version 1.1 produces ASCII MOCs when serialising MOC-values database columns.
  • The Moc class within the AST/PyAST WCS library now supports reading and writing ASCII and JSON MOCs in addition to FITS.
To see non-trivial examples for ASCII MOCs coming out of a database, run something like

select top 100 * from rr.stc_spatial
where 1=contains(point(10, 10), coverage)
and 0=contains(point(20, 20), coverage)

on http://dc.g-vo.org/tap.

Implementations Validators

Comments Prior to Official RFC Period

As I was considering semantic validation cases, I noticed this sentence in section 2.3.2:

"Warning: In this basic simple ASCII string format the values may be not sorted, and the MOC may be not well-formed."

which seems to contradict section 2.2.3:

" An encoded MOC must also be “well-formed”. To keep the canonical property of MOC, the redundant cells are not allowed and the hierarchical encoding principle must be respected. Thus it is not allowed to encode 4 sibling cells instead of their parent (only at level 0 for the 12 first diamonds). Using this simple constraint, there is one and only one way to describe a specific region and this makes the interoperability principle easy to implement."

Maybe the former was just a legacy from MOC 1.0 where ASCII MOC was less formal, perhaps intended for hand-crafted MOCs not intended for serious use. With ASCII MOC now being formalized, can we remove that sentence and require that ASCII MOCs must be just as well-formed as their FITS counterparts?

-- TomDonaldson - 2019-06-14




Comments from the IVOA Community during RFC/TCG review period: 2019-06-14 - 2019-07-29

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document



Comments from TCG member during the RFC/TCG Review Period: 2019-06-14 - 2019-07-29

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

Added:
>
>
MOC specification changes to bring String serialization to the normative part looks a good addition to this standard. The document itself is fine in general. I have two small things I'd like to point out.
  • letting [space,CR,LF,comma], multiples/mixes of them, as the elements separator has a human readability effect (see email), not so alerting, but also requires some code in MOC usage. Isn't it really possible to force a bit this serialization to be more smooth with, at least, single and homogeneous separators throughout a single ASCII MOC? Could it be at least warmly suggested?
  • MOCORDER is a MUST and clearly described in the FITS serialization, while in the String (and JSON) one(s) it is less clear how to grab that piece of information. This because it is only said to be a must "If the best HEALPix resolution of the MOC (MOCORDER) is greater than the greatest stored order" (in which case it would be the last token of the String, followed by a slash and no npix). I agree that if it's not so it's implicit the MOCORDER is the last/highest order available, but I'd prefer it to be explicit.
-- MarcoMolinaro - 2019-07-11
 

Data Model Working Group

Grid & Web Services Working Group

Registry Working Group

Semantics Working Group

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Operations

Standards and Processes Committee


TCG Vote: 2019-06-14 - 2019-07-29

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL        
DM        
GWS        
Registry        
Semantics        
DCP        
KDIG        
SSIG        
Theory        
TD        
Ops        
StdProc        



<--

-->
  • MOC_ASCII_in_Aladin.png:
    MOC_ASCII_in_Aladin.png

META FILEATTACHMENT attachment="MOC_ASCII_in_Aladin.png" attr="" comment="" date="1558081955" name="MOC_ASCII_in_Aladin.png" path="MOC_ASCII_in_Aladin.png" size="9507" user="PierreFernique" version="1"

Revision 82019-06-14 - TomDonaldson

 
META TOPICPARENT name="MocInfo"

MOC 1.1 Proposed Recommendation: Request for Comments


Summary

Latest Draft: MOC 1.1 (Proposed Recommendation)

The substantive differences between version 1.1 of MOC and the preceding version 1.0 are:

  • The String MOC serialization was moved from an informative section (suggested syntax) to the normative section.
  • A MOCORDER convention for String MOC and JSON MOC was added.

Reference Interoperable Implementations

(Indicate here the links to at least two Reference Interoperable Implementations)

  • MOCPy >=0.5.6
  • Aladin (>v10.126): read/write ASCII MOC (select ASCII format when exporting MOC - see snapshot below) + new script command dedicated to ASCII MOC (ex: draw MOC 3/1,2,3 4/56-67)
  • The relational registry at http://dc.g-vo.org/tap parses the ASCII MOCs in Registry records into the rr.stc_spatial table (consumer)
  • DaCHS since version 1.1 produces ASCII MOCs when serialising MOC-values database columns.
  • The Moc class within the AST/PyAST WCS library now supports reading and writing ASCII and JSON MOCs in addition to FITS.
To see non-trivial examples for ASCII MOCs coming out of a database, run something like

select top 100 * from rr.stc_spatial
where 1=contains(point(10, 10), coverage)
and 0=contains(point(20, 20), coverage)

on http://dc.g-vo.org/tap.

Implementations Validators

Changed:
<
<
(If any, indicate here the links to Implementations Validators)
>
>
 

Comments Prior to Official RFC Period

Changed:
<
<

>
>
Added:
>
>
As I was considering semantic validation cases, I noticed this sentence in section 2.3.2:
"Warning: In this basic simple ASCII string format the values may be not sorted, and the MOC may be not well-formed."

which seems to contradict section 2.2.3:

" An encoded MOC must also be “well-formed”. To keep the canonical property of MOC, the redundant cells are not allowed and the hierarchical encoding principle must be respected. Thus it is not allowed to encode 4 sibling cells instead of their parent (only at level 0 for the 12 first diamonds). Using this simple constraint, there is one and only one way to describe a specific region and this makes the interoperability principle easy to implement."

Maybe the former was just a legacy from MOC 1.0 where ASCII MOC was less formal, perhaps intended for hand-crafted MOCs not intended for serious use. With ASCII MOC now being formalized, can we remove that sentence and require that ASCII MOCs must be just as well-formed as their FITS counterparts?

-- TomDonaldson - 2019-06-14

 

Added:
>
>

 
Changed:
<
<

Comments from the IVOA Community during RFC/TCG review period: RFC_start_date_TBD - RFC_end_date_TBD

>
>

Comments from the IVOA Community during RFC/TCG review period: 2019-06-14 - 2019-07-29

  The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document



Changed:
<
<

Comments from TCG member during the RFC/TCG Review Period: RFC_start_date_TBD - RFC_end_date_TBD

>
>

Comments from TCG member during the RFC/TCG Review Period: 2019-06-14 - 2019-07-29

  WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

Data Model Working Group

Grid & Web Services Working Group

Registry Working Group

Semantics Working Group

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Operations

Standards and Processes Committee


Changed:
<
<

TCG Vote: RFC_start_date_TBD - RFC_end_date_TBD

>
>

TCG Vote: 2019-06-14 - 2019-07-29

  If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL        
DM        
GWS        
Registry        
Semantics        
DCP        
KDIG        
SSIG        
Theory        
TD        
Ops        
StdProc        



<--

-->
  • MOC_ASCII_in_Aladin.png:
    MOC_ASCII_in_Aladin.png

META FILEATTACHMENT attachment="MOC_ASCII_in_Aladin.png" attr="" comment="" date="1558081955" name="MOC_ASCII_in_Aladin.png" path="MOC_ASCII_in_Aladin.png" size="9507" user="PierreFernique" version="1"

Revision 72019-05-30 - TomDonaldson

 
META TOPICPARENT name="MocInfo"

MOC 1.1 Proposed Recommendation: Request for Comments


Summary

Changed:
<
<
Latest Draft: MOC 1.1 (officially Working Draft)
>
>
Latest Draft: MOC 1.1 (Proposed Recommendation)
  The substantive differences between version 1.1 of MOC and the preceding version 1.0 are:
Deleted:
<
<

 

  • The String MOC serialization was moved from an informative section (suggested syntax) to the normative section.
  • A MOCORDER convention for String MOC and JSON MOC was added.

Reference Interoperable Implementations

(Indicate here the links to at least two Reference Interoperable Implementations)

  • MOCPy >=0.5.6
  • Aladin (>v10.126): read/write ASCII MOC (select ASCII format when exporting MOC - see snapshot below) + new script command dedicated to ASCII MOC (ex: draw MOC 3/1,2,3 4/56-67)
  • The relational registry at http://dc.g-vo.org/tap parses the ASCII MOCs in Registry records into the rr.stc_spatial table (consumer)
  • DaCHS since version 1.1 produces ASCII MOCs when serialising MOC-values database columns.
  • The Moc class within the AST/PyAST WCS library now supports reading and writing ASCII and JSON MOCs in addition to FITS.
To see non-trivial examples for ASCII MOCs coming out of a database, run something like

select top 100 * from rr.stc_spatial
where 1=contains(point(10, 10), coverage)
and 0=contains(point(20, 20), coverage)

on http://dc.g-vo.org/tap.

Implementations Validators

(If any, indicate here the links to Implementations Validators)

Comments Prior to Official RFC Period

Deleted:
<
<
Comment here prior to the official RFC period. Include your Wiki Name. Discussion can be continued on the mailing list (apps@ivoa.net).
 


Changed:
<
<

Ignore Below Until RFC Period Starts

>
>

Comments from the IVOA Community during RFC/TCG review period: RFC_start_date_TBD - RFC_end_date_TBD

Deleted:
<
<

Comments from the IVOA Community during RFC/TCG review period: RFC_start_date_TBD - RFC_end_date_TBD

  The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document



Changed:
<
<

Comments from TCG member during the RFC/TCG Review Period: TCG_start_date_TBD - TCG_end_date_TBD

>
>

Comments from TCG member during the RFC/TCG Review Period: RFC_start_date_TBD - RFC_end_date_TBD

  WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

Data Model Working Group

Grid & Web Services Working Group

Registry Working Group

Semantics Working Group

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Operations

Standards and Processes Committee


Changed:
<
<

TCG Vote : Vote_start_date - Vote_end_date

>
>

TCG Vote: RFC_start_date_TBD - RFC_end_date_TBD

  If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL        
DM        
GWS        
Registry        
Semantics        
DCP        
KDIG        
SSIG        
Theory        
TD        
Ops        
StdProc        



<--

-->
Changed:
<
<
  • MOC_ASCII_in_Aladin.png:
    MOC_ASCII_in_Aladin.png
>
>
  • MOC_ASCII_in_Aladin.png:
    MOC_ASCII_in_Aladin.png
 
META FILEATTACHMENT attachment="MOC_ASCII_in_Aladin.png" attr="" comment="" date="1558081955" name="MOC_ASCII_in_Aladin.png" path="MOC_ASCII_in_Aladin.png" size="9507" user="PierreFernique" version="1"

Revision 62019-05-17 - DavidBerry

 
META TOPICPARENT name="MocInfo"

MOC 1.1 Proposed Recommendation: Request for Comments


Summary

Latest Draft: MOC 1.1 (officially Working Draft)

The substantive differences between version 1.1 of MOC and the preceding version 1.0 are:


  • The String MOC serialization was moved from an informative section (suggested syntax) to the normative section.
  • A MOCORDER convention for String MOC and JSON MOC was added.

Reference Interoperable Implementations

(Indicate here the links to at least two Reference Interoperable Implementations)

  • MOCPy >=0.5.6
  • Aladin (>v10.126): read/write ASCII MOC (select ASCII format when exporting MOC - see snapshot below) + new script command dedicated to ASCII MOC (ex: draw MOC 3/1,2,3 4/56-67)
  • The relational registry at http://dc.g-vo.org/tap parses the ASCII MOCs in Registry records into the rr.stc_spatial table (consumer)
  • DaCHS since version 1.1 produces ASCII MOCs when serialising MOC-values database columns.
Added:
>
>
  • The Moc class within the AST/PyAST WCS library now supports reading and writing ASCII and JSON MOCs in addition to FITS.
 To see non-trivial examples for ASCII MOCs coming out of a database, run something like

select top 100 * from rr.stc_spatial
where 1=contains(point(10, 10), coverage)
and 0=contains(point(20, 20), coverage)

on http://dc.g-vo.org/tap.

Implementations Validators

(If any, indicate here the links to Implementations Validators)

Comments Prior to Official RFC Period

Comment here prior to the official RFC period. Include your Wiki Name. Discussion can be continued on the mailing list (apps@ivoa.net).




Ignore Below Until RFC Period Starts

Comments from the IVOA Community during RFC/TCG review period: RFC_start_date_TBD - RFC_end_date_TBD

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document



Comments from TCG member during the RFC/TCG Review Period: TCG_start_date_TBD - TCG_end_date_TBD

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

Data Model Working Group

Grid & Web Services Working Group

Registry Working Group

Semantics Working Group

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Operations

Standards and Processes Committee


TCG Vote : Vote_start_date - Vote_end_date

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL        
DM        
GWS        
Registry        
Semantics        
DCP        
KDIG        
SSIG        
Theory        
TD        
Ops        
StdProc        



<--

-->
  • MOC_ASCII_in_Aladin.png:
    MOC_ASCII_in_Aladin.png

META FILEATTACHMENT attachment="MOC_ASCII_in_Aladin.png" attr="" comment="" date="1558081955" name="MOC_ASCII_in_Aladin.png" path="MOC_ASCII_in_Aladin.png" size="9507" user="PierreFernique" version="1"

Revision 52019-05-17 - PierreFernique

 
META TOPICPARENT name="MocInfo"

MOC 1.1 Proposed Recommendation: Request for Comments


Summary

Latest Draft: MOC 1.1 (officially Working Draft)

The substantive differences between version 1.1 of MOC and the preceding version 1.0 are:


  • The String MOC serialization was moved from an informative section (suggested syntax) to the normative section.
  • A MOCORDER convention for String MOC and JSON MOC was added.

Reference Interoperable Implementations

(Indicate here the links to at least two Reference Interoperable Implementations)

Changed:
<
<
>
>
  • Aladin (>v10.126): read/write ASCII MOC (select ASCII format when exporting MOC - see snapshot below) + new script command dedicated to ASCII MOC (ex: draw MOC 3/1,2,3 4/56-67)
 
  • The relational registry at http://dc.g-vo.org/tap parses the ASCII MOCs in Registry records into the rr.stc_spatial table (consumer)
  • DaCHS since version 1.1 produces ASCII MOCs when serialising MOC-values database columns.
To see non-trivial examples for ASCII MOCs coming out of a database, run something like

select top 100 * from rr.stc_spatial
where 1=contains(point(10, 10), coverage)
and 0=contains(point(20, 20), coverage)

on http://dc.g-vo.org/tap.

Implementations Validators

(If any, indicate here the links to Implementations Validators)

Comments Prior to Official RFC Period

Comment here prior to the official RFC period. Include your Wiki Name. Discussion can be continued on the mailing list (apps@ivoa.net).




Ignore Below Until RFC Period Starts

Comments from the IVOA Community during RFC/TCG review period: RFC_start_date_TBD - RFC_end_date_TBD

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document



Comments from TCG member during the RFC/TCG Review Period: TCG_start_date_TBD - TCG_end_date_TBD

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

Data Model Working Group

Grid & Web Services Working Group

Registry Working Group

Semantics Working Group

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Operations

Standards and Processes Committee


TCG Vote : Vote_start_date - Vote_end_date

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL        
DM        
GWS        
Registry        
Semantics        
DCP        
KDIG        
SSIG        
Theory        
TD        
Ops        
StdProc        



<--

-->
Added:
>
>
  • MOC_ASCII_in_Aladin.png:
    MOC_ASCII_in_Aladin.png

META FILEATTACHMENT attachment="MOC_ASCII_in_Aladin.png" attr="" comment="" date="1558081955" name="MOC_ASCII_in_Aladin.png" path="MOC_ASCII_in_Aladin.png" size="9507" user="PierreFernique" version="1"
 

Revision 42019-05-06 - PierreFernique

 
META TOPICPARENT name="MocInfo"

MOC 1.1 Proposed Recommendation: Request for Comments


Summary

Latest Draft: MOC 1.1 (officially Working Draft)

The substantive differences between version 1.1 of MOC and the preceding version 1.0 are:


  • The String MOC serialization was moved from an informative section (suggested syntax) to the normative section.
  • A MOCORDER convention for String MOC and JSON MOC was added.

Reference Interoperable Implementations

(Indicate here the links to at least two Reference Interoperable Implementations)

Added:
>
>
 
  • The relational registry at http://dc.g-vo.org/tap parses the ASCII MOCs in Registry records into the rr.stc_spatial table (consumer)
  • DaCHS since version 1.1 produces ASCII MOCs when serialising MOC-values database columns.
Deleted:
<
<
 To see non-trivial examples for ASCII MOCs coming out of a database, run something like
Changed:
<
<

>
>
select top 100 * from rr.stc_spatial

Deleted:
<
<
select top 100 * from rr.stc_spatial
 where 1=contains(point(10, 10), coverage) and 0=contains(point(20, 20), coverage)

on http://dc.g-vo.org/tap.

Implementations Validators

(If any, indicate here the links to Implementations Validators)

Comments Prior to Official RFC Period

Comment here prior to the official RFC period. Include your Wiki Name. Discussion can be continued on the mailing list (apps@ivoa.net).




Ignore Below Until RFC Period Starts

Comments from the IVOA Community during RFC/TCG review period: RFC_start_date_TBD - RFC_end_date_TBD

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document



Comments from TCG member during the RFC/TCG Review Period: TCG_start_date_TBD - TCG_end_date_TBD

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

Data Model Working Group

Grid & Web Services Working Group

Registry Working Group

Semantics Working Group

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Operations

Standards and Processes Committee


TCG Vote : Vote_start_date - Vote_end_date

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL        
DM        
GWS        
Registry        
Semantics        
DCP        
KDIG        
SSIG        
Theory        
TD        
Ops        
StdProc        



<--

-->

Revision 32019-05-06 - MarkusDemleitner

 
META TOPICPARENT name="MocInfo"

MOC 1.1 Proposed Recommendation: Request for Comments


Summary

Latest Draft: MOC 1.1 (officially Working Draft)

The substantive differences between version 1.1 of MOC and the preceding version 1.0 are:


  • The String MOC serialization was moved from an informative section (suggested syntax) to the normative section.
  • A MOCORDER convention for String MOC and JSON MOC was added.

Reference Interoperable Implementations

(Indicate here the links to at least two Reference Interoperable Implementations)

Added:
>
>
  • The relational registry at http://dc.g-vo.org/tap parses the ASCII MOCs in Registry records into the rr.stc_spatial table (consumer)
  • DaCHS since version 1.1 produces ASCII MOCs when serialising MOC-values database columns.

To see non-trivial examples for ASCII MOCs coming out of a database, run something like

select top 100 * from rr.stc_spatial
where 1=contains(point(10, 10), coverage)
and 0=contains(point(20, 20), coverage)

on http://dc.g-vo.org/tap.

 

Implementations Validators

(If any, indicate here the links to Implementations Validators)

Comments Prior to Official RFC Period

Comment here prior to the official RFC period. Include your Wiki Name. Discussion can be continued on the mailing list (apps@ivoa.net).




Ignore Below Until RFC Period Starts

Comments from the IVOA Community during RFC/TCG review period: RFC_start_date_TBD - RFC_end_date_TBD

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document



Comments from TCG member during the RFC/TCG Review Period: TCG_start_date_TBD - TCG_end_date_TBD

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

Data Model Working Group

Grid & Web Services Working Group

Registry Working Group

Semantics Working Group

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Operations

Standards and Processes Committee


TCG Vote : Vote_start_date - Vote_end_date

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL        
DM        
GWS        
Registry        
Semantics        
DCP        
KDIG        
SSIG        
Theory        
TD        
Ops        
StdProc        



<--

-->

Revision 22019-05-06 - ThomasBoch

 
META TOPICPARENT name="MocInfo"

MOC 1.1 Proposed Recommendation: Request for Comments


Summary

Latest Draft: MOC 1.1 (officially Working Draft)

The substantive differences between version 1.1 of MOC and the preceding version 1.0 are:


Added:
>
>
 
  • The String MOC serialization was moved from an informative section (suggested syntax) to the normative section.
  • A MOCORDER convention for String MOC and JSON MOC was added.

Reference Interoperable Implementations

(Indicate here the links to at least two Reference Interoperable Implementations)

Added:
>
>
 

Implementations Validators

(If any, indicate here the links to Implementations Validators)

Comments Prior to Official RFC Period

Comment here prior to the official RFC period. Include your Wiki Name. Discussion can be continued on the mailing list (apps@ivoa.net).




Ignore Below Until RFC Period Starts

Comments from the IVOA Community during RFC/TCG review period: RFC_start_date_TBD - RFC_end_date_TBD

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document



Comments from TCG member during the RFC/TCG Review Period: TCG_start_date_TBD - TCG_end_date_TBD

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

Data Model Working Group

Grid & Web Services Working Group

Registry Working Group

Semantics Working Group

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Operations

Standards and Processes Committee


TCG Vote : Vote_start_date - Vote_end_date

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL        
DM        
GWS        
Registry        
Semantics        
DCP        
KDIG        
SSIG        
Theory        
TD        
Ops        
StdProc        



<--

-->

Revision 12019-05-03 - TomDonaldson

 
META TOPICPARENT name="MocInfo"

MOC 1.1 Proposed Recommendation: Request for Comments


Summary

Latest Draft: MOC 1.1 (officially Working Draft)

The substantive differences between version 1.1 of MOC and the preceding version 1.0 are:


  • The String MOC serialization was moved from an informative section (suggested syntax) to the normative section.
  • A MOCORDER convention for String MOC and JSON MOC was added.

Reference Interoperable Implementations

(Indicate here the links to at least two Reference Interoperable Implementations)

Implementations Validators

(If any, indicate here the links to Implementations Validators)

Comments Prior to Official RFC Period

Comment here prior to the official RFC period. Include your Wiki Name. Discussion can be continued on the mailing list (apps@ivoa.net).




Ignore Below Until RFC Period Starts

Comments from the IVOA Community during RFC/TCG review period: RFC_start_date_TBD - RFC_end_date_TBD

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document



Comments from TCG member during the RFC/TCG Review Period: TCG_start_date_TBD - TCG_end_date_TBD

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

Data Model Working Group

Grid & Web Services Working Group

Registry Working Group

Semantics Working Group

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Operations

Standards and Processes Committee


TCG Vote : Vote_start_date - Vote_end_date

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL        
DM        
GWS        
Registry        
Semantics        
DCP        
KDIG        
SSIG        
Theory        
TD        
Ops        
StdProc        



<--

-->
 
This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback