CoordsRFC < IVOA

---+ STC2:Coords Proposed Recommendation: Request for Comments

%TOC{depth="2"}%

---++ Summary

Version 1 of STC was developed in 2007, prior to the development and adoption of vo-dml modeling practices. As we progress to the development of vo-dml compliant component models, it is necessary to revisit those models which define core content. Additionally, the scope of the STC-1.0 model is very broad, making a complete implementation and development of validators, very difficult. As such it may be prudent to break the content of STC-1.0 into component models itself, which as a group, cover the scope of the original.

This effort will start from first principles with respect to defining a specific project use-case, from which requirements will be drawn, satisfied by the model, and implemented in the use-case. We will make use of the original model to ensure that the coverage of concepts is complete and that the models will be compatible. However, the form and structure may be quite different. This model will use vo-dml modeling practices, and model elements may be structured differently to more efficiently represent the concepts.

This model describes the Coordinates model and covers the following concepts.
   * Description of single and multi-dimensional coordinate space, and coordinates within that space.
   * Coordinate Frames, providing metadata describing the origin and orientation of the coordinate space.
   * Definition of simple domain-specific coordinate types for the most common use cases.
   * Coordinate Systems, a collection of coordinate frames.
The latest version of the model and supporting docs:
   * Model document: [[http://www.ivoa.net/documents/Coords/index.html][here]]
   * VO-DML/XML representation: [[http://www.ivoa.net/xml/STC/STC_coords-v1.0.vo-dml.xml][here]]
   * XML Schema: [[https://volute.g-vo.org/svn/trunk/projects/dm/STC/Coords/vo-dml/Coords_v1.0.xsd][here]]
---++ Implementation Requirements

(from [[IvoaDataModel#Specifications][DM Working group twiki]]):

The "IVOA Document Standards" standard has a broad outline of the implementation requirements for IVOA standards. These requirements fit best into the higher level standards for applications and protocols than for data models themselves. At the Oct 2017 interop in Trieste, the following implementation requirements for Data Model Standards was agreed upon, which allow the models to be vetted against their requirements and use cases, without needing full science use cases to be implemented.
   * VO-DML models must validate against schema
   * Serializations which touch each entity of the model. These serializations may be 'fake' (ie: not based on actual data files), and are to be produced by the modeler as unit tests/examples.
   * Real world serializations covering use cases, produced by others following the model, in a mutually agreed upon format.
   * Software which interprets these serializations and demonstrates proper interpretation of the content.
---+++ VO-DML Validation:
   * The Coords model was developed using the Modelio UML tool, and exported to xmi format.
   * The xmi model description was then processed using an xslt script into VO-DML/XML format.
   * This VO-DML/XML format file was then validated against the vo-dml schema with no reported violations. 
      * the xml file is available in the IVOA document repository, [[http://www.ivoa.net/xml/STC/STC_coords-v1.0.vo-dml.xml][here]].
---+++ Serializations:
   * Modeler Generated Examples: 
      * using home grown python code, the modeler has generated [[https://volute.g-vo.org/svn/trunk/projects/dm/STC/Coords/examples/][example serializations]] which span all elements of the model. The examples are generated in 4 formats: 
         * VOTable-1.3 standard syntax; Validates using votlint
         * VOTable-1.3 annotated with VO-DML/Mapping syntax; Validates using xmllint to a VOTable-1.3 schema enhanced with an imported [[https://volute.g-vo.org/svn/trunk/projects/dm/vo-dml/xsd/votable_ext/VOTable-1.4_vodml.xsd][VO-DML mapping syntax schema]]
         * XML format; Validates against the model schema
         * An internal DOC format; XML/DOM structure representing the instances generated when interpreting the instance templates.
   * Real world serializations: 
      * jovial: A set of example serializations of various Coordinate and Measurement instances generated by Omar Laurino using his Jovial DSL package 
         * <a target="_blank" href="https://olaurino.gitlab.io/ivoa-dm-examples/0000/01/03/Coords.html">https://olaurino.gitlab.io/ivoa-dm-examples/0000/01/03/Coords.html</a>
         * <a target="_top" href="https://olaurino.gitlab.io/ivoa-dm-examples/0000/01/02/STC-current.html">https://olaurino.gitlab.io/ivoa-dm-examples/0000/01/02/STC-current.html</a> - examples of Measurement and Coords based on earlier versions of the models
      * TDIG: Working project of Time Series as Cube. 
         * Example serializations have been generated using various annotation schemes to identify the model elements: [[https://volute.g-vo.org/svn/trunk/projects/time-domain/time-series/standardized_votables/][here]]. 
            * these include elements from the Measurement and Coordinates models
      * VOTable: COOSYS 
         * this represents a standardized serialization of a Coordinate model SpaceFrame 
            * COOSYS =&gt; SpaceFrame
            * COOSYS.system =&gt; SpaceFrame.spaceRefFrame
            * COOSYS.equinox =&gt; SpaceFrame.equinox
            * COOSYS.epoch =&gt; would map to epoch of a particular measurement set, outside the scope of SpaceFrame
            * NOTE: COOSYS lacks the 'refPosition' present in SpaceFrame.. this is on the list as a probable enhancement to COOSYS
      * VOTable TIMESYS 
         * recently adopted into VOTable 1.4, this is similarly, a standardized serialization of Coordinate model TimeFrame
         * TIMESYS =&gt; TimeFrame
         * TIMESYS.timescale =&gt; TimeFrame.timescale
         * TIMESYS.refPosition =&gt; TimeFrame.refPosition
         * TIMESYS.timeorigin =&gt; TimeOffset.time0; centralizing this information high in the serialization
---+++ Software:
   * vodml Parser: Notebook developed by Omar Laurino parses vo-dml/Mapping annotation to generate instances of TimeSeries-Cube-Meas-Coord using AstroPy and generates plots of the content using Matplotlib. Note: this was developed and presented in 2017 using earlier drafts of the models. These vary only in detail, and the demo could be updated to the current model suite. 
      * <a target="_top" href="https://olaurino.gitlab.io/ivoa-dm-examples/0000/01/10/PythonParserDemo.html">https://olaurino.gitlab.io/ivoa-dm-examples/0000/01/10/PythonParserDemo.html</a>
   * TDIG: Working project of Time Series as Cube. 
      * An ongoing project is underway to enhance SPLAT to load/interpret/analyze TimeSeries data. This was most recently presented at the IVOA Interop in Paris 2019 (see [[%PUBURL%/IVOA/InterOpMay2019TDIG/splat_timeseries.pdf][PDF]])
      * The tool currently uses a combination of TIMESYS, table column descriptions, UCDs and UTypes to identify and interpret the data automatically. Each of these are annotation schemes which tie directly to model components.
      * Delays in resolving on a standard annotation syntax has delayed progress on this project to fully realize the possibilities. This is a high-priority for upcoming work.
   * pyVO: extract_skycoord_from_votable() 
      * Demonstrated at the IVOA Interop in Paris 2019, this product of the hack-a-thon generates AstroPy SkyCoord instances from VOTables using various elements embedded in the VOTable. 
         * Interrogates a VOTable-1.3 serialization, identifies key information and uses that to automatically generate instances of SkyCoord. 
            * UCD: 'pos.eq.ra', 'pos.eq.dec'
            * COOSYS.system: "ICRS", "FK4", "FK5"
            * COOSYS.equinox
         * The COOSYS maps directly to SpaceFrame, and the value of the system
         * The UCD 'pos.eq' maps directly to meas:EquatorialPosition; with 'pos.eq.ra|dec' identifying the corresponding attributes (EquatorialPosition.ra|dec) as coordinates coords:Longitude and coords:Latitude.
         * This illustrates that even with minimal annotation, this sort of automatic discovery/instantiation can take place. With a defined annotation syntax, this utility could be expanded to generate other AstroPy objects very easily.
   * AST: Starlink's library for handline World Coordinate Systems 
      * An ongoing project to exercise the Transform model is being worked, which includes pulling in elements from the Coordinates model to define the 'Frames' on either end of the transform.
      * This work is being done with YAML serializations, annotated to the model elements
      * Currently includes the CoordSpace, PixelSpace and SpaceFrame elements. There is a high level of correlation between the AST Frame objects and the Coordinate model elements.
   * 
      * 
         * 
---++ Validators

As noted above, the serializations may be validated to various degrees using the corresponding schema
   * VOTable-1.3 using votlint: verifies the serialization complies with VOTable syntax
   * VOTable-1.3 + VODML: verifies the serialization is properly annotated
   * XML using xmllint with model schema: verifies the serialization is a valid instance of the model.
   * NOTE: The modeler examples undergo all levels of validation, showing that the VOTable serializations are also valid instances of the model.
I don't believe there are validators for the various software utilities. Their purpose is to show that given an agreed serialization which can be mapped to the model(s), the data can be interpreted in an accurate and useful manner.

---++ Links with Meas

The Coordinates model a base component, primarily used to support the development of other models. Most significantly is in support of the Measurement model, which is the core model of interest to most use cases. Information about the relation to that model, how the use case requirements divide up, etc. can be found on the [[https://wiki.ivoa.net/twiki/bin/view/IVOA/STC2][STC2 page]]

---++ Comments from the IVOA Community during RFC/TCG review period: 2019-09-17 - 2019-10-21

<strong>Comments by Mark C-D: </strong>-- IVOA.MarkCresitelloDittmar - 2019-09-20

In generating the XML format example files I found an issue which probably should be changed:
   1 SpaceFrame.planetaryEphem has a multiplicity of 1, so must be included to be valid, but is not needed in many cases. Also, the text in the document says "A planetary ephemeris MAY be provided if relevant". This attribute should have multiplicity [0..1].
*Comments by Mark C-D*: -- IVOA.MarkCresitelloDittmar - 2019-10-01

I'm reporting this on behalf of David Berry, who is working the Transform model implementation thread (see [[STC2#Uses_cases][twiki use case #2]] ). The description there involves passing transform model Mapping instances between gWCS and AST, and so didn't have any lien on the Coordinates model. In the process, he has expanded the scope to a more useful thread for the AST library, which is to: "import/export the full WCS associated with an image" This includes descriptions of the Pixel axes, image axes, and the associated transforms, which brings the Coords model into the scope.

The thread wants to specifically NOT include the coordinates themselves.

NOTE: this is similar to a usage discussed in the context of SIA2 for image cutouts, so may be more broadly applicable. In working this, he is having some trouble due to a change made some iterations ago, back in 2017, which separated the Frame and CoordSpace elements. From that point, they have been linked only via the Coordinate value itself, which references both a Frame and an axis of the CoordSpace.

To satisfy this extended usage case, would add a requirement to the Coords model:

"enable the description of a complete WCS ( axes, frames, mappings between ) independent of the coordinates in that space"

This seems like a reasonable extension of the usage case to me.<br />I am looking into the options for satisfying that requirement and its impact on the coords model currently in RFC.

---++ Comments from TCG member during the RFC/TCG Review Period: 2019-09-17 - 2019-10-21

---+++ TCG Chair & Vice Chair

I made a related comment in the MeasRFC page: At the end of section 2.1.1 under Physical Data (Observables) I see mention of spatial, time, polarization, and spectral. Then in the model I see GenericCoordFrame, SpatialFrame, and TimeFrame... and lower down GenericCoordValue, SpatialCoord, TimeStamp, and Standard1DCoord. Several issues here:

1. naming consistency: why TimeStamp and not TimeCoord? why GenericCoordValue and not GenericCoord?

2. for polarization, would one use generic or standard1d (coord) with a BinnedAxis?

3. again, what about spectral? there is no SpectralFrame but I recall that a minimal spectral axis (FITS WCS paper III and reflected in CAOM) requires a fair bit of special pieces of metadata and a GenericCoordFrame only has a refPosition... the treatment of spectral axis may be complete but it is very opaque and I can't really tell.

TODO: CompositeCoord vs CoordValue and all those subclasses of SpatialCoord
---+++ [[IvoaApplications][Applications Working Group]]
---+++ [[IvoaDAL][Data Access Layer Working Group]]
---+++ [[IvoaDataModel][Data Model Working Group]]
---+++ [[IvoaGridAndWebServices][Grid & Web Services Working Group]]
---+++ [[IvoaResReg][Registry Working Group]]
---+++ [[IvoaSemantics][Semantics Working Group]]
---+++ [[IvoaDCP][Data Curation & Preservation Interest Group]]
---+++ [[IvoaTheory][Education Interest Group]]
---+++ [[IvoaKDD][Knowledge Discovery Interest Group]]
---+++ [[IvoaSS][Solar System Interest Group]]
---+++ [[IvoaTheory][Theory Interest Group]]
---+++ [[IvoaVOEvent][Time Domain Interest Group]]
---+++ [[IvoaOps][Operations]]

Minor comments:
   * Sec 5.3.2, 5.4.2, 5.7.1: Vocabularies here are given in the form "https://www.ivoa.net/rdf/..." I think those URLs should be "http://..." not "https://..."; that's what is written in VOTable 1.4 sec 3.5, and I don't believe there is an HTTPS server from www.ivoa.net.
   * Appx C.3: the time factors listed for TCG and TCB are formatted weirdly in the PDF output; I think (e.g.) "<code>7*10\^-10</code>" in the !LaTeX source ought to read "<code>7*10$^{-10}$</code>".
-- IVOA.MarkTaylor - 2019-09-27

---+++ [[IvoaStdsDocsProc][Standards and Processes Committee]]

---++ TCG Vote:

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

| Group | Yes | No | Abstain | Comments |
| TCG | | | | |
| Apps | | | | |
| DAL | | | | |
| DM | | | | |
| GWS | | | | |
| Registry | | | | |
| Semantics | | | | |
| DCP | | | | |
| KDIG | | | | |
| SSIG | | | | |
| Theory | | | | |
| TD | | | | |
| Ops | | | | |
| &lt;nop&gt;StdProc | | | | |

---
This topic: IVOA > WebHome > IvoaDataModel > CoordsRFC
Topic revision: r8 - 2020-01-20 - PatrickDowler