(r10) SpectrumDM12RFC < IVOA

IVOA Web>IvoaDataModel>SpectrumDM12RFC (revision 10) (raw view)~~EditAttach~~
---++ Spectrum 1.2 Proposed Recommendation: Request for Comments

%TOC{depth="3"}%

---+++ Introduction

The Spectrum Data Model describes the structure of spectrophotometric datasets with spectral and temporal coordinates and associated metadata.

This update satisfies an enhancement request by IPAC to add support for the following use cases:
   * 1-D spectra from Spitzer’s Infrared Spectrograph have multiple spectral orders
      * Spectral orders can overlap in wavelength
      * Plotting a Spitzer spectrum without accounting for orders gives you a mess
   * SEDs often include limits on measurements
      * these can be upper or lower limits
      * Plotters need to indicate limits clearly to avoid scientific misunderstanding
      * SDM1.1 has an ad-hoc representation for upper limits (only) which may be usable, but is conceptually incorrect.
The main changes in the v1.2 document are:
   * Section 4.1.1: addition of 'order' and 'relorder' attributes
   * Section 4.6.3: addition of 'UpperLimit' and 'LowerLimit' to explicitly express these concepts
   * Corrected UTypes in VOTable example #1
   * Updated diagrams to proper UML with improved consistency with the schema and text.
Latest version of !SpectrumDM can be found at:
   * https://www.ivoa.net/documents/SpectrumDM/20230907/index.html
      * associated schema: https://www.ivoa.net/xml/Spectrum/Spectrum-1.2.xsd
The !GitHub repository for issues and model development can be found at:
   * [[https://github.com/ivoa-std/SpectrumDM/tree/wd-v1.2][https://github.com/ivoa-std/SpectrumDM]]
The IVOA Twiki page for the project can be found at:
   * [[Spectrum][https://wiki.ivoa.net/twiki/bin/view/IVOA/Spectrum]]

---+++ Reference Interoperable Implementations

The implementation level for this RFE is not as comprehensive as we would typically look for in a REC.

The DM working group consulted with the TCG prior to the start of this RFC and it was agreed that given
   * this is a minor revision based on a specific institution request
   * the request is targeted to serve a very specific purpose
   * the model is not VO-DML compliant, so the level of effort required to produce additional implementations, serializations, validators is out of proportion to the impact of the RFE.
the level of implementation for this enhancement is acceptable.

*Data Providers:*
   * GAVO: Updated SSA Server to
      * include a PARAM with UTYPE="spec:Spectrum.Data.SpectralAxis.order" into each Spectrum Table.
      * also includes other changes/corrections from a review of the output against the Spectrum model document.
         * example file: <a href="http://dc.zah.uni-heidelberg.de/getproduct/flashheros/data_raw/ca98/blue/n0393.mt" target="_blank">http://dc.zah.uni-heidelberg.de/getproduct/flashheros/data_raw/ca98/blue/n0393.mt</a>
*Applications:*
   * IPAC Firefly Archive toolkit:
      * "For some time, IPAC's Firefly archive GUI toolkit has taken advantage of IVOA Spectral Data Model (SDM) v1.1 to provide users with automatic plots of spectra that are the result of a query or have been uploaded to the tool. The plotting capability uses the SDM to identify which data to plot on the x and y axes and to change units. Firefly has recently been updated to recognize the new utypes specified in the proposed Spectral Data Model v1.2, namely:<br /><br />spec:Spectrum.Data.SpectralAxis.Order<br />spec:Spectrum.Data.FluxAxis.Accuracy.LowerLimit<br />spec.Spectrum.Data.FluxAxis.Accuracy.UpperLimit<br /><br />The toolkit uses these new utypes to plot orders (as found in e.g. Spitzer spectra) as separate traces, and to plot upper and lower limits as arrows (as found in e.g. SEDs). These changes are currently in test and will be released in May. Upon release, these features will be available in IRSA, Rubin, and NED interfaces, all of which are built upon the open-source Firefly toolkit. <p> </p> In a future release, Firefly will recognize the spec:Spectrum.Data.SpectralAxis.RelOrder utype proposed in SDMv1.2. When Firefly encounters a spectrum with only !RelOrder and not Order, it will automatically plot each unique !RelOrder as a different trace, analogous to how it treats Order. If a spectrum contains both Order and !RelOrder, Firefly will automatically plot traces based on Order but will provide the user a toggle to plot !RelOrder."

---+++ Implementations Validators

There is no dedicated validator for Spectrum data model instances to update for this enhancement.

Since this is not a VO-DML compliant model, the mechanisms for
   * validating the data model itself
   * generating and validating example serializations in VOTable, XML, etc.
   * generating model based code
are not available.

---
---

---++ Comments from the IVOA Community during RFC/TCG review period: 2023-09-11 - 2023-10-23

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

   * Sample comment by TWiki.WikiName
      * Response (by TWiki.WikiName)

---+++++ Comment by IVOA.JesusSalgado

This is a preliminary comment: Version 1.0 had a java library implemented by Kelly McCusker (not in IVOA anymore) and Jonathan McDowell that allowed parsing and writing of spectral data compatible with 1.0. It would be great to obtain a version of this library and adapt it for these minor changes.

This library was used by some tools (e.g. VOSpec)

IVOA.JesusSalgado - 2023-09-22
---
---

---++ Comments from TCG member during the RFC/TCG Review Period: %RED%2023-09-11 - 2023-10-23%ENDCOLOR%

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

---+++ TCG Chair & Vice Chair

---+++ [[IvoaApplications][Applications Working Group]]

---+++ [[IvoaDAL][Data Access Layer Working Group]]

Good to see support of orders as this has been raised by the community.

-- IVOA.JamesDempsey - 2023-11-07
---+++ [[IvoaDataModel][Data Model Working Group]]

---+++ [[IvoaGridAndWebServices][Grid & Web Services Working Group]]

Changes are minimal and controlled from the previous version. The only point raised about the reference implementation (there was one that could be modified easily, I think, to cope with the changes but it is quite old so I am not even sure if the code is recoverable) (reference, java library implemented by Kelly McCusker and Jonathan McDowell (maybe Jonathan could help on this)). This library is a full implementation able to read and write SpectrumDM 1.1 compatible files.

Maybe there is another implementation. Could you clarify it?

In any case, as the changes are fully controlled, we approve the update.

IVOA.JesusSalgado - 2023-11-06
---+++ [[IvoaResReg][Registry Working Group]]

---+++ [[IvoaSemantics][Semantics Working Group]]

* In section *3.3 Units*, it is stated that the unit formatting standard is *WCS/OGIP convention for unit*. Is there any reason not to use the VOUnit formalism (see ongoing RFC)? It would be a pity if we can't converge here.

* In section *3.4 UCDs* there seems to be a typo at the end of the first paragraph: _[...] so you can’t put a literal em.* in a UTYPE field [...]_. Do you mean _UCD field_ ?

* In section *5.1 CoordSys Fields*, the proposed list of terms should be submitted as a VEP to Semantics, in order to update the [[https://www.ivoa.net/rdf/refposition/2019-03-15/refposition.html][RefPosition]] vocabulary. And then refer to that vocabulary for acceptable values.

* In section *5.1.5 STC*, the same remark concerning the [[https://www.ivoa.net/rdf/refframe/2022-02-22/refframe.htmll][RefFrame]] vocabulary

* On pages 17-19, the table mentions "TimeFrameUCD", SpaceFrameUCD": I don't see what this refers to. 

IVOA.BaptisteCecconi - 2023-11-09

---+++ [[IvoaDCP][Data Curation & Preservation Interest Group]]

---+++ [[IvoaEducation][Education Interest Group]]

---+++ [[IvoaKDD][Knowledge Discovery Interest Group]]

---+++ [[IvoaOps][Operations Interest Group]]

---+++ [[IvoaRadio][Radio Astronomy Interest Group]]

---+++ [[IvoaSS][Solar System Interest Group]]

---+++ [[IvoaTheory][Theory Interest Group]]

---+++ [[IvoaVOEvent][Time Domain Interest Group]]

---+++ [[IvoaStdsDocsProc][Standards and Processes Committee]]

---

---++ TCG Vote : %RED%Vote_start_date - Vote_end_date%ENDCOLOR%

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

| Group | Yes | No | Abstain | Comments |
| TCG | | | | |
| Apps | | | | |
| DAL | * | | | |
| DM | * | | | |
| GWS | * | | | |
| Registry | * | | | |
| Semantics | *  | | | Semantics WG comments should be addressed in next version |
| DCP | | | | |
| Edu | | | | |
| KDIG | * | | | |
| Ops | | | | |
| Radio | | | | |
| SSIG | | | | |
| Theory | | | | |
| TD | | | | |
| <nop>StdProc | | | | |

---
Topic revision: r10 - 2023-11-10 - BaptisteCecconi
IVOA
Log in or Register
IVOA.net
Wiki Home
WebChanges
WebTopicList
WebStatistics
Twiki Meta & Help
IVOA
Know
Main
Sandbox
TWiki
TWiki intro
TWiki tutorial
User registration
Notify me
Working Groups
Interest Groups
Committees
Stds&Procs
www.ivoa.net
Documents
Events
Members
XML Schema