Difference: SpectrumDM12RFC (1 vs. 19)

Revision 192023-11-12 - JanetEvans

 
META TOPICPARENT name="IvoaDataModel"

Spectrum 1.2 Proposed Recommendation: Request for Comments


Introduction

The Spectrum Data Model describes the structure of spectrophotometric datasets with spectral and temporal coordinates and associated metadata.

This update satisfies an enhancement request by IPAC to add support for the following use cases:

  • 1-D spectra from Spitzer’s Infrared Spectrograph have multiple spectral orders
    • Spectral orders can overlap in wavelength
    • Plotting a Spitzer spectrum without accounting for orders gives you a mess
  • SEDs often include limits on measurements
    • these can be upper or lower limits
    • Plotters need to indicate limits clearly to avoid scientific misunderstanding
    • SDM1.1 has an ad-hoc representation for upper limits (only) which may be usable, but is conceptually incorrect.
The main changes in the v1.2 document are:
  • Section 4.1.1: addition of 'order' and 'relorder' attributes
  • Section 4.6.3: addition of 'UpperLimit' and 'LowerLimit' to explicitly express these concepts
  • Corrected UTypes in VOTable example #1
  • Updated diagrams to proper UML with improved consistency with the schema and text.
Latest version of SpectrumDM can be found at: The GitHub repository for issues and model development can be found at: The IVOA Twiki page for the project can be found at:

Reference Interoperable Implementations

The implementation level for this RFE is not as comprehensive as we would typically look for in a REC.

The DM working group consulted with the TCG prior to the start of this RFC and it was agreed that given

  • this is a minor revision based on a specific institution request
  • the request is targeted to serve a very specific purpose
  • the model is not VO-DML compliant, so the level of effort required to produce additional implementations, serializations, validators is out of proportion to the impact of the RFE.
the level of implementation for this enhancement is acceptable.

Data Providers:

Applications:
  • IPAC Firefly Archive toolkit:
    • "For some time, IPAC's Firefly archive GUI toolkit has taken advantage of IVOA Spectral Data Model (SDM) v1.1 to provide users with automatic plots of spectra that are the result of a query or have been uploaded to the tool. The plotting capability uses the SDM to identify which data to plot on the x and y axes and to change units. Firefly has recently been updated to recognize the new utypes specified in the proposed Spectral Data Model v1.2, namely:

      spec:Spectrum.Data.SpectralAxis.Order
      spec:Spectrum.Data.FluxAxis.Accuracy.LowerLimit
      spec.Spectrum.Data.FluxAxis.Accuracy.UpperLimit

      The toolkit uses these new utypes to plot orders (as found in e.g. Spitzer spectra) as separate traces, and to plot upper and lower limits as arrows (as found in e.g. SEDs). These changes are currently in test and will be released in May. Upon release, these features will be available in IRSA, Rubin, and NED interfaces, all of which are built upon the open-source Firefly toolkit.

      In a future release, Firefly will recognize the spec:Spectrum.Data.SpectralAxis.RelOrder utype proposed in SDMv1.2. When Firefly encounters a spectrum with only RelOrder and not Order, it will automatically plot each unique RelOrder as a different trace, analogous to how it treats Order. If a spectrum contains both Order and RelOrder, Firefly will automatically plot traces based on Order but will provide the user a toggle to plot RelOrder."

Implementations Validators

There is no dedicated validator for Spectrum data model instances to update for this enhancement.

Since this is not a VO-DML compliant model, the mechanisms for

  • validating the data model itself
  • generating and validating example serializations in VOTable, XML, etc.
  • generating model based code
are not available.

Comments from the IVOA Community during RFC/TCG review period: 2023-09-11 - 2023-10-23

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Comment by JesusSalgado

This is a preliminary comment: Version 1.0 had a java library implemented by Kelly McCusker (not in IVOA anymore) and Jonathan McDowell that allowed parsing and writing of spectral data compatible with 1.0. It would be great to obtain a version of this library and adapt it for these minor changes.

This library was used by some tools (e.g. VOSpec)

JesusSalgado - 2023-09-22

Comments from TCG member during the RFC/TCG Review Period: 2023-09-11 - 2023-10-23

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Added:
>
>		With review and knowledge that this was a specific update to meet the needs of project feedback, and with the provision in the document identifying the limited change, the TCG chair and V Chair approve and vote positively with the WG/IGs.
 

Applications Working Group

We appreciate the initiative to evolve the spectrum data model, which needs it.
Compared with other IVOA standards, there is a lot of work to be done in the document to bring it up to date with the advances made by other working groups since the initial version.
Paragraph 3.3 Units should be replaced by VO Units
paragraph 3.4 UCD should be updated
Usage of STC is a reference to STC 1.0


ref CoordSystem should be listed elsewhere to allow evolution of the document or presented as a possible choice to be consistent with the xsd schema
<xs:complexType name="coordFrameType">
<xs:annotation>
<xs:documentation>Simplification of STC version: RefPos is string</xs:documentation>

VOTable serialisation is 1.1 version
DM's initiative to unify data models seems to have a place in the redesign of spectrum data model, as it would be a textbook case.

minor thing
on VOTABLE serialisation example it seem that there is problem on validation for
ucd="sdm:spect.frame"
on <PARAM there is no datatype on 3 of them

Response by (MarkCresitelloDittmar - 2023-11-11)

  • Similar to the Semantics group comments, we wanted to keep the scope of this update focused on the requested updates. The items you mention are good candidates for additional updates, and we can add Issues for them in the Spectrum repository. I believe many were included in the Spectrum2.0 effort back in 2016 which was tabled partially from lack of implementation, and partially because of the DM model refactoring effort. The 'plan' for the evolution of this model has been to represent it as a specialization of the Cube model, much like TimeSeries. We're happy to work with anyone interested in exercising that workflow!.
  • Re: the invalid ucd. There are several invalid and incorrect ucds in the document, this is issue #7 in the Spectrum repository. It was decided in the context of the recent Cone Search (?) that the documented UCDs should not be consided normative. For this reason we did not perform a sweep to resolve and correct the listed ucds.
  • Re: PARAM with no datatype, this could be resolved with this update.
  • To address the concerns about the outdated practices and references, we will add two statement to the document
    • To the abstract (and landing page), a statement to the effect of: "This is a minor update to the model to satisfy a specific request. We acknowledge that there are elements to the model which do not follow current IVOA practices and would benefit from modernization. This was intentionally not done during this revision to minimize the impact on existing implementations. They can be addressed in a subsequent update to the model."
    • To the UCD section, a statement to the effect of: "The UCDs specified in this document are non-normative, and should be considered as suggested or example values to convey the intent. Users should take care to select and validate UCDs which are appropriate and valid for their data."

Data Access Layer Working Group

Good to see support of orders as this has been raised by the community.

-- JamesDempsey - 2023-11-07

Data Model Working Group

Grid & Web Services Working Group

Changes are minimal and controlled from the previous version. The only point raised about the reference implementation (there was one that could be modified easily, I think, to cope with the changes but it is quite old so I am not even sure if the code is recoverable) (reference, java library implemented by Kelly McCusker and Jonathan McDowell (maybe Jonathan could help on this)). This library is a full implementation able to read and write SpectrumDM 1.1 compatible files.

Maybe there is another implementation. Could you clarify it?

In any case, as the changes are fully controlled, we approve the update.

JesusSalgado - 2023-11-06

Response by: MarkCresitelloDittmar - 2023-11-10

I agree that this would be a good thing to work toward, and will make an effort to track down the library and see if there is a path toward updating it to V1.2.

Registry Working Group

Semantics Working Group

  • In section 3.3 Units, it is stated that the unit formatting standard is WCS/OGIP convention for unit. Is there any reason not to use the VOUnit formalism (see ongoing RFC)? It would be a pity if we can't converge here.

  • In section 3.4 UCDs there seems to be a typo at the end of the first paragraph: [...] so you can’t put a literal em.* in a UTYPE field [...]. Do you mean UCD field ?

  • In section 5.1 CoordSys Fields, the proposed list of terms should be submitted as a VEP to Semantics, in order to update the RefPosition vocabulary. And then refer to that vocabulary for acceptable values.

  • In section 5.1.5 STC, the same remark concerning the RefFrame vocabulary

  • On pages 17-19, the table mentions "TimeFrameUCD", SpaceFrameUCD ": I don't see what this refers to.

BaptisteCecconi - 2023-11-09

  • Response * The goal for this RFE was to keep the changes as focused on the request as possible. I agree with your statements and I suggest we add these as Issues on the Spectrum project to be addressed in a 'modernizing' RFE. I will correct the typo.

MarkCresitelloDittmar - 2023-11-10

  • General comment: I think it is a bad idea to adopt a document which so much outdated.

  • In UCD section, it is stated that UCDs should be case insensitive. If you use UCDs, you don't change the definition/usage/syntax of UCDs in tis document. This sentence MUST be removed.

BaptisteCecconi - 2023-11-11

  • see bullet in Applications section regarding addtions to the document abstract and UCD section to address the concerns about outdated modeling practices in the model.
MarkCresitelloDittmar - 2023-11-11

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Operations Interest Group

Nothing to add at this time. Encourage continuing efforts to adopt smaller changes as they can be generally agreed on, to help minimize delays involved in working out larger, more controversial topics. - Steve Groom - 2023-11-11

Radio Astronomy Interest Group

Solar System Interest Group

Nothing to add.

-- Anne Raugh - 2023-11-10

Theory Interest Group

Time Domain Interest Group

Standards and Processes Committee

TCG Vote : Vote_start_date - Vote_end_date

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
Changed:
<
<
TCG        
>
>
TCG *      
 
Apps *      
DAL *      
DM *      
GWS *      
Registry *      
Semantics *     Semantics WG comments should be addressed in next version
DCP        
Edu        
KDIG *      
Ops *      
Radio        
SSIG *      
Theory        
TD        
<nop>StdProc        

Deleted:
<
<

Revision 182023-11-12 - SteveGroom

 
META TOPICPARENT name="IvoaDataModel"

Spectrum 1.2 Proposed Recommendation: Request for Comments


Introduction

The Spectrum Data Model describes the structure of spectrophotometric datasets with spectral and temporal coordinates and associated metadata.

This update satisfies an enhancement request by IPAC to add support for the following use cases:

  • 1-D spectra from Spitzer’s Infrared Spectrograph have multiple spectral orders
    • Spectral orders can overlap in wavelength
    • Plotting a Spitzer spectrum without accounting for orders gives you a mess
  • SEDs often include limits on measurements
    • these can be upper or lower limits
    • Plotters need to indicate limits clearly to avoid scientific misunderstanding
    • SDM1.1 has an ad-hoc representation for upper limits (only) which may be usable, but is conceptually incorrect.
The main changes in the v1.2 document are:
  • Section 4.1.1: addition of 'order' and 'relorder' attributes
  • Section 4.6.3: addition of 'UpperLimit' and 'LowerLimit' to explicitly express these concepts
  • Corrected UTypes in VOTable example #1
  • Updated diagrams to proper UML with improved consistency with the schema and text.
Latest version of SpectrumDM can be found at: The GitHub repository for issues and model development can be found at: The IVOA Twiki page for the project can be found at:

Reference Interoperable Implementations

The implementation level for this RFE is not as comprehensive as we would typically look for in a REC.

The DM working group consulted with the TCG prior to the start of this RFC and it was agreed that given

  • this is a minor revision based on a specific institution request
  • the request is targeted to serve a very specific purpose
  • the model is not VO-DML compliant, so the level of effort required to produce additional implementations, serializations, validators is out of proportion to the impact of the RFE.
the level of implementation for this enhancement is acceptable.

Data Providers:

Applications:
  • IPAC Firefly Archive toolkit:
    • "For some time, IPAC's Firefly archive GUI toolkit has taken advantage of IVOA Spectral Data Model (SDM) v1.1 to provide users with automatic plots of spectra that are the result of a query or have been uploaded to the tool. The plotting capability uses the SDM to identify which data to plot on the x and y axes and to change units. Firefly has recently been updated to recognize the new utypes specified in the proposed Spectral Data Model v1.2, namely:

      spec:Spectrum.Data.SpectralAxis.Order
      spec:Spectrum.Data.FluxAxis.Accuracy.LowerLimit
      spec.Spectrum.Data.FluxAxis.Accuracy.UpperLimit

      The toolkit uses these new utypes to plot orders (as found in e.g. Spitzer spectra) as separate traces, and to plot upper and lower limits as arrows (as found in e.g. SEDs). These changes are currently in test and will be released in May. Upon release, these features will be available in IRSA, Rubin, and NED interfaces, all of which are built upon the open-source Firefly toolkit.

      In a future release, Firefly will recognize the spec:Spectrum.Data.SpectralAxis.RelOrder utype proposed in SDMv1.2. When Firefly encounters a spectrum with only RelOrder and not Order, it will automatically plot each unique RelOrder as a different trace, analogous to how it treats Order. If a spectrum contains both Order and RelOrder, Firefly will automatically plot traces based on Order but will provide the user a toggle to plot RelOrder."

Implementations Validators

There is no dedicated validator for Spectrum data model instances to update for this enhancement.

Since this is not a VO-DML compliant model, the mechanisms for

  • validating the data model itself
  • generating and validating example serializations in VOTable, XML, etc.
  • generating model based code
are not available.

Comments from the IVOA Community during RFC/TCG review period: 2023-09-11 - 2023-10-23

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Comment by JesusSalgado

This is a preliminary comment: Version 1.0 had a java library implemented by Kelly McCusker (not in IVOA anymore) and Jonathan McDowell that allowed parsing and writing of spectral data compatible with 1.0. It would be great to obtain a version of this library and adapt it for these minor changes.

This library was used by some tools (e.g. VOSpec)

JesusSalgado - 2023-09-22

Comments from TCG member during the RFC/TCG Review Period: 2023-09-11 - 2023-10-23

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

We appreciate the initiative to evolve the spectrum data model, which needs it.
Compared with other IVOA standards, there is a lot of work to be done in the document to bring it up to date with the advances made by other working groups since the initial version.
Paragraph 3.3 Units should be replaced by VO Units
paragraph 3.4 UCD should be updated
Usage of STC is a reference to STC 1.0


ref CoordSystem should be listed elsewhere to allow evolution of the document or presented as a possible choice to be consistent with the xsd schema
<xs:complexType name="coordFrameType">
<xs:annotation>
<xs:documentation>Simplification of STC version: RefPos is string</xs:documentation>

VOTable serialisation is 1.1 version
DM's initiative to unify data models seems to have a place in the redesign of spectrum data model, as it would be a textbook case.

minor thing
on VOTABLE serialisation example it seem that there is problem on validation for
ucd="sdm:spect.frame"
on <PARAM there is no datatype on 3 of them

Response by (MarkCresitelloDittmar - 2023-11-11)

  • Similar to the Semantics group comments, we wanted to keep the scope of this update focused on the requested updates. The items you mention are good candidates for additional updates, and we can add Issues for them in the Spectrum repository. I believe many were included in the Spectrum2.0 effort back in 2016 which was tabled partially from lack of implementation, and partially because of the DM model refactoring effort. The 'plan' for the evolution of this model has been to represent it as a specialization of the Cube model, much like TimeSeries. We're happy to work with anyone interested in exercising that workflow!.
  • Re: the invalid ucd. There are several invalid and incorrect ucds in the document, this is issue #7 in the Spectrum repository. It was decided in the context of the recent Cone Search (?) that the documented UCDs should not be consided normative. For this reason we did not perform a sweep to resolve and correct the listed ucds.
  • Re: PARAM with no datatype, this could be resolved with this update.
  • To address the concerns about the outdated practices and references, we will add two statement to the document
    • To the abstract (and landing page), a statement to the effect of: "This is a minor update to the model to satisfy a specific request. We acknowledge that there are elements to the model which do not follow current IVOA practices and would benefit from modernization. This was intentionally not done during this revision to minimize the impact on existing implementations. They can be addressed in a subsequent update to the model."
    • To the UCD section, a statement to the effect of: "The UCDs specified in this document are non-normative, and should be considered as suggested or example values to convey the intent. Users should take care to select and validate UCDs which are appropriate and valid for their data."

Data Access Layer Working Group

Good to see support of orders as this has been raised by the community.

-- JamesDempsey - 2023-11-07

Data Model Working Group

Grid & Web Services Working Group

Changes are minimal and controlled from the previous version. The only point raised about the reference implementation (there was one that could be modified easily, I think, to cope with the changes but it is quite old so I am not even sure if the code is recoverable) (reference, java library implemented by Kelly McCusker and Jonathan McDowell (maybe Jonathan could help on this)). This library is a full implementation able to read and write SpectrumDM 1.1 compatible files.

Maybe there is another implementation. Could you clarify it?

In any case, as the changes are fully controlled, we approve the update.

JesusSalgado - 2023-11-06

Response by: MarkCresitelloDittmar - 2023-11-10

I agree that this would be a good thing to work toward, and will make an effort to track down the library and see if there is a path toward updating it to V1.2.

Registry Working Group

Semantics Working Group

  • In section 3.3 Units, it is stated that the unit formatting standard is WCS/OGIP convention for unit. Is there any reason not to use the VOUnit formalism (see ongoing RFC)? It would be a pity if we can't converge here.

  • In section 3.4 UCDs there seems to be a typo at the end of the first paragraph: [...] so you can’t put a literal em.* in a UTYPE field [...]. Do you mean UCD field ?

  • In section 5.1 CoordSys Fields, the proposed list of terms should be submitted as a VEP to Semantics, in order to update the RefPosition vocabulary. And then refer to that vocabulary for acceptable values.

  • In section 5.1.5 STC, the same remark concerning the RefFrame vocabulary

  • On pages 17-19, the table mentions "TimeFrameUCD", SpaceFrameUCD ": I don't see what this refers to.

BaptisteCecconi - 2023-11-09

  • Response * The goal for this RFE was to keep the changes as focused on the request as possible. I agree with your statements and I suggest we add these as Issues on the Spectrum project to be addressed in a 'modernizing' RFE. I will correct the typo.

MarkCresitelloDittmar - 2023-11-10

  • General comment: I think it is a bad idea to adopt a document which so much outdated.

  • In UCD section, it is stated that UCDs should be case insensitive. If you use UCDs, you don't change the definition/usage/syntax of UCDs in tis document. This sentence MUST be removed.

BaptisteCecconi - 2023-11-11

  • see bullet in Applications section regarding addtions to the document abstract and UCD section to address the concerns about outdated modeling practices in the model.
MarkCresitelloDittmar - 2023-11-11

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Operations Interest Group

Added:
>
>		Nothing to add at this time. Encourage continuing efforts to adopt smaller changes as they can be generally agreed on, to help minimize delays involved in working out larger, more controversial topics. - Steve Groom - 2023-11-11
 

Radio Astronomy Interest Group

Solar System Interest Group

Nothing to add.

-- Anne Raugh - 2023-11-10

Theory Interest Group

Time Domain Interest Group

Standards and Processes Committee

TCG Vote : Vote_start_date - Vote_end_date

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Changed:
<
<
Apps *      
>
>
Apps *      
 
DAL *      
DM *      
GWS *      
Registry *      
Semantics *     Semantics WG comments should be addressed in next version
DCP        
Edu        
KDIG *      
Changed:
<
<
Ops        
>
>
Ops *      
 
Radio        
SSIG *      
Theory        
TD        
<nop>StdProc        

Revision 172023-11-12 - BaptisteCecconi

 
META TOPICPARENT name="IvoaDataModel"

Spectrum 1.2 Proposed Recommendation: Request for Comments


Introduction

The Spectrum Data Model describes the structure of spectrophotometric datasets with spectral and temporal coordinates and associated metadata.

This update satisfies an enhancement request by IPAC to add support for the following use cases:

  • 1-D spectra from Spitzer’s Infrared Spectrograph have multiple spectral orders
    • Spectral orders can overlap in wavelength
    • Plotting a Spitzer spectrum without accounting for orders gives you a mess
  • SEDs often include limits on measurements
    • these can be upper or lower limits
    • Plotters need to indicate limits clearly to avoid scientific misunderstanding
    • SDM1.1 has an ad-hoc representation for upper limits (only) which may be usable, but is conceptually incorrect.
The main changes in the v1.2 document are:
  • Section 4.1.1: addition of 'order' and 'relorder' attributes
  • Section 4.6.3: addition of 'UpperLimit' and 'LowerLimit' to explicitly express these concepts
  • Corrected UTypes in VOTable example #1
  • Updated diagrams to proper UML with improved consistency with the schema and text.
Latest version of SpectrumDM can be found at: The GitHub repository for issues and model development can be found at: The IVOA Twiki page for the project can be found at:

Reference Interoperable Implementations

The implementation level for this RFE is not as comprehensive as we would typically look for in a REC.

The DM working group consulted with the TCG prior to the start of this RFC and it was agreed that given

  • this is a minor revision based on a specific institution request
  • the request is targeted to serve a very specific purpose
  • the model is not VO-DML compliant, so the level of effort required to produce additional implementations, serializations, validators is out of proportion to the impact of the RFE.
the level of implementation for this enhancement is acceptable.

Data Providers:

Applications:
  • IPAC Firefly Archive toolkit:
    • "For some time, IPAC's Firefly archive GUI toolkit has taken advantage of IVOA Spectral Data Model (SDM) v1.1 to provide users with automatic plots of spectra that are the result of a query or have been uploaded to the tool. The plotting capability uses the SDM to identify which data to plot on the x and y axes and to change units. Firefly has recently been updated to recognize the new utypes specified in the proposed Spectral Data Model v1.2, namely:

      spec:Spectrum.Data.SpectralAxis.Order
      spec:Spectrum.Data.FluxAxis.Accuracy.LowerLimit
      spec.Spectrum.Data.FluxAxis.Accuracy.UpperLimit

      The toolkit uses these new utypes to plot orders (as found in e.g. Spitzer spectra) as separate traces, and to plot upper and lower limits as arrows (as found in e.g. SEDs). These changes are currently in test and will be released in May. Upon release, these features will be available in IRSA, Rubin, and NED interfaces, all of which are built upon the open-source Firefly toolkit.

      In a future release, Firefly will recognize the spec:Spectrum.Data.SpectralAxis.RelOrder utype proposed in SDMv1.2. When Firefly encounters a spectrum with only RelOrder and not Order, it will automatically plot each unique RelOrder as a different trace, analogous to how it treats Order. If a spectrum contains both Order and RelOrder, Firefly will automatically plot traces based on Order but will provide the user a toggle to plot RelOrder."

Implementations Validators

There is no dedicated validator for Spectrum data model instances to update for this enhancement.

Since this is not a VO-DML compliant model, the mechanisms for

  • validating the data model itself
  • generating and validating example serializations in VOTable, XML, etc.
  • generating model based code
are not available.

Comments from the IVOA Community during RFC/TCG review period: 2023-09-11 - 2023-10-23

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Comment by JesusSalgado

This is a preliminary comment: Version 1.0 had a java library implemented by Kelly McCusker (not in IVOA anymore) and Jonathan McDowell that allowed parsing and writing of spectral data compatible with 1.0. It would be great to obtain a version of this library and adapt it for these minor changes.

This library was used by some tools (e.g. VOSpec)

JesusSalgado - 2023-09-22

Comments from TCG member during the RFC/TCG Review Period: 2023-09-11 - 2023-10-23

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

We appreciate the initiative to evolve the spectrum data model, which needs it.
Compared with other IVOA standards, there is a lot of work to be done in the document to bring it up to date with the advances made by other working groups since the initial version.
Paragraph 3.3 Units should be replaced by VO Units
paragraph 3.4 UCD should be updated
Usage of STC is a reference to STC 1.0


ref CoordSystem should be listed elsewhere to allow evolution of the document or presented as a possible choice to be consistent with the xsd schema
<xs:complexType name="coordFrameType">
<xs:annotation>
<xs:documentation>Simplification of STC version: RefPos is string</xs:documentation>

VOTable serialisation is 1.1 version
DM's initiative to unify data models seems to have a place in the redesign of spectrum data model, as it would be a textbook case.

minor thing
on VOTABLE serialisation example it seem that there is problem on validation for
ucd="sdm:spect.frame"
on <PARAM there is no datatype on 3 of them

Response by (MarkCresitelloDittmar - 2023-11-11)

  • Similar to the Semantics group comments, we wanted to keep the scope of this update focused on the requested updates. The items you mention are good candidates for additional updates, and we can add Issues for them in the Spectrum repository. I believe many were included in the Spectrum2.0 effort back in 2016 which was tabled partially from lack of implementation, and partially because of the DM model refactoring effort. The 'plan' for the evolution of this model has been to represent it as a specialization of the Cube model, much like TimeSeries. We're happy to work with anyone interested in exercising that workflow!.
  • Re: the invalid ucd. There are several invalid and incorrect ucds in the document, this is issue #7 in the Spectrum repository. It was decided in the context of the recent Cone Search (?) that the documented UCDs should not be consided normative. For this reason we did not perform a sweep to resolve and correct the listed ucds.
  • Re: PARAM with no datatype, this could be resolved with this update.
  • To address the concerns about the outdated practices and references, we will add two statement to the document
    • To the abstract (and landing page), a statement to the effect of: "This is a minor update to the model to satisfy a specific request. We acknowledge that there are elements to the model which do not follow current IVOA practices and would benefit from modernization. This was intentionally not done during this revision to minimize the impact on existing implementations. They can be addressed in a subsequent update to the model."
    • To the UCD section, a statement to the effect of: "The UCDs specified in this document are non-normative, and should be considered as suggested or example values to convey the intent. Users should take care to select and validate UCDs which are appropriate and valid for their data."

Data Access Layer Working Group

Good to see support of orders as this has been raised by the community.

-- JamesDempsey - 2023-11-07

Data Model Working Group

Grid & Web Services Working Group

Changes are minimal and controlled from the previous version. The only point raised about the reference implementation (there was one that could be modified easily, I think, to cope with the changes but it is quite old so I am not even sure if the code is recoverable) (reference, java library implemented by Kelly McCusker and Jonathan McDowell (maybe Jonathan could help on this)). This library is a full implementation able to read and write SpectrumDM 1.1 compatible files.

Maybe there is another implementation. Could you clarify it?

In any case, as the changes are fully controlled, we approve the update.

JesusSalgado - 2023-11-06

Response by: MarkCresitelloDittmar - 2023-11-10

I agree that this would be a good thing to work toward, and will make an effort to track down the library and see if there is a path toward updating it to V1.2.

Registry Working Group

Semantics Working Group

  • In section 3.3 Units, it is stated that the unit formatting standard is WCS/OGIP convention for unit. Is there any reason not to use the VOUnit formalism (see ongoing RFC)? It would be a pity if we can't converge here.

  • In section 3.4 UCDs there seems to be a typo at the end of the first paragraph: [...] so you can’t put a literal em.* in a UTYPE field [...]. Do you mean UCD field ?

  • In section 5.1 CoordSys Fields, the proposed list of terms should be submitted as a VEP to Semantics, in order to update the RefPosition vocabulary. And then refer to that vocabulary for acceptable values.

  • In section 5.1.5 STC, the same remark concerning the RefFrame vocabulary

  • On pages 17-19, the table mentions "TimeFrameUCD", SpaceFrameUCD ": I don't see what this refers to.

BaptisteCecconi - 2023-11-09

  • Response * The goal for this RFE was to keep the changes as focused on the request as possible. I agree with your statements and I suggest we add these as Issues on the Spectrum project to be addressed in a 'modernizing' RFE. I will correct the typo.

MarkCresitelloDittmar - 2023-11-10

  • General comment: I think it is a bad idea to adopt a document which so much outdated.

  • In UCD section, it is stated that UCDs should be case insensitive. If you use UCDs, you don't change the definition/usage/syntax of UCDs in tis document. This sentence MUST be removed.

BaptisteCecconi - 2023-11-11

  • see bullet in Applications section regarding addtions to the document abstract and UCD section to address the concerns about outdated modeling practices in the model.
MarkCresitelloDittmar - 2023-11-11

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Operations Interest Group

Radio Astronomy Interest Group

Solar System Interest Group

Nothing to add.

-- Anne Raugh - 2023-11-10

Theory Interest Group

Time Domain Interest Group

Standards and Processes Committee

TCG Vote : Vote_start_date - Vote_end_date

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Changed:
<
<
Apps        
>
>
Apps *      
 
DAL *      
DM *      
GWS *      
Registry *      
Changed:
<
<
Semantics     * Semantics WG comments should be addressed in next version
>
>
Semantics *     Semantics WG comments should be addressed in next version
 
DCP        
Edu        
KDIG *      
Ops        
Radio        
SSIG *      
Theory        
TD        
<nop>StdProc        

Added:
>
>

Revision 162023-11-11 - MarkCresitelloDittmar

 
META TOPICPARENT name="IvoaDataModel"

Spectrum 1.2 Proposed Recommendation: Request for Comments


Introduction

The Spectrum Data Model describes the structure of spectrophotometric datasets with spectral and temporal coordinates and associated metadata.

This update satisfies an enhancement request by IPAC to add support for the following use cases:

  • 1-D spectra from Spitzer’s Infrared Spectrograph have multiple spectral orders
    • Spectral orders can overlap in wavelength
    • Plotting a Spitzer spectrum without accounting for orders gives you a mess
  • SEDs often include limits on measurements
    • these can be upper or lower limits
    • Plotters need to indicate limits clearly to avoid scientific misunderstanding
    • SDM1.1 has an ad-hoc representation for upper limits (only) which may be usable, but is conceptually incorrect.
The main changes in the v1.2 document are:
  • Section 4.1.1: addition of 'order' and 'relorder' attributes
  • Section 4.6.3: addition of 'UpperLimit' and 'LowerLimit' to explicitly express these concepts
  • Corrected UTypes in VOTable example #1
  • Updated diagrams to proper UML with improved consistency with the schema and text.
Latest version of SpectrumDM can be found at: The GitHub repository for issues and model development can be found at: The IVOA Twiki page for the project can be found at:

Reference Interoperable Implementations

The implementation level for this RFE is not as comprehensive as we would typically look for in a REC.

The DM working group consulted with the TCG prior to the start of this RFC and it was agreed that given

  • this is a minor revision based on a specific institution request
  • the request is targeted to serve a very specific purpose
  • the model is not VO-DML compliant, so the level of effort required to produce additional implementations, serializations, validators is out of proportion to the impact of the RFE.
the level of implementation for this enhancement is acceptable.

Data Providers:

Applications:
  • IPAC Firefly Archive toolkit:
    • "For some time, IPAC's Firefly archive GUI toolkit has taken advantage of IVOA Spectral Data Model (SDM) v1.1 to provide users with automatic plots of spectra that are the result of a query or have been uploaded to the tool. The plotting capability uses the SDM to identify which data to plot on the x and y axes and to change units. Firefly has recently been updated to recognize the new utypes specified in the proposed Spectral Data Model v1.2, namely:

      spec:Spectrum.Data.SpectralAxis.Order
      spec:Spectrum.Data.FluxAxis.Accuracy.LowerLimit
      spec.Spectrum.Data.FluxAxis.Accuracy.UpperLimit

      The toolkit uses these new utypes to plot orders (as found in e.g. Spitzer spectra) as separate traces, and to plot upper and lower limits as arrows (as found in e.g. SEDs). These changes are currently in test and will be released in May. Upon release, these features will be available in IRSA, Rubin, and NED interfaces, all of which are built upon the open-source Firefly toolkit.

      In a future release, Firefly will recognize the spec:Spectrum.Data.SpectralAxis.RelOrder utype proposed in SDMv1.2. When Firefly encounters a spectrum with only RelOrder and not Order, it will automatically plot each unique RelOrder as a different trace, analogous to how it treats Order. If a spectrum contains both Order and RelOrder, Firefly will automatically plot traces based on Order but will provide the user a toggle to plot RelOrder."

Implementations Validators

There is no dedicated validator for Spectrum data model instances to update for this enhancement.

Since this is not a VO-DML compliant model, the mechanisms for

  • validating the data model itself
  • generating and validating example serializations in VOTable, XML, etc.
  • generating model based code
are not available.

Comments from the IVOA Community during RFC/TCG review period: 2023-09-11 - 2023-10-23

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Comment by JesusSalgado

This is a preliminary comment: Version 1.0 had a java library implemented by Kelly McCusker (not in IVOA anymore) and Jonathan McDowell that allowed parsing and writing of spectral data compatible with 1.0. It would be great to obtain a version of this library and adapt it for these minor changes.

This library was used by some tools (e.g. VOSpec)

JesusSalgado - 2023-09-22

Comments from TCG member during the RFC/TCG Review Period: 2023-09-11 - 2023-10-23

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

We appreciate the initiative to evolve the spectrum data model, which needs it.
Compared with other IVOA standards, there is a lot of work to be done in the document to bring it up to date with the advances made by other working groups since the initial version.
Paragraph 3.3 Units should be replaced by VO Units
paragraph 3.4 UCD should be updated
Usage of STC is a reference to STC 1.0


ref CoordSystem should be listed elsewhere to allow evolution of the document or presented as a possible choice to be consistent with the xsd schema
<xs:complexType name="coordFrameType">
<xs:annotation>
<xs:documentation>Simplification of STC version: RefPos is string</xs:documentation>

VOTable serialisation is 1.1 version
DM's initiative to unify data models seems to have a place in the redesign of spectrum data model, as it would be a textbook case.

minor thing
on VOTABLE serialisation example it seem that there is problem on validation for
ucd="sdm:spect.frame"
on <PARAM there is no datatype on 3 of them

Changed:
<
<		Response by (MarkCresitelloDittmar - 2023-11-11)
>
>		Response by (MarkCresitelloDittmar - 2023-11-11)
Added:
>
>
  • Similar to the Semantics group comments, we wanted to keep the scope of this update focused on the requested updates. The items you mention are good candidates for additional updates, and we can add Issues for them in the Spectrum repository. I believe many were included in the Spectrum2.0 effort back in 2016 which was tabled partially from lack of implementation, and partially because of the DM model refactoring effort. The 'plan' for the evolution of this model has been to represent it as a specialization of the Cube model, much like TimeSeries. We're happy to work with anyone interested in exercising that workflow!.
  • Re: the invalid ucd. There are several invalid and incorrect ucds in the document, this is issue #7 in the Spectrum repository. It was decided in the context of the recent Cone Search (?) that the documented UCDs should not be consided normative. For this reason we did not perform a sweep to resolve and correct the listed ucds.
  • Re: PARAM with no datatype, this could be resolved with this update.
  • To address the concerns about the outdated practices and references, we will add two statement to the document
    • To the abstract (and landing page), a statement to the effect of: "This is a minor update to the model to satisfy a specific request. We acknowledge that there are elements to the model which do not follow current IVOA practices and would benefit from modernization. This was intentionally not done during this revision to minimize the impact on existing implementations. They can be addressed in a subsequent update to the model."
    • To the UCD section, a statement to the effect of: "The UCDs specified in this document are non-normative, and should be considered as suggested or example values to convey the intent. Users should take care to select and validate UCDs which are appropriate and valid for their data."
 
Deleted:
<
<		Similar to the Semantics group comments, we wanted to keep the scope of this update focused on the requested updates. The items you mention are good candidates for additional updates, and we can add Issues for them in the Spectrum repository. I believe many were included in the Spectrum2.0 effort back in 2016 which was tabled partially from lack of implementation, and partially because of the DM model refactoring effort. The 'plan' for the evolution of this model has been to represent it as a specialization of the Cube model, much like TimeSeries. We're happy to work with anyone interested in exercising that workflow!.

Re: the invalid ucd. There are several invalid and incorrect ucds in the document, this is issue #7 in the Spectrum repository. It was decided in the context of the recent Cone Search (?) that the documented UCDs should not be consided normative. For this reason we did not perform a sweep to resolve and correct the listed ucds.

Re: PARAM with no datatype, this could be resolved with this update.

 

Data Access Layer Working Group

Good to see support of orders as this has been raised by the community.

-- JamesDempsey - 2023-11-07

Data Model Working Group

Grid & Web Services Working Group

Changes are minimal and controlled from the previous version. The only point raised about the reference implementation (there was one that could be modified easily, I think, to cope with the changes but it is quite old so I am not even sure if the code is recoverable) (reference, java library implemented by Kelly McCusker and Jonathan McDowell (maybe Jonathan could help on this)). This library is a full implementation able to read and write SpectrumDM 1.1 compatible files.

Maybe there is another implementation. Could you clarify it?

In any case, as the changes are fully controlled, we approve the update.

JesusSalgado - 2023-11-06

Response by: MarkCresitelloDittmar - 2023-11-10

I agree that this would be a good thing to work toward, and will make an effort to track down the library and see if there is a path toward updating it to V1.2.

Registry Working Group

Semantics Working Group

  • In section 3.3 Units, it is stated that the unit formatting standard is WCS/OGIP convention for unit. Is there any reason not to use the VOUnit formalism (see ongoing RFC)? It would be a pity if we can't converge here.

  • In section 3.4 UCDs there seems to be a typo at the end of the first paragraph: [...] so you can’t put a literal em.* in a UTYPE field [...]. Do you mean UCD field ?

  • In section 5.1 CoordSys Fields, the proposed list of terms should be submitted as a VEP to Semantics, in order to update the RefPosition vocabulary. And then refer to that vocabulary for acceptable values.

  • In section 5.1.5 STC, the same remark concerning the RefFrame vocabulary
Changed:
<
<
  • On pages 17-19, the table mentions "TimeFrameUCD", SpaceFrameUCD ": I don't see what this refers to.
>
>
  • On pages 17-19, the table mentions "TimeFrameUCD", SpaceFrameUCD ": I don't see what this refers to.
  BaptisteCecconi - 2023-11-09
Changed:
<
<
  • Response * The goal for this RFE was to keep the changes as focused on the request as possible. I agree with your statements and I suggest we add these as Issues on the Spectrum project to be addressed in a 'modernizing' RFE. I will correct the typo.
>
>
  • Response * The goal for this RFE was to keep the changes as focused on the request as possible. I agree with your statements and I suggest we add these as Issues on the Spectrum project to be addressed in a 'modernizing' RFE. I will correct the typo.
  MarkCresitelloDittmar - 2023-11-10
Changed:
<
<
  • General comment: I think it is a bad idea to adopt a document which so much outdated.
>
>
  • General comment: I think it is a bad idea to adopt a document which so much outdated.
 
  • In UCD section, it is stated that UCDs should be case insensitive. If you use UCDs, you don't change the definition/usage/syntax of UCDs in tis document. This sentence MUST be removed.

BaptisteCecconi - 2023-11-11

Added:
>
>
  • see bullet in Applications section regarding addtions to the document abstract and UCD section to address the concerns about outdated modeling practices in the model.
MarkCresitelloDittmar - 2023-11-11
 

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Operations Interest Group

Radio Astronomy Interest Group

Solar System Interest Group

Nothing to add.

-- Anne Raugh - 2023-11-10

Theory Interest Group

Time Domain Interest Group

Standards and Processes Committee

TCG Vote : Vote_start_date - Vote_end_date

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL *      
DM *      
GWS *      
Registry *      
Semantics     * Semantics WG comments should be addressed in next version
DCP        
Edu        
KDIG *      
Ops        
Radio        
SSIG *      
Theory        
TD        
<nop>StdProc        

Deleted:
<
<

Revision 152023-11-11 - BaptisteCecconi

 
META TOPICPARENT name="IvoaDataModel"

Spectrum 1.2 Proposed Recommendation: Request for Comments


Introduction

The Spectrum Data Model describes the structure of spectrophotometric datasets with spectral and temporal coordinates and associated metadata.

This update satisfies an enhancement request by IPAC to add support for the following use cases:

  • 1-D spectra from Spitzer’s Infrared Spectrograph have multiple spectral orders
    • Spectral orders can overlap in wavelength
    • Plotting a Spitzer spectrum without accounting for orders gives you a mess
  • SEDs often include limits on measurements
    • these can be upper or lower limits
    • Plotters need to indicate limits clearly to avoid scientific misunderstanding
    • SDM1.1 has an ad-hoc representation for upper limits (only) which may be usable, but is conceptually incorrect.
The main changes in the v1.2 document are:
  • Section 4.1.1: addition of 'order' and 'relorder' attributes
  • Section 4.6.3: addition of 'UpperLimit' and 'LowerLimit' to explicitly express these concepts
  • Corrected UTypes in VOTable example #1
  • Updated diagrams to proper UML with improved consistency with the schema and text.
Latest version of SpectrumDM can be found at: The GitHub repository for issues and model development can be found at: The IVOA Twiki page for the project can be found at:

Reference Interoperable Implementations

The implementation level for this RFE is not as comprehensive as we would typically look for in a REC.

The DM working group consulted with the TCG prior to the start of this RFC and it was agreed that given

  • this is a minor revision based on a specific institution request
  • the request is targeted to serve a very specific purpose
  • the model is not VO-DML compliant, so the level of effort required to produce additional implementations, serializations, validators is out of proportion to the impact of the RFE.
the level of implementation for this enhancement is acceptable.

Data Providers:

Applications:
  • IPAC Firefly Archive toolkit:
    • "For some time, IPAC's Firefly archive GUI toolkit has taken advantage of IVOA Spectral Data Model (SDM) v1.1 to provide users with automatic plots of spectra that are the result of a query or have been uploaded to the tool. The plotting capability uses the SDM to identify which data to plot on the x and y axes and to change units. Firefly has recently been updated to recognize the new utypes specified in the proposed Spectral Data Model v1.2, namely:

      spec:Spectrum.Data.SpectralAxis.Order
      spec:Spectrum.Data.FluxAxis.Accuracy.LowerLimit
      spec.Spectrum.Data.FluxAxis.Accuracy.UpperLimit

      The toolkit uses these new utypes to plot orders (as found in e.g. Spitzer spectra) as separate traces, and to plot upper and lower limits as arrows (as found in e.g. SEDs). These changes are currently in test and will be released in May. Upon release, these features will be available in IRSA, Rubin, and NED interfaces, all of which are built upon the open-source Firefly toolkit.

      In a future release, Firefly will recognize the spec:Spectrum.Data.SpectralAxis.RelOrder utype proposed in SDMv1.2. When Firefly encounters a spectrum with only RelOrder and not Order, it will automatically plot each unique RelOrder as a different trace, analogous to how it treats Order. If a spectrum contains both Order and RelOrder, Firefly will automatically plot traces based on Order but will provide the user a toggle to plot RelOrder."

Implementations Validators

There is no dedicated validator for Spectrum data model instances to update for this enhancement.

Since this is not a VO-DML compliant model, the mechanisms for

  • validating the data model itself
  • generating and validating example serializations in VOTable, XML, etc.
  • generating model based code
are not available.

Comments from the IVOA Community during RFC/TCG review period: 2023-09-11 - 2023-10-23

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Comment by JesusSalgado

This is a preliminary comment: Version 1.0 had a java library implemented by Kelly McCusker (not in IVOA anymore) and Jonathan McDowell that allowed parsing and writing of spectral data compatible with 1.0. It would be great to obtain a version of this library and adapt it for these minor changes.

This library was used by some tools (e.g. VOSpec)

JesusSalgado - 2023-09-22

Comments from TCG member during the RFC/TCG Review Period: 2023-09-11 - 2023-10-23

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

We appreciate the initiative to evolve the spectrum data model, which needs it.
Compared with other IVOA standards, there is a lot of work to be done in the document to bring it up to date with the advances made by other working groups since the initial version.
Paragraph 3.3 Units should be replaced by VO Units
paragraph 3.4 UCD should be updated
Usage of STC is a reference to STC 1.0


ref CoordSystem should be listed elsewhere to allow evolution of the document or presented as a possible choice to be consistent with the xsd schema
<xs:complexType name="coordFrameType">
<xs:annotation>
<xs:documentation>Simplification of STC version: RefPos is string</xs:documentation>

VOTable serialisation is 1.1 version
DM's initiative to unify data models seems to have a place in the redesign of spectrum data model, as it would be a textbook case.

minor thing
on VOTABLE serialisation example it seem that there is problem on validation for
ucd="sdm:spect.frame"
on <PARAM there is no datatype on 3 of them

Response by (MarkCresitelloDittmar - 2023-11-11)

Similar to the Semantics group comments, we wanted to keep the scope of this update focused on the requested updates. The items you mention are good candidates for additional updates, and we can add Issues for them in the Spectrum repository. I believe many were included in the Spectrum2.0 effort back in 2016 which was tabled partially from lack of implementation, and partially because of the DM model refactoring effort. The 'plan' for the evolution of this model has been to represent it as a specialization of the Cube model, much like TimeSeries. We're happy to work with anyone interested in exercising that workflow!.

Re: the invalid ucd. There are several invalid and incorrect ucds in the document, this is issue #7 in the Spectrum repository. It was decided in the context of the recent Cone Search (?) that the documented UCDs should not be consided normative. For this reason we did not perform a sweep to resolve and correct the listed ucds.

Re: PARAM with no datatype, this could be resolved with this update.

Data Access Layer Working Group

Good to see support of orders as this has been raised by the community.

-- JamesDempsey - 2023-11-07

Data Model Working Group

Grid & Web Services Working Group

Changes are minimal and controlled from the previous version. The only point raised about the reference implementation (there was one that could be modified easily, I think, to cope with the changes but it is quite old so I am not even sure if the code is recoverable) (reference, java library implemented by Kelly McCusker and Jonathan McDowell (maybe Jonathan could help on this)). This library is a full implementation able to read and write SpectrumDM 1.1 compatible files.

Maybe there is another implementation. Could you clarify it?

In any case, as the changes are fully controlled, we approve the update.

JesusSalgado - 2023-11-06

Response by: MarkCresitelloDittmar - 2023-11-10

I agree that this would be a good thing to work toward, and will make an effort to track down the library and see if there is a path toward updating it to V1.2.

Registry Working Group

Semantics Working Group

Changed:
<
<		* In section 3.3 Units, it is stated that the unit formatting standard is WCS/OGIP convention for unit. Is there any reason not to use the VOUnit formalism (see ongoing RFC)? It would be a pity if we can't converge here.
>
>
  • In section 3.3 Units, it is stated that the unit formatting standard is WCS/OGIP convention for unit. Is there any reason not to use the VOUnit formalism (see ongoing RFC)? It would be a pity if we can't converge here.
 
Changed:
<
<		* In section 3.4 UCDs there seems to be a typo at the end of the first paragraph: [...] so you can’t put a literal em.* in a UTYPE field [...]. Do you mean UCD field ?
>
>
  • In section 3.4 UCDs there seems to be a typo at the end of the first paragraph: [...] so you can’t put a literal em.* in a UTYPE field [...]. Do you mean UCD field ?
 
Changed:
<
<		* In section 5.1 CoordSys Fields, the proposed list of terms should be submitted as a VEP to Semantics, in order to update the RefPosition vocabulary. And then refer to that vocabulary for acceptable values.
>
>
  • In section 5.1 CoordSys Fields, the proposed list of terms should be submitted as a VEP to Semantics, in order to update the RefPosition vocabulary. And then refer to that vocabulary for acceptable values.
 
Changed:
<
<		* In section 5.1.5 STC, the same remark concerning the RefFrame vocabulary
>
>
  • In section 5.1.5 STC, the same remark concerning the RefFrame vocabulary
 
Changed:
<
<		* On pages 17-19, the table mentions "TimeFrameUCD", SpaceFrameUCD ": I don't see what this refers to.
>
>
  • On pages 17-19, the table mentions "TimeFrameUCD", SpaceFrameUCD ": I don't see what this refers to.
 
Deleted:
<
<		 * Response MarkCresitelloDittmar - 2023-11-10 * The goal for this RFE was to keep the changes as focused on the request as possible. I agree with your statements and I suggest we add these as Issues on the Spectrum project to be addressed in a 'modernizing' RFE. I will correct the typo.
 BaptisteCecconi - 2023-11-09
Added:
>
>
  • Response * The goal for this RFE was to keep the changes as focused on the request as possible. I agree with your statements and I suggest we add these as Issues on the Spectrum project to be addressed in a 'modernizing' RFE. I will correct the typo.

MarkCresitelloDittmar - 2023-11-10

  • General comment: I think it is a bad idea to adopt a document which so much outdated.

  • In UCD section, it is stated that UCDs should be case insensitive. If you use UCDs, you don't change the definition/usage/syntax of UCDs in tis document. This sentence MUST be removed.

BaptisteCecconi - 2023-11-11

 

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Operations Interest Group

Radio Astronomy Interest Group

Solar System Interest Group

Nothing to add.

-- Anne Raugh - 2023-11-10

Theory Interest Group

Time Domain Interest Group

Standards and Processes Committee

TCG Vote : Vote_start_date - Vote_end_date

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL *      
DM *      
GWS *      
Registry *      
Changed:
<
<
Semantics *     Semantics WG comments should be addressed in next version
>
>
Semantics     * Semantics WG comments should be addressed in next version
 
DCP        
Edu        
KDIG *      
Ops        
Radio        
SSIG *      
Theory        
TD        
<nop>StdProc        

Added:
>
>

Revision 142023-11-11 - MarkCresitelloDittmar

 
META TOPICPARENT name="IvoaDataModel"

Spectrum 1.2 Proposed Recommendation: Request for Comments

Changed:
<
<
>
>
 

Introduction

The Spectrum Data Model describes the structure of spectrophotometric datasets with spectral and temporal coordinates and associated metadata.

This update satisfies an enhancement request by IPAC to add support for the following use cases:

  • 1-D spectra from Spitzer’s Infrared Spectrograph have multiple spectral orders
    • Spectral orders can overlap in wavelength
    • Plotting a Spitzer spectrum without accounting for orders gives you a mess
  • SEDs often include limits on measurements
    • these can be upper or lower limits
    • Plotters need to indicate limits clearly to avoid scientific misunderstanding
    • SDM1.1 has an ad-hoc representation for upper limits (only) which may be usable, but is conceptually incorrect.
The main changes in the v1.2 document are:
  • Section 4.1.1: addition of 'order' and 'relorder' attributes
  • Section 4.6.3: addition of 'UpperLimit' and 'LowerLimit' to explicitly express these concepts
  • Corrected UTypes in VOTable example #1
  • Updated diagrams to proper UML with improved consistency with the schema and text.
Latest version of SpectrumDM can be found at: The GitHub repository for issues and model development can be found at: The IVOA Twiki page for the project can be found at:

Reference Interoperable Implementations

The implementation level for this RFE is not as comprehensive as we would typically look for in a REC.

The DM working group consulted with the TCG prior to the start of this RFC and it was agreed that given

  • this is a minor revision based on a specific institution request
  • the request is targeted to serve a very specific purpose
  • the model is not VO-DML compliant, so the level of effort required to produce additional implementations, serializations, validators is out of proportion to the impact of the RFE.
the level of implementation for this enhancement is acceptable.

Data Providers:

Applications:
  • IPAC Firefly Archive toolkit:
    • "For some time, IPAC's Firefly archive GUI toolkit has taken advantage of IVOA Spectral Data Model (SDM) v1.1 to provide users with automatic plots of spectra that are the result of a query or have been uploaded to the tool. The plotting capability uses the SDM to identify which data to plot on the x and y axes and to change units. Firefly has recently been updated to recognize the new utypes specified in the proposed Spectral Data Model v1.2, namely:

      spec:Spectrum.Data.SpectralAxis.Order
      spec:Spectrum.Data.FluxAxis.Accuracy.LowerLimit
      spec.Spectrum.Data.FluxAxis.Accuracy.UpperLimit

      The toolkit uses these new utypes to plot orders (as found in e.g. Spitzer spectra) as separate traces, and to plot upper and lower limits as arrows (as found in e.g. SEDs). These changes are currently in test and will be released in May. Upon release, these features will be available in IRSA, Rubin, and NED interfaces, all of which are built upon the open-source Firefly toolkit.

      In a future release, Firefly will recognize the spec:Spectrum.Data.SpectralAxis.RelOrder utype proposed in SDMv1.2. When Firefly encounters a spectrum with only RelOrder and not Order, it will automatically plot each unique RelOrder as a different trace, analogous to how it treats Order. If a spectrum contains both Order and RelOrder, Firefly will automatically plot traces based on Order but will provide the user a toggle to plot RelOrder."

Implementations Validators

There is no dedicated validator for Spectrum data model instances to update for this enhancement.

Since this is not a VO-DML compliant model, the mechanisms for

  • validating the data model itself
  • generating and validating example serializations in VOTable, XML, etc.
  • generating model based code
are not available.

Comments from the IVOA Community during RFC/TCG review period: 2023-09-11 - 2023-10-23

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Comment by JesusSalgado

This is a preliminary comment: Version 1.0 had a java library implemented by Kelly McCusker (not in IVOA anymore) and Jonathan McDowell that allowed parsing and writing of spectral data compatible with 1.0. It would be great to obtain a version of this library and adapt it for these minor changes.

This library was used by some tools (e.g. VOSpec)

JesusSalgado - 2023-09-22

Changed:
<
<

Comments from TCG member during the RFC/TCG Review Period: 2023-09-11 - 2023-10-23

>
>

Comments from TCG member during the RFC/TCG Review Period: 2023-09-11 - 2023-10-23

  WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

We appreciate the initiative to evolve the spectrum data model, which needs it.
Compared with other IVOA standards, there is a lot of work to be done in the document to bring it up to date with the advances made by other working groups since the initial version.
Paragraph 3.3 Units should be replaced by VO Units
paragraph 3.4 UCD should be updated
Usage of STC is a reference to STC 1.0

Changed:
<
<
ref CoordSystem should be listed elsewhere to allow evolution of the document or presented as a possible choice to be consistent with the xsd schema
<xs:complexType name="coordFrameType">
<xs:annotation>
<xs:documentation>Simplification of STC version: RefPos is string</xs:documentation>

VOTable serialisation is 1.1 version
DM's initiative to unify data models seems to have a place in the redesign of spectrum data model, as it would be a textbook case.
>
>
ref CoordSystem should be listed elsewhere to allow evolution of the document or presented as a possible choice to be consistent with the xsd schema
<xs:complexType name="coordFrameType">
<xs:annotation>
<xs:documentation>Simplification of STC version: RefPos is string</xs:documentation>

VOTable serialisation is 1.1 version
DM's initiative to unify data models seems to have a place in the redesign of spectrum data model, as it would be a textbook case.
  minor thing
on VOTABLE serialisation example it seem that there is problem on validation for
ucd="sdm:spect.frame"
on <PARAM there is no datatype on 3 of them

Added:
>
>		 Response by (MarkCresitelloDittmar - 2023-11-11)

Similar to the Semantics group comments, we wanted to keep the scope of this update focused on the requested updates. The items you mention are good candidates for additional updates, and we can add Issues for them in the Spectrum repository. I believe many were included in the Spectrum2.0 effort back in 2016 which was tabled partially from lack of implementation, and partially because of the DM model refactoring effort. The 'plan' for the evolution of this model has been to represent it as a specialization of the Cube model, much like TimeSeries. We're happy to work with anyone interested in exercising that workflow!.

Re: the invalid ucd. There are several invalid and incorrect ucds in the document, this is issue #7 in the Spectrum repository. It was decided in the context of the recent Cone Search (?) that the documented UCDs should not be consided normative. For this reason we did not perform a sweep to resolve and correct the listed ucds.

Re: PARAM with no datatype, this could be resolved with this update.

 

Data Access Layer Working Group

Good to see support of orders as this has been raised by the community.

-- JamesDempsey - 2023-11-07

Data Model Working Group

Grid & Web Services Working Group

Changes are minimal and controlled from the previous version. The only point raised about the reference implementation (there was one that could be modified easily, I think, to cope with the changes but it is quite old so I am not even sure if the code is recoverable) (reference, java library implemented by Kelly McCusker and Jonathan McDowell (maybe Jonathan could help on this)). This library is a full implementation able to read and write SpectrumDM 1.1 compatible files.

Maybe there is another implementation. Could you clarify it?

In any case, as the changes are fully controlled, we approve the update.

JesusSalgado - 2023-11-06

Changed:
<
<		 * Response MarkCresitelloDittmar - 2023-11-10
>
>		Response by: MarkCresitelloDittmar - 2023-11-10
Deleted:
<
<		 * I agree that this would be a good thing to work toward, and will make an effort to track down the library and see if there is a path toward updating it to V1.2.
 
Added:
>
>		I agree that this would be a good thing to work toward, and will make an effort to track down the library and see if there is a path toward updating it to V1.2.
 

Registry Working Group

Semantics Working Group

* In section 3.3 Units, it is stated that the unit formatting standard is WCS/OGIP convention for unit. Is there any reason not to use the VOUnit formalism (see ongoing RFC)? It would be a pity if we can't converge here.

* In section 3.4 UCDs there seems to be a typo at the end of the first paragraph: [...] so you can’t put a literal em.* in a UTYPE field [...]. Do you mean UCD field ?

* In section 5.1 CoordSys Fields, the proposed list of terms should be submitted as a VEP to Semantics, in order to update the RefPosition vocabulary. And then refer to that vocabulary for acceptable values.

* In section 5.1.5 STC, the same remark concerning the RefFrame vocabulary

* On pages 17-19, the table mentions "TimeFrameUCD", SpaceFrameUCD ": I don't see what this refers to.

Changed:
<
<		 * Response MarkCresitelloDittmar - 2023-11-10
>
>		 * Response MarkCresitelloDittmar - 2023-11-10 * The goal for this RFE was to keep the changes as focused on the request as possible. I agree with your statements and I suggest we add these as Issues on the Spectrum project to be addressed in a 'modernizing' RFE. I will correct the typo.
Deleted:
<
<		 * The goal for this RFE was to keep the changes as focused on the request as possible. I agree with your statements and I suggest we add these as Issues on the Spectrum project to be addressed in a 'modernizing' RFE. I will correct the typo.
  BaptisteCecconi - 2023-11-09

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Operations Interest Group

Radio Astronomy Interest Group

Solar System Interest Group

Nothing to add.

-- Anne Raugh - 2023-11-10

Theory Interest Group

Time Domain Interest Group

Standards and Processes Committee

Changed:
<
<

TCG Vote : Vote_start_date - Vote_end_date

>
>

TCG Vote : Vote_start_date - Vote_end_date

  If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL *      
DM *      
GWS *      
Registry *      
Semantics *     Semantics WG comments should be addressed in next version
DCP        
Edu        
KDIG *      
Ops        
Radio        
SSIG *      
Theory        
TD        
Changed:
<
<
StdProc        
>
>
<nop>StdProc        
 
Deleted:
<
<

Revision 132023-11-11 - PierreLeSidaner

 
META TOPICPARENT name="IvoaDataModel"

Spectrum 1.2 Proposed Recommendation: Request for Comments

Introduction

The Spectrum Data Model describes the structure of spectrophotometric datasets with spectral and temporal coordinates and associated metadata.

This update satisfies an enhancement request by IPAC to add support for the following use cases:

  • 1-D spectra from Spitzer’s Infrared Spectrograph have multiple spectral orders
    • Spectral orders can overlap in wavelength
    • Plotting a Spitzer spectrum without accounting for orders gives you a mess
  • SEDs often include limits on measurements
    • these can be upper or lower limits
    • Plotters need to indicate limits clearly to avoid scientific misunderstanding
    • SDM1.1 has an ad-hoc representation for upper limits (only) which may be usable, but is conceptually incorrect.
The main changes in the v1.2 document are:
  • Section 4.1.1: addition of 'order' and 'relorder' attributes
  • Section 4.6.3: addition of 'UpperLimit' and 'LowerLimit' to explicitly express these concepts
  • Corrected UTypes in VOTable example #1
  • Updated diagrams to proper UML with improved consistency with the schema and text.
Latest version of SpectrumDM can be found at: The GitHub repository for issues and model development can be found at: The IVOA Twiki page for the project can be found at:

Reference Interoperable Implementations

The implementation level for this RFE is not as comprehensive as we would typically look for in a REC.

The DM working group consulted with the TCG prior to the start of this RFC and it was agreed that given

  • this is a minor revision based on a specific institution request
  • the request is targeted to serve a very specific purpose
  • the model is not VO-DML compliant, so the level of effort required to produce additional implementations, serializations, validators is out of proportion to the impact of the RFE.
the level of implementation for this enhancement is acceptable.

Data Providers:

Applications:
  • IPAC Firefly Archive toolkit:
    • "For some time, IPAC's Firefly archive GUI toolkit has taken advantage of IVOA Spectral Data Model (SDM) v1.1 to provide users with automatic plots of spectra that are the result of a query or have been uploaded to the tool. The plotting capability uses the SDM to identify which data to plot on the x and y axes and to change units. Firefly has recently been updated to recognize the new utypes specified in the proposed Spectral Data Model v1.2, namely:

      spec:Spectrum.Data.SpectralAxis.Order
      spec:Spectrum.Data.FluxAxis.Accuracy.LowerLimit
      spec.Spectrum.Data.FluxAxis.Accuracy.UpperLimit

      The toolkit uses these new utypes to plot orders (as found in e.g. Spitzer spectra) as separate traces, and to plot upper and lower limits as arrows (as found in e.g. SEDs). These changes are currently in test and will be released in May. Upon release, these features will be available in IRSA, Rubin, and NED interfaces, all of which are built upon the open-source Firefly toolkit.

      In a future release, Firefly will recognize the spec:Spectrum.Data.SpectralAxis.RelOrder utype proposed in SDMv1.2. When Firefly encounters a spectrum with only RelOrder and not Order, it will automatically plot each unique RelOrder as a different trace, analogous to how it treats Order. If a spectrum contains both Order and RelOrder, Firefly will automatically plot traces based on Order but will provide the user a toggle to plot RelOrder."

Implementations Validators

There is no dedicated validator for Spectrum data model instances to update for this enhancement.

Since this is not a VO-DML compliant model, the mechanisms for

  • validating the data model itself
  • generating and validating example serializations in VOTable, XML, etc.
  • generating model based code
are not available.

Comments from the IVOA Community during RFC/TCG review period: 2023-09-11 - 2023-10-23

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Comment by JesusSalgado

This is a preliminary comment: Version 1.0 had a java library implemented by Kelly McCusker (not in IVOA anymore) and Jonathan McDowell that allowed parsing and writing of spectral data compatible with 1.0. It would be great to obtain a version of this library and adapt it for these minor changes.

This library was used by some tools (e.g. VOSpec)

JesusSalgado - 2023-09-22

Comments from TCG member during the RFC/TCG Review Period: 2023-09-11 - 2023-10-23

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Added:
>
>		We appreciate the initiative to evolve the spectrum data model, which needs it.
Compared with other IVOA standards, there is a lot of work to be done in the document to bring it up to date with the advances made by other working groups since the initial version.
Paragraph 3.3 Units should be replaced by VO Units
paragraph 3.4 UCD should be updated
Usage of STC is a reference to STC 1.0


ref CoordSystem should be listed elsewhere to allow evolution of the document or presented as a possible choice to be consistent with the xsd schema
<xs:complexType name="coordFrameType">
<xs:annotation>
<xs:documentation>Simplification of STC version: RefPos is string</xs:documentation>

VOTable serialisation is 1.1 version
DM's initiative to unify data models seems to have a place in the redesign of spectrum data model, as it would be a textbook case.

minor thing
on VOTABLE serialisation example it seem that there is problem on validation for
ucd="sdm:spect.frame"
on <PARAM there is no datatype on 3 of them

 

Data Access Layer Working Group

Good to see support of orders as this has been raised by the community.

-- JamesDempsey - 2023-11-07

Data Model Working Group

Grid & Web Services Working Group

Changes are minimal and controlled from the previous version. The only point raised about the reference implementation (there was one that could be modified easily, I think, to cope with the changes but it is quite old so I am not even sure if the code is recoverable) (reference, java library implemented by Kelly McCusker and Jonathan McDowell (maybe Jonathan could help on this)). This library is a full implementation able to read and write SpectrumDM 1.1 compatible files.

Maybe there is another implementation. Could you clarify it?

In any case, as the changes are fully controlled, we approve the update.

JesusSalgado - 2023-11-06

* Response MarkCresitelloDittmar - 2023-11-10 * I agree that this would be a good thing to work toward, and will make an effort to track down the library and see if there is a path toward updating it to V1.2.

Registry Working Group

Semantics Working Group

* In section 3.3 Units, it is stated that the unit formatting standard is WCS/OGIP convention for unit. Is there any reason not to use the VOUnit formalism (see ongoing RFC)? It would be a pity if we can't converge here.

* In section 3.4 UCDs there seems to be a typo at the end of the first paragraph: [...] so you can’t put a literal em.* in a UTYPE field [...]. Do you mean UCD field ?

* In section 5.1 CoordSys Fields, the proposed list of terms should be submitted as a VEP to Semantics, in order to update the RefPosition vocabulary. And then refer to that vocabulary for acceptable values.

* In section 5.1.5 STC, the same remark concerning the RefFrame vocabulary

* On pages 17-19, the table mentions "TimeFrameUCD", SpaceFrameUCD ": I don't see what this refers to.

* Response MarkCresitelloDittmar - 2023-11-10 * The goal for this RFE was to keep the changes as focused on the request as possible. I agree with your statements and I suggest we add these as Issues on the Spectrum project to be addressed in a 'modernizing' RFE. I will correct the typo.

BaptisteCecconi - 2023-11-09

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Operations Interest Group

Radio Astronomy Interest Group

Solar System Interest Group

Nothing to add.

-- Anne Raugh - 2023-11-10

Theory Interest Group

Time Domain Interest Group

Standards and Processes Committee

TCG Vote : Vote_start_date - Vote_end_date

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL *      
DM *      
GWS *      
Registry *      
Semantics *     Semantics WG comments should be addressed in next version
DCP        
Edu        
KDIG *      
Ops        
Radio        
SSIG *      
Theory        
TD        
StdProc        

Revision 122023-11-10 - MarkCresitelloDittmar

 
META TOPICPARENT name="IvoaDataModel"

Spectrum 1.2 Proposed Recommendation: Request for Comments

Introduction

The Spectrum Data Model describes the structure of spectrophotometric datasets with spectral and temporal coordinates and associated metadata.

This update satisfies an enhancement request by IPAC to add support for the following use cases:

  • 1-D spectra from Spitzer’s Infrared Spectrograph have multiple spectral orders
    • Spectral orders can overlap in wavelength
    • Plotting a Spitzer spectrum without accounting for orders gives you a mess
  • SEDs often include limits on measurements
    • these can be upper or lower limits
    • Plotters need to indicate limits clearly to avoid scientific misunderstanding
    • SDM1.1 has an ad-hoc representation for upper limits (only) which may be usable, but is conceptually incorrect.
The main changes in the v1.2 document are:
  • Section 4.1.1: addition of 'order' and 'relorder' attributes
  • Section 4.6.3: addition of 'UpperLimit' and 'LowerLimit' to explicitly express these concepts
  • Corrected UTypes in VOTable example #1
  • Updated diagrams to proper UML with improved consistency with the schema and text.
Latest version of SpectrumDM can be found at: The GitHub repository for issues and model development can be found at: The IVOA Twiki page for the project can be found at:

Reference Interoperable Implementations

The implementation level for this RFE is not as comprehensive as we would typically look for in a REC.

The DM working group consulted with the TCG prior to the start of this RFC and it was agreed that given

  • this is a minor revision based on a specific institution request
  • the request is targeted to serve a very specific purpose
  • the model is not VO-DML compliant, so the level of effort required to produce additional implementations, serializations, validators is out of proportion to the impact of the RFE.
the level of implementation for this enhancement is acceptable.

Data Providers:

Applications:
  • IPAC Firefly Archive toolkit:
    • "For some time, IPAC's Firefly archive GUI toolkit has taken advantage of IVOA Spectral Data Model (SDM) v1.1 to provide users with automatic plots of spectra that are the result of a query or have been uploaded to the tool. The plotting capability uses the SDM to identify which data to plot on the x and y axes and to change units. Firefly has recently been updated to recognize the new utypes specified in the proposed Spectral Data Model v1.2, namely:

      spec:Spectrum.Data.SpectralAxis.Order
      spec:Spectrum.Data.FluxAxis.Accuracy.LowerLimit
      spec.Spectrum.Data.FluxAxis.Accuracy.UpperLimit

      The toolkit uses these new utypes to plot orders (as found in e.g. Spitzer spectra) as separate traces, and to plot upper and lower limits as arrows (as found in e.g. SEDs). These changes are currently in test and will be released in May. Upon release, these features will be available in IRSA, Rubin, and NED interfaces, all of which are built upon the open-source Firefly toolkit.

      In a future release, Firefly will recognize the spec:Spectrum.Data.SpectralAxis.RelOrder utype proposed in SDMv1.2. When Firefly encounters a spectrum with only RelOrder and not Order, it will automatically plot each unique RelOrder as a different trace, analogous to how it treats Order. If a spectrum contains both Order and RelOrder, Firefly will automatically plot traces based on Order but will provide the user a toggle to plot RelOrder."

Implementations Validators

There is no dedicated validator for Spectrum data model instances to update for this enhancement.

Since this is not a VO-DML compliant model, the mechanisms for

  • validating the data model itself
  • generating and validating example serializations in VOTable, XML, etc.
  • generating model based code
are not available.

Comments from the IVOA Community during RFC/TCG review period: 2023-09-11 - 2023-10-23

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Comment by JesusSalgado

This is a preliminary comment: Version 1.0 had a java library implemented by Kelly McCusker (not in IVOA anymore) and Jonathan McDowell that allowed parsing and writing of spectral data compatible with 1.0. It would be great to obtain a version of this library and adapt it for these minor changes.

This library was used by some tools (e.g. VOSpec)

JesusSalgado - 2023-09-22

Comments from TCG member during the RFC/TCG Review Period: 2023-09-11 - 2023-10-23

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

Good to see support of orders as this has been raised by the community.

-- JamesDempsey - 2023-11-07

Data Model Working Group

Grid & Web Services Working Group

Changes are minimal and controlled from the previous version. The only point raised about the reference implementation (there was one that could be modified easily, I think, to cope with the changes but it is quite old so I am not even sure if the code is recoverable) (reference, java library implemented by Kelly McCusker and Jonathan McDowell (maybe Jonathan could help on this)). This library is a full implementation able to read and write SpectrumDM 1.1 compatible files.

Maybe there is another implementation. Could you clarify it?

In any case, as the changes are fully controlled, we approve the update.

JesusSalgado - 2023-11-06

Added:
>
>		 * Response MarkCresitelloDittmar - 2023-11-10 * I agree that this would be a good thing to work toward, and will make an effort to track down the library and see if there is a path toward updating it to V1.2.
 

Registry Working Group

Semantics Working Group

* In section 3.3 Units, it is stated that the unit formatting standard is WCS/OGIP convention for unit. Is there any reason not to use the VOUnit formalism (see ongoing RFC)? It would be a pity if we can't converge here.

* In section 3.4 UCDs there seems to be a typo at the end of the first paragraph: [...] so you can’t put a literal em.* in a UTYPE field [...]. Do you mean UCD field ?

* In section 5.1 CoordSys Fields, the proposed list of terms should be submitted as a VEP to Semantics, in order to update the RefPosition vocabulary. And then refer to that vocabulary for acceptable values.

* In section 5.1.5 STC, the same remark concerning the RefFrame vocabulary

* On pages 17-19, the table mentions "TimeFrameUCD", SpaceFrameUCD ": I don't see what this refers to.

Added:
>
>		 * Response MarkCresitelloDittmar - 2023-11-10 * The goal for this RFE was to keep the changes as focused on the request as possible. I agree with your statements and I suggest we add these as Issues on the Spectrum project to be addressed in a 'modernizing' RFE. I will correct the typo.
 BaptisteCecconi - 2023-11-09

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Operations Interest Group

Radio Astronomy Interest Group

Solar System Interest Group

Nothing to add.

-- Anne Raugh - 2023-11-10

Theory Interest Group

Time Domain Interest Group

Standards and Processes Committee

TCG Vote : Vote_start_date - Vote_end_date

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL *      
DM *      
GWS *      
Registry *      
Semantics *     Semantics WG comments should be addressed in next version
DCP        
Edu        
KDIG *      
Ops        
Radio        
SSIG *      
Theory        
TD        
StdProc        

Added:
>
>

Revision 112023-11-10 - AnneRaugh

 
META TOPICPARENT name="IvoaDataModel"

Spectrum 1.2 Proposed Recommendation: Request for Comments

Introduction

The Spectrum Data Model describes the structure of spectrophotometric datasets with spectral and temporal coordinates and associated metadata.

This update satisfies an enhancement request by IPAC to add support for the following use cases:

  • 1-D spectra from Spitzer’s Infrared Spectrograph have multiple spectral orders
    • Spectral orders can overlap in wavelength
    • Plotting a Spitzer spectrum without accounting for orders gives you a mess
  • SEDs often include limits on measurements
    • these can be upper or lower limits
    • Plotters need to indicate limits clearly to avoid scientific misunderstanding
    • SDM1.1 has an ad-hoc representation for upper limits (only) which may be usable, but is conceptually incorrect.
The main changes in the v1.2 document are:
  • Section 4.1.1: addition of 'order' and 'relorder' attributes
  • Section 4.6.3: addition of 'UpperLimit' and 'LowerLimit' to explicitly express these concepts
  • Corrected UTypes in VOTable example #1
  • Updated diagrams to proper UML with improved consistency with the schema and text.
Latest version of SpectrumDM can be found at: The GitHub repository for issues and model development can be found at: The IVOA Twiki page for the project can be found at:

Reference Interoperable Implementations

The implementation level for this RFE is not as comprehensive as we would typically look for in a REC.

The DM working group consulted with the TCG prior to the start of this RFC and it was agreed that given

  • this is a minor revision based on a specific institution request
  • the request is targeted to serve a very specific purpose
  • the model is not VO-DML compliant, so the level of effort required to produce additional implementations, serializations, validators is out of proportion to the impact of the RFE.
the level of implementation for this enhancement is acceptable.

Data Providers:

Applications:
  • IPAC Firefly Archive toolkit:
    • "For some time, IPAC's Firefly archive GUI toolkit has taken advantage of IVOA Spectral Data Model (SDM) v1.1 to provide users with automatic plots of spectra that are the result of a query or have been uploaded to the tool. The plotting capability uses the SDM to identify which data to plot on the x and y axes and to change units. Firefly has recently been updated to recognize the new utypes specified in the proposed Spectral Data Model v1.2, namely:

      spec:Spectrum.Data.SpectralAxis.Order
      spec:Spectrum.Data.FluxAxis.Accuracy.LowerLimit
      spec.Spectrum.Data.FluxAxis.Accuracy.UpperLimit

      The toolkit uses these new utypes to plot orders (as found in e.g. Spitzer spectra) as separate traces, and to plot upper and lower limits as arrows (as found in e.g. SEDs). These changes are currently in test and will be released in May. Upon release, these features will be available in IRSA, Rubin, and NED interfaces, all of which are built upon the open-source Firefly toolkit.

      In a future release, Firefly will recognize the spec:Spectrum.Data.SpectralAxis.RelOrder utype proposed in SDMv1.2. When Firefly encounters a spectrum with only RelOrder and not Order, it will automatically plot each unique RelOrder as a different trace, analogous to how it treats Order. If a spectrum contains both Order and RelOrder, Firefly will automatically plot traces based on Order but will provide the user a toggle to plot RelOrder."

Implementations Validators

There is no dedicated validator for Spectrum data model instances to update for this enhancement.

Since this is not a VO-DML compliant model, the mechanisms for

  • validating the data model itself
  • generating and validating example serializations in VOTable, XML, etc.
  • generating model based code
are not available.

Comments from the IVOA Community during RFC/TCG review period: 2023-09-11 - 2023-10-23

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Comment by JesusSalgado

This is a preliminary comment: Version 1.0 had a java library implemented by Kelly McCusker (not in IVOA anymore) and Jonathan McDowell that allowed parsing and writing of spectral data compatible with 1.0. It would be great to obtain a version of this library and adapt it for these minor changes.

This library was used by some tools (e.g. VOSpec)

JesusSalgado - 2023-09-22

Comments from TCG member during the RFC/TCG Review Period: 2023-09-11 - 2023-10-23

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

Good to see support of orders as this has been raised by the community.

-- JamesDempsey - 2023-11-07

Data Model Working Group

Grid & Web Services Working Group

Changes are minimal and controlled from the previous version. The only point raised about the reference implementation (there was one that could be modified easily, I think, to cope with the changes but it is quite old so I am not even sure if the code is recoverable) (reference, java library implemented by Kelly McCusker and Jonathan McDowell (maybe Jonathan could help on this)). This library is a full implementation able to read and write SpectrumDM 1.1 compatible files.

Maybe there is another implementation. Could you clarify it?

In any case, as the changes are fully controlled, we approve the update.

JesusSalgado - 2023-11-06

Registry Working Group

Semantics Working Group

* In section 3.3 Units, it is stated that the unit formatting standard is WCS/OGIP convention for unit. Is there any reason not to use the VOUnit formalism (see ongoing RFC)? It would be a pity if we can't converge here.

* In section 3.4 UCDs there seems to be a typo at the end of the first paragraph: [...] so you can’t put a literal em.* in a UTYPE field [...]. Do you mean UCD field ?

* In section 5.1 CoordSys Fields, the proposed list of terms should be submitted as a VEP to Semantics, in order to update the RefPosition vocabulary. And then refer to that vocabulary for acceptable values.

* In section 5.1.5 STC, the same remark concerning the RefFrame vocabulary

Changed:
<
<		* On pages 17-19, the table mentions "TimeFrameUCD", SpaceFrameUCD": I don't see what this refers to.
>
>		* On pages 17-19, the table mentions "TimeFrameUCD", SpaceFrameUCD ": I don't see what this refers to.
  BaptisteCecconi - 2023-11-09

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Operations Interest Group

Radio Astronomy Interest Group

Solar System Interest Group

Added:
>
>		Nothing to add.

-- Anne Raugh - 2023-11-10

 

Theory Interest Group

Time Domain Interest Group

Standards and Processes Committee

TCG Vote : Vote_start_date - Vote_end_date

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL *      
DM *      
GWS *      
Registry *      
Semantics *     Semantics WG comments should be addressed in next version
DCP        
Edu        
KDIG *      
Ops        
Radio        
Changed:
<
<
SSIG        
>
>
SSIG *      
 
Theory        
TD        
StdProc        

Deleted:
<
<

Revision 102023-11-10 - BaptisteCecconi

 
META TOPICPARENT name="IvoaDataModel"

Spectrum 1.2 Proposed Recommendation: Request for Comments

Introduction

The Spectrum Data Model describes the structure of spectrophotometric datasets with spectral and temporal coordinates and associated metadata.

This update satisfies an enhancement request by IPAC to add support for the following use cases:

  • 1-D spectra from Spitzer’s Infrared Spectrograph have multiple spectral orders
    • Spectral orders can overlap in wavelength
    • Plotting a Spitzer spectrum without accounting for orders gives you a mess
  • SEDs often include limits on measurements
    • these can be upper or lower limits
    • Plotters need to indicate limits clearly to avoid scientific misunderstanding
    • SDM1.1 has an ad-hoc representation for upper limits (only) which may be usable, but is conceptually incorrect.
The main changes in the v1.2 document are:
  • Section 4.1.1: addition of 'order' and 'relorder' attributes
  • Section 4.6.3: addition of 'UpperLimit' and 'LowerLimit' to explicitly express these concepts
  • Corrected UTypes in VOTable example #1
  • Updated diagrams to proper UML with improved consistency with the schema and text.
Latest version of SpectrumDM can be found at: The GitHub repository for issues and model development can be found at: The IVOA Twiki page for the project can be found at:

Reference Interoperable Implementations

The implementation level for this RFE is not as comprehensive as we would typically look for in a REC.

The DM working group consulted with the TCG prior to the start of this RFC and it was agreed that given

  • this is a minor revision based on a specific institution request
  • the request is targeted to serve a very specific purpose
  • the model is not VO-DML compliant, so the level of effort required to produce additional implementations, serializations, validators is out of proportion to the impact of the RFE.
the level of implementation for this enhancement is acceptable.

Data Providers:

Applications:
  • IPAC Firefly Archive toolkit:
    • "For some time, IPAC's Firefly archive GUI toolkit has taken advantage of IVOA Spectral Data Model (SDM) v1.1 to provide users with automatic plots of spectra that are the result of a query or have been uploaded to the tool. The plotting capability uses the SDM to identify which data to plot on the x and y axes and to change units. Firefly has recently been updated to recognize the new utypes specified in the proposed Spectral Data Model v1.2, namely:

      spec:Spectrum.Data.SpectralAxis.Order
      spec:Spectrum.Data.FluxAxis.Accuracy.LowerLimit
      spec.Spectrum.Data.FluxAxis.Accuracy.UpperLimit

      The toolkit uses these new utypes to plot orders (as found in e.g. Spitzer spectra) as separate traces, and to plot upper and lower limits as arrows (as found in e.g. SEDs). These changes are currently in test and will be released in May. Upon release, these features will be available in IRSA, Rubin, and NED interfaces, all of which are built upon the open-source Firefly toolkit.

      In a future release, Firefly will recognize the spec:Spectrum.Data.SpectralAxis.RelOrder utype proposed in SDMv1.2. When Firefly encounters a spectrum with only RelOrder and not Order, it will automatically plot each unique RelOrder as a different trace, analogous to how it treats Order. If a spectrum contains both Order and RelOrder, Firefly will automatically plot traces based on Order but will provide the user a toggle to plot RelOrder."

Implementations Validators

There is no dedicated validator for Spectrum data model instances to update for this enhancement.

Since this is not a VO-DML compliant model, the mechanisms for

  • validating the data model itself
  • generating and validating example serializations in VOTable, XML, etc.
  • generating model based code
are not available.

Comments from the IVOA Community during RFC/TCG review period: 2023-09-11 - 2023-10-23

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Comment by JesusSalgado

This is a preliminary comment: Version 1.0 had a java library implemented by Kelly McCusker (not in IVOA anymore) and Jonathan McDowell that allowed parsing and writing of spectral data compatible with 1.0. It would be great to obtain a version of this library and adapt it for these minor changes.

This library was used by some tools (e.g. VOSpec)

JesusSalgado - 2023-09-22

Comments from TCG member during the RFC/TCG Review Period: 2023-09-11 - 2023-10-23

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

Good to see support of orders as this has been raised by the community.

-- JamesDempsey - 2023-11-07

Data Model Working Group

Grid & Web Services Working Group

Changes are minimal and controlled from the previous version. The only point raised about the reference implementation (there was one that could be modified easily, I think, to cope with the changes but it is quite old so I am not even sure if the code is recoverable) (reference, java library implemented by Kelly McCusker and Jonathan McDowell (maybe Jonathan could help on this)). This library is a full implementation able to read and write SpectrumDM 1.1 compatible files.

Maybe there is another implementation. Could you clarify it?

In any case, as the changes are fully controlled, we approve the update.

JesusSalgado - 2023-11-06

Registry Working Group

Semantics Working Group

* In section 3.3 Units, it is stated that the unit formatting standard is WCS/OGIP convention for unit. Is there any reason not to use the VOUnit formalism (see ongoing RFC)? It would be a pity if we can't converge here.

* In section 3.4 UCDs there seems to be a typo at the end of the first paragraph: [...] so you can’t put a literal em.* in a UTYPE field [...]. Do you mean UCD field ?

* In section 5.1 CoordSys Fields, the proposed list of terms should be submitted as a VEP to Semantics, in order to update the RefPosition vocabulary. And then refer to that vocabulary for acceptable values.

* In section 5.1.5 STC, the same remark concerning the RefFrame vocabulary

Added:
>
>		* On pages 17-19, the table mentions "TimeFrameUCD", SpaceFrameUCD": I don't see what this refers to.
 BaptisteCecconi - 2023-11-09

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Operations Interest Group

Radio Astronomy Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Standards and Processes Committee

TCG Vote : Vote_start_date - Vote_end_date

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL *      
DM *      
GWS *      
Registry *      
Changed:
<
<
Semantics        
>
>
Semantics *     Semantics WG comments should be addressed in next version
 
DCP        
Edu        
KDIG *      
Ops        
Radio        
SSIG        
Theory        
TD        
StdProc        

Added:
>
>

Revision 92023-11-10 - RenaudSavalle

 
META TOPICPARENT name="IvoaDataModel"

Spectrum 1.2 Proposed Recommendation: Request for Comments

Introduction

The Spectrum Data Model describes the structure of spectrophotometric datasets with spectral and temporal coordinates and associated metadata.

This update satisfies an enhancement request by IPAC to add support for the following use cases:

  • 1-D spectra from Spitzer’s Infrared Spectrograph have multiple spectral orders
    • Spectral orders can overlap in wavelength
    • Plotting a Spitzer spectrum without accounting for orders gives you a mess
  • SEDs often include limits on measurements
    • these can be upper or lower limits
    • Plotters need to indicate limits clearly to avoid scientific misunderstanding
    • SDM1.1 has an ad-hoc representation for upper limits (only) which may be usable, but is conceptually incorrect.
The main changes in the v1.2 document are:
  • Section 4.1.1: addition of 'order' and 'relorder' attributes
  • Section 4.6.3: addition of 'UpperLimit' and 'LowerLimit' to explicitly express these concepts
  • Corrected UTypes in VOTable example #1
  • Updated diagrams to proper UML with improved consistency with the schema and text.
Latest version of SpectrumDM can be found at: The GitHub repository for issues and model development can be found at: The IVOA Twiki page for the project can be found at:

Reference Interoperable Implementations

The implementation level for this RFE is not as comprehensive as we would typically look for in a REC.

The DM working group consulted with the TCG prior to the start of this RFC and it was agreed that given

  • this is a minor revision based on a specific institution request
  • the request is targeted to serve a very specific purpose
  • the model is not VO-DML compliant, so the level of effort required to produce additional implementations, serializations, validators is out of proportion to the impact of the RFE.
the level of implementation for this enhancement is acceptable.

Data Providers:

Applications:
  • IPAC Firefly Archive toolkit:
    • "For some time, IPAC's Firefly archive GUI toolkit has taken advantage of IVOA Spectral Data Model (SDM) v1.1 to provide users with automatic plots of spectra that are the result of a query or have been uploaded to the tool. The plotting capability uses the SDM to identify which data to plot on the x and y axes and to change units. Firefly has recently been updated to recognize the new utypes specified in the proposed Spectral Data Model v1.2, namely:

      spec:Spectrum.Data.SpectralAxis.Order
      spec:Spectrum.Data.FluxAxis.Accuracy.LowerLimit
      spec.Spectrum.Data.FluxAxis.Accuracy.UpperLimit

      The toolkit uses these new utypes to plot orders (as found in e.g. Spitzer spectra) as separate traces, and to plot upper and lower limits as arrows (as found in e.g. SEDs). These changes are currently in test and will be released in May. Upon release, these features will be available in IRSA, Rubin, and NED interfaces, all of which are built upon the open-source Firefly toolkit.

      In a future release, Firefly will recognize the spec:Spectrum.Data.SpectralAxis.RelOrder utype proposed in SDMv1.2. When Firefly encounters a spectrum with only RelOrder and not Order, it will automatically plot each unique RelOrder as a different trace, analogous to how it treats Order. If a spectrum contains both Order and RelOrder, Firefly will automatically plot traces based on Order but will provide the user a toggle to plot RelOrder."

Implementations Validators

There is no dedicated validator for Spectrum data model instances to update for this enhancement.

Since this is not a VO-DML compliant model, the mechanisms for

  • validating the data model itself
  • generating and validating example serializations in VOTable, XML, etc.
  • generating model based code
are not available.

Comments from the IVOA Community during RFC/TCG review period: 2023-09-11 - 2023-10-23

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Comment by JesusSalgado

This is a preliminary comment: Version 1.0 had a java library implemented by Kelly McCusker (not in IVOA anymore) and Jonathan McDowell that allowed parsing and writing of spectral data compatible with 1.0. It would be great to obtain a version of this library and adapt it for these minor changes.

This library was used by some tools (e.g. VOSpec)

JesusSalgado - 2023-09-22

Comments from TCG member during the RFC/TCG Review Period: 2023-09-11 - 2023-10-23

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

Good to see support of orders as this has been raised by the community.

-- JamesDempsey - 2023-11-07

Data Model Working Group

Grid & Web Services Working Group

Changes are minimal and controlled from the previous version. The only point raised about the reference implementation (there was one that could be modified easily, I think, to cope with the changes but it is quite old so I am not even sure if the code is recoverable) (reference, java library implemented by Kelly McCusker and Jonathan McDowell (maybe Jonathan could help on this)). This library is a full implementation able to read and write SpectrumDM 1.1 compatible files.

Maybe there is another implementation. Could you clarify it?

In any case, as the changes are fully controlled, we approve the update.

JesusSalgado - 2023-11-06

Registry Working Group

Semantics Working Group

Changed:
<
<		* In section 3.3 Units, it is stated that the unit formatting standard is WCS/OGIP convention for unit. Is there any reason not to use the VOUnit formalism (see ongoing RFC)? It would be a pity if we can't converge here.
>
>		* In section 3.3 Units, it is stated that the unit formatting standard is WCS/OGIP convention for unit. Is there any reason not to use the VOUnit formalism (see ongoing RFC)? It would be a pity if we can't converge here.
 
Changed:
<
<		* In section 3.4 UCDs there seems to be a typo at the end of the first paragraph: [...] so you can’t put a literal em.* in a UTYPE field [...]. Do you mean UCD field ?
>
>		* In section 3.4 UCDs there seems to be a typo at the end of the first paragraph: [...] so you can’t put a literal em.* in a UTYPE field [...]. Do you mean UCD field ?
  * In section 5.1 CoordSys Fields, the proposed list of terms should be submitted as a VEP to Semantics, in order to update the RefPosition vocabulary. And then refer to that vocabulary for acceptable values.

* In section 5.1.5 STC, the same remark concerning the RefFrame vocabulary

BaptisteCecconi - 2023-11-09

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Operations Interest Group

Radio Astronomy Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Standards and Processes Committee

TCG Vote : Vote_start_date - Vote_end_date

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL *      
DM *      
GWS *      
Changed:
<
<
Registry        
>
>
Registry *      
 
Semantics        
DCP        
Edu        
KDIG *      
Ops        
Radio        
SSIG        
Theory        
TD        
StdProc        

Deleted:
<
<

Revision 82023-11-09 - BaptisteCecconi

 
META TOPICPARENT name="IvoaDataModel"

Spectrum 1.2 Proposed Recommendation: Request for Comments

Introduction

The Spectrum Data Model describes the structure of spectrophotometric datasets with spectral and temporal coordinates and associated metadata.

This update satisfies an enhancement request by IPAC to add support for the following use cases:

  • 1-D spectra from Spitzer’s Infrared Spectrograph have multiple spectral orders
    • Spectral orders can overlap in wavelength
    • Plotting a Spitzer spectrum without accounting for orders gives you a mess
  • SEDs often include limits on measurements
    • these can be upper or lower limits
    • Plotters need to indicate limits clearly to avoid scientific misunderstanding
    • SDM1.1 has an ad-hoc representation for upper limits (only) which may be usable, but is conceptually incorrect.
The main changes in the v1.2 document are:
  • Section 4.1.1: addition of 'order' and 'relorder' attributes
  • Section 4.6.3: addition of 'UpperLimit' and 'LowerLimit' to explicitly express these concepts
  • Corrected UTypes in VOTable example #1
  • Updated diagrams to proper UML with improved consistency with the schema and text.
Latest version of SpectrumDM can be found at: The GitHub repository for issues and model development can be found at: The IVOA Twiki page for the project can be found at:

Reference Interoperable Implementations

The implementation level for this RFE is not as comprehensive as we would typically look for in a REC.

The DM working group consulted with the TCG prior to the start of this RFC and it was agreed that given

  • this is a minor revision based on a specific institution request
  • the request is targeted to serve a very specific purpose
  • the model is not VO-DML compliant, so the level of effort required to produce additional implementations, serializations, validators is out of proportion to the impact of the RFE.
the level of implementation for this enhancement is acceptable.

Data Providers:

Applications:
  • IPAC Firefly Archive toolkit:
    • "For some time, IPAC's Firefly archive GUI toolkit has taken advantage of IVOA Spectral Data Model (SDM) v1.1 to provide users with automatic plots of spectra that are the result of a query or have been uploaded to the tool. The plotting capability uses the SDM to identify which data to plot on the x and y axes and to change units. Firefly has recently been updated to recognize the new utypes specified in the proposed Spectral Data Model v1.2, namely:

      spec:Spectrum.Data.SpectralAxis.Order
      spec:Spectrum.Data.FluxAxis.Accuracy.LowerLimit
      spec.Spectrum.Data.FluxAxis.Accuracy.UpperLimit

      The toolkit uses these new utypes to plot orders (as found in e.g. Spitzer spectra) as separate traces, and to plot upper and lower limits as arrows (as found in e.g. SEDs). These changes are currently in test and will be released in May. Upon release, these features will be available in IRSA, Rubin, and NED interfaces, all of which are built upon the open-source Firefly toolkit.

      In a future release, Firefly will recognize the spec:Spectrum.Data.SpectralAxis.RelOrder utype proposed in SDMv1.2. When Firefly encounters a spectrum with only RelOrder and not Order, it will automatically plot each unique RelOrder as a different trace, analogous to how it treats Order. If a spectrum contains both Order and RelOrder, Firefly will automatically plot traces based on Order but will provide the user a toggle to plot RelOrder."

Implementations Validators

There is no dedicated validator for Spectrum data model instances to update for this enhancement.

Since this is not a VO-DML compliant model, the mechanisms for

  • validating the data model itself
  • generating and validating example serializations in VOTable, XML, etc.
  • generating model based code
are not available.

Comments from the IVOA Community during RFC/TCG review period: 2023-09-11 - 2023-10-23

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Comment by JesusSalgado

This is a preliminary comment: Version 1.0 had a java library implemented by Kelly McCusker (not in IVOA anymore) and Jonathan McDowell that allowed parsing and writing of spectral data compatible with 1.0. It would be great to obtain a version of this library and adapt it for these minor changes.

This library was used by some tools (e.g. VOSpec)

JesusSalgado - 2023-09-22

Comments from TCG member during the RFC/TCG Review Period: 2023-09-11 - 2023-10-23

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

Good to see support of orders as this has been raised by the community.

-- JamesDempsey - 2023-11-07

Data Model Working Group

Grid & Web Services Working Group

Changes are minimal and controlled from the previous version. The only point raised about the reference implementation (there was one that could be modified easily, I think, to cope with the changes but it is quite old so I am not even sure if the code is recoverable) (reference, java library implemented by Kelly McCusker and Jonathan McDowell (maybe Jonathan could help on this)). This library is a full implementation able to read and write SpectrumDM 1.1 compatible files.

Maybe there is another implementation. Could you clarify it?

In any case, as the changes are fully controlled, we approve the update.

JesusSalgado - 2023-11-06

Registry Working Group

Semantics Working Group

Added:
>
>		* In section 3.3 Units, it is stated that the unit formatting standard is WCS/OGIP convention for unit. Is there any reason not to use the VOUnit formalism (see ongoing RFC)? It would be a pity if we can't converge here.

* In section 3.4 UCDs there seems to be a typo at the end of the first paragraph: [...] so you can’t put a literal em.* in a UTYPE field [...]. Do you mean UCD field ?

* In section 5.1 CoordSys Fields, the proposed list of terms should be submitted as a VEP to Semantics, in order to update the RefPosition vocabulary. And then refer to that vocabulary for acceptable values.

* In section 5.1.5 STC, the same remark concerning the RefFrame vocabulary

BaptisteCecconi - 2023-11-09

 

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Operations Interest Group

Radio Astronomy Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Standards and Processes Committee

TCG Vote : Vote_start_date - Vote_end_date

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL *      
DM *      
GWS *      
Registry        
Semantics        
DCP        
Edu        
KDIG *      
Ops        
Radio        
SSIG        
Theory        
TD        
StdProc        

Added:
>
>

Revision 72023-11-08 - RaffaeleDAbrusco

 
META TOPICPARENT name="IvoaDataModel"

Spectrum 1.2 Proposed Recommendation: Request for Comments

Introduction

The Spectrum Data Model describes the structure of spectrophotometric datasets with spectral and temporal coordinates and associated metadata.

This update satisfies an enhancement request by IPAC to add support for the following use cases:

  • 1-D spectra from Spitzer’s Infrared Spectrograph have multiple spectral orders
    • Spectral orders can overlap in wavelength
    • Plotting a Spitzer spectrum without accounting for orders gives you a mess
  • SEDs often include limits on measurements
    • these can be upper or lower limits
    • Plotters need to indicate limits clearly to avoid scientific misunderstanding
    • SDM1.1 has an ad-hoc representation for upper limits (only) which may be usable, but is conceptually incorrect.
The main changes in the v1.2 document are:
  • Section 4.1.1: addition of 'order' and 'relorder' attributes
  • Section 4.6.3: addition of 'UpperLimit' and 'LowerLimit' to explicitly express these concepts
  • Corrected UTypes in VOTable example #1
  • Updated diagrams to proper UML with improved consistency with the schema and text.
Latest version of SpectrumDM can be found at: The GitHub repository for issues and model development can be found at: The IVOA Twiki page for the project can be found at:

Reference Interoperable Implementations

The implementation level for this RFE is not as comprehensive as we would typically look for in a REC.

The DM working group consulted with the TCG prior to the start of this RFC and it was agreed that given

  • this is a minor revision based on a specific institution request
  • the request is targeted to serve a very specific purpose
  • the model is not VO-DML compliant, so the level of effort required to produce additional implementations, serializations, validators is out of proportion to the impact of the RFE.
the level of implementation for this enhancement is acceptable.

Data Providers:

Applications:
  • IPAC Firefly Archive toolkit:
    • "For some time, IPAC's Firefly archive GUI toolkit has taken advantage of IVOA Spectral Data Model (SDM) v1.1 to provide users with automatic plots of spectra that are the result of a query or have been uploaded to the tool. The plotting capability uses the SDM to identify which data to plot on the x and y axes and to change units. Firefly has recently been updated to recognize the new utypes specified in the proposed Spectral Data Model v1.2, namely:

      spec:Spectrum.Data.SpectralAxis.Order
      spec:Spectrum.Data.FluxAxis.Accuracy.LowerLimit
      spec.Spectrum.Data.FluxAxis.Accuracy.UpperLimit

      The toolkit uses these new utypes to plot orders (as found in e.g. Spitzer spectra) as separate traces, and to plot upper and lower limits as arrows (as found in e.g. SEDs). These changes are currently in test and will be released in May. Upon release, these features will be available in IRSA, Rubin, and NED interfaces, all of which are built upon the open-source Firefly toolkit.

      In a future release, Firefly will recognize the spec:Spectrum.Data.SpectralAxis.RelOrder utype proposed in SDMv1.2. When Firefly encounters a spectrum with only RelOrder and not Order, it will automatically plot each unique RelOrder as a different trace, analogous to how it treats Order. If a spectrum contains both Order and RelOrder, Firefly will automatically plot traces based on Order but will provide the user a toggle to plot RelOrder."

Implementations Validators

There is no dedicated validator for Spectrum data model instances to update for this enhancement.

Since this is not a VO-DML compliant model, the mechanisms for

  • validating the data model itself
  • generating and validating example serializations in VOTable, XML, etc.
  • generating model based code
are not available.

Comments from the IVOA Community during RFC/TCG review period: 2023-09-11 - 2023-10-23

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Comment by JesusSalgado

This is a preliminary comment: Version 1.0 had a java library implemented by Kelly McCusker (not in IVOA anymore) and Jonathan McDowell that allowed parsing and writing of spectral data compatible with 1.0. It would be great to obtain a version of this library and adapt it for these minor changes.

This library was used by some tools (e.g. VOSpec)

JesusSalgado - 2023-09-22

Comments from TCG member during the RFC/TCG Review Period: 2023-09-11 - 2023-10-23

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

Good to see support of orders as this has been raised by the community.

-- JamesDempsey - 2023-11-07

Data Model Working Group

Grid & Web Services Working Group

Changes are minimal and controlled from the previous version. The only point raised about the reference implementation (there was one that could be modified easily, I think, to cope with the changes but it is quite old so I am not even sure if the code is recoverable) (reference, java library implemented by Kelly McCusker and Jonathan McDowell (maybe Jonathan could help on this)). This library is a full implementation able to read and write SpectrumDM 1.1 compatible files.

Maybe there is another implementation. Could you clarify it?

In any case, as the changes are fully controlled, we approve the update.

JesusSalgado - 2023-11-06

Registry Working Group

Semantics Working Group

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Operations Interest Group

Radio Astronomy Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Standards and Processes Committee

TCG Vote : Vote_start_date - Vote_end_date

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL *      
DM *      
GWS *      
Registry        
Semantics        
DCP        
Edu        
Changed:
<
<
KDIG        
>
>
KDIG *      
 
Ops        
Radio        
SSIG        
Theory        
TD        
StdProc        

Revision 62023-11-07 - JamesDempsey

 
META TOPICPARENT name="IvoaDataModel"

Spectrum 1.2 Proposed Recommendation: Request for Comments

Introduction

The Spectrum Data Model describes the structure of spectrophotometric datasets with spectral and temporal coordinates and associated metadata.

This update satisfies an enhancement request by IPAC to add support for the following use cases:

  • 1-D spectra from Spitzer’s Infrared Spectrograph have multiple spectral orders
    • Spectral orders can overlap in wavelength
    • Plotting a Spitzer spectrum without accounting for orders gives you a mess
  • SEDs often include limits on measurements
    • these can be upper or lower limits
    • Plotters need to indicate limits clearly to avoid scientific misunderstanding
    • SDM1.1 has an ad-hoc representation for upper limits (only) which may be usable, but is conceptually incorrect.
The main changes in the v1.2 document are:
  • Section 4.1.1: addition of 'order' and 'relorder' attributes
  • Section 4.6.3: addition of 'UpperLimit' and 'LowerLimit' to explicitly express these concepts
  • Corrected UTypes in VOTable example #1
  • Updated diagrams to proper UML with improved consistency with the schema and text.
Latest version of SpectrumDM can be found at: The GitHub repository for issues and model development can be found at: The IVOA Twiki page for the project can be found at:

Reference Interoperable Implementations

The implementation level for this RFE is not as comprehensive as we would typically look for in a REC.

The DM working group consulted with the TCG prior to the start of this RFC and it was agreed that given

  • this is a minor revision based on a specific institution request
  • the request is targeted to serve a very specific purpose
  • the model is not VO-DML compliant, so the level of effort required to produce additional implementations, serializations, validators is out of proportion to the impact of the RFE.
the level of implementation for this enhancement is acceptable.

Data Providers:

Applications:
  • IPAC Firefly Archive toolkit:
    • "For some time, IPAC's Firefly archive GUI toolkit has taken advantage of IVOA Spectral Data Model (SDM) v1.1 to provide users with automatic plots of spectra that are the result of a query or have been uploaded to the tool. The plotting capability uses the SDM to identify which data to plot on the x and y axes and to change units. Firefly has recently been updated to recognize the new utypes specified in the proposed Spectral Data Model v1.2, namely:

      spec:Spectrum.Data.SpectralAxis.Order
      spec:Spectrum.Data.FluxAxis.Accuracy.LowerLimit
      spec.Spectrum.Data.FluxAxis.Accuracy.UpperLimit

      The toolkit uses these new utypes to plot orders (as found in e.g. Spitzer spectra) as separate traces, and to plot upper and lower limits as arrows (as found in e.g. SEDs). These changes are currently in test and will be released in May. Upon release, these features will be available in IRSA, Rubin, and NED interfaces, all of which are built upon the open-source Firefly toolkit.

      In a future release, Firefly will recognize the spec:Spectrum.Data.SpectralAxis.RelOrder utype proposed in SDMv1.2. When Firefly encounters a spectrum with only RelOrder and not Order, it will automatically plot each unique RelOrder as a different trace, analogous to how it treats Order. If a spectrum contains both Order and RelOrder, Firefly will automatically plot traces based on Order but will provide the user a toggle to plot RelOrder."

Implementations Validators

There is no dedicated validator for Spectrum data model instances to update for this enhancement.

Since this is not a VO-DML compliant model, the mechanisms for

  • validating the data model itself
  • generating and validating example serializations in VOTable, XML, etc.
  • generating model based code
are not available.

Comments from the IVOA Community during RFC/TCG review period: 2023-09-11 - 2023-10-23

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Comment by JesusSalgado

This is a preliminary comment: Version 1.0 had a java library implemented by Kelly McCusker (not in IVOA anymore) and Jonathan McDowell that allowed parsing and writing of spectral data compatible with 1.0. It would be great to obtain a version of this library and adapt it for these minor changes.

This library was used by some tools (e.g. VOSpec)

JesusSalgado - 2023-09-22

Comments from TCG member during the RFC/TCG Review Period: 2023-09-11 - 2023-10-23

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

Added:
>
>		Good to see support of orders as this has been raised by the community.

-- JamesDempsey - 2023-11-07

 

Data Model Working Group

Grid & Web Services Working Group

Changes are minimal and controlled from the previous version. The only point raised about the reference implementation (there was one that could be modified easily, I think, to cope with the changes but it is quite old so I am not even sure if the code is recoverable) (reference, java library implemented by Kelly McCusker and Jonathan McDowell (maybe Jonathan could help on this)). This library is a full implementation able to read and write SpectrumDM 1.1 compatible files.

Maybe there is another implementation. Could you clarify it?

In any case, as the changes are fully controlled, we approve the update.

JesusSalgado - 2023-11-06

Registry Working Group

Semantics Working Group

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Operations Interest Group

Radio Astronomy Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Standards and Processes Committee

TCG Vote : Vote_start_date - Vote_end_date

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
Changed:
<
<
DAL        
>
>
DAL *      
 
DM *      
GWS *      
Registry        
Semantics        
DCP        
Edu        
KDIG        
Ops        
Radio        
SSIG        
Theory        
TD        
StdProc        

Deleted:
<
<

Revision 52023-11-07 - JesusSalgado

 
META TOPICPARENT name="IvoaDataModel"

Spectrum 1.2 Proposed Recommendation: Request for Comments

Introduction

The Spectrum Data Model describes the structure of spectrophotometric datasets with spectral and temporal coordinates and associated metadata.

This update satisfies an enhancement request by IPAC to add support for the following use cases:

  • 1-D spectra from Spitzer’s Infrared Spectrograph have multiple spectral orders
    • Spectral orders can overlap in wavelength
    • Plotting a Spitzer spectrum without accounting for orders gives you a mess
  • SEDs often include limits on measurements
    • these can be upper or lower limits
    • Plotters need to indicate limits clearly to avoid scientific misunderstanding
    • SDM1.1 has an ad-hoc representation for upper limits (only) which may be usable, but is conceptually incorrect.
The main changes in the v1.2 document are:
  • Section 4.1.1: addition of 'order' and 'relorder' attributes
  • Section 4.6.3: addition of 'UpperLimit' and 'LowerLimit' to explicitly express these concepts
  • Corrected UTypes in VOTable example #1
  • Updated diagrams to proper UML with improved consistency with the schema and text.
Latest version of SpectrumDM can be found at: The GitHub repository for issues and model development can be found at: The IVOA Twiki page for the project can be found at:

Reference Interoperable Implementations

The implementation level for this RFE is not as comprehensive as we would typically look for in a REC.

The DM working group consulted with the TCG prior to the start of this RFC and it was agreed that given

  • this is a minor revision based on a specific institution request
  • the request is targeted to serve a very specific purpose
  • the model is not VO-DML compliant, so the level of effort required to produce additional implementations, serializations, validators is out of proportion to the impact of the RFE.
the level of implementation for this enhancement is acceptable.

Data Providers:

Applications:
  • IPAC Firefly Archive toolkit:
    • "For some time, IPAC's Firefly archive GUI toolkit has taken advantage of IVOA Spectral Data Model (SDM) v1.1 to provide users with automatic plots of spectra that are the result of a query or have been uploaded to the tool. The plotting capability uses the SDM to identify which data to plot on the x and y axes and to change units. Firefly has recently been updated to recognize the new utypes specified in the proposed Spectral Data Model v1.2, namely:

      spec:Spectrum.Data.SpectralAxis.Order
      spec:Spectrum.Data.FluxAxis.Accuracy.LowerLimit
      spec.Spectrum.Data.FluxAxis.Accuracy.UpperLimit

      The toolkit uses these new utypes to plot orders (as found in e.g. Spitzer spectra) as separate traces, and to plot upper and lower limits as arrows (as found in e.g. SEDs). These changes are currently in test and will be released in May. Upon release, these features will be available in IRSA, Rubin, and NED interfaces, all of which are built upon the open-source Firefly toolkit.

      In a future release, Firefly will recognize the spec:Spectrum.Data.SpectralAxis.RelOrder utype proposed in SDMv1.2. When Firefly encounters a spectrum with only RelOrder and not Order, it will automatically plot each unique RelOrder as a different trace, analogous to how it treats Order. If a spectrum contains both Order and RelOrder, Firefly will automatically plot traces based on Order but will provide the user a toggle to plot RelOrder."

Implementations Validators

There is no dedicated validator for Spectrum data model instances to update for this enhancement.

Since this is not a VO-DML compliant model, the mechanisms for

  • validating the data model itself
  • generating and validating example serializations in VOTable, XML, etc.
  • generating model based code
are not available.

Comments from the IVOA Community during RFC/TCG review period: 2023-09-11 - 2023-10-23

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Comment by JesusSalgado

This is a preliminary comment: Version 1.0 had a java library implemented by Kelly McCusker (not in IVOA anymore) and Jonathan McDowell that allowed parsing and writing of spectral data compatible with 1.0. It would be great to obtain a version of this library and adapt it for these minor changes.

This library was used by some tools (e.g. VOSpec)

JesusSalgado - 2023-09-22

Comments from TCG member during the RFC/TCG Review Period: 2023-09-11 - 2023-10-23

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

Data Model Working Group

Grid & Web Services Working Group

Added:
>
>		Changes are minimal and controlled from the previous version. The only point raised about the reference implementation (there was one that could be modified easily, I think, to cope with the changes but it is quite old so I am not even sure if the code is recoverable) (reference, java library implemented by Kelly McCusker and Jonathan McDowell (maybe Jonathan could help on this)). This library is a full implementation able to read and write SpectrumDM 1.1 compatible files.

Maybe there is another implementation. Could you clarify it?

In any case, as the changes are fully controlled, we approve the update.

JesusSalgado - 2023-11-06

 

Registry Working Group

Semantics Working Group

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Operations Interest Group

Radio Astronomy Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Standards and Processes Committee

TCG Vote : Vote_start_date - Vote_end_date

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Changed:
<
<
Group Yes No Abstain Comments
TCG        
Apps        
DAL        
DM *      
GWS        
Registry        
Semantics        
DCP        
Edu        
KDIG        
Ops        
Radio        
SSIG        
Theory        
TD        
StdProc        
>
>
Group Yes No Abstain Comments
TCG        
Apps        
DAL        
DM *      
GWS *      
Registry        
Semantics        
DCP        
Edu        
KDIG        
Ops        
Radio        
SSIG        
Theory        
TD        
StdProc        
 

Revision 42023-11-02 - MarkCresitelloDittmar

 
META TOPICPARENT name="IvoaDataModel"

Spectrum 1.2 Proposed Recommendation: Request for Comments

Changed:
<
<
>
>
 

Introduction

The Spectrum Data Model describes the structure of spectrophotometric datasets with spectral and temporal coordinates and associated metadata.

This update satisfies an enhancement request by IPAC to add support for the following use cases:

  • 1-D spectra from Spitzer’s Infrared Spectrograph have multiple spectral orders
    • Spectral orders can overlap in wavelength
    • Plotting a Spitzer spectrum without accounting for orders gives you a mess
  • SEDs often include limits on measurements
    • these can be upper or lower limits
    • Plotters need to indicate limits clearly to avoid scientific misunderstanding
    • SDM1.1 has an ad-hoc representation for upper limits (only) which may be usable, but is conceptually incorrect.
The main changes in the v1.2 document are:
  • Section 4.1.1: addition of 'order' and 'relorder' attributes
  • Section 4.6.3: addition of 'UpperLimit' and 'LowerLimit' to explicitly express these concepts
  • Corrected UTypes in VOTable example #1
  • Updated diagrams to proper UML with improved consistency with the schema and text.
Latest version of SpectrumDM can be found at: The GitHub repository for issues and model development can be found at: The IVOA Twiki page for the project can be found at:
Changed:
<
<

<a name="Reference Interoperable Implemen"></a>Reference Interoperable Implementations

>
>

Reference Interoperable Implementations

  The implementation level for this RFE is not as comprehensive as we would typically look for in a REC.

The DM working group consulted with the TCG prior to the start of this RFC and it was agreed that given

  • this is a minor revision based on a specific institution request
  • the request is targeted to serve a very specific purpose
  • the model is not VO-DML compliant, so the level of effort required to produce additional implementations, serializations, validators is out of proportion to the impact of the RFE.
the level of implementation for this enhancement is acceptable.

Data Providers:

Applications:
  • IPAC Firefly Archive toolkit:
    • "For some time, IPAC's Firefly archive GUI toolkit has taken advantage of IVOA Spectral Data Model (SDM) v1.1 to provide users with automatic plots of spectra that are the result of a query or have been uploaded to the tool. The plotting capability uses the SDM to identify which data to plot on the x and y axes and to change units. Firefly has recently been updated to recognize the new utypes specified in the proposed Spectral Data Model v1.2, namely:

      spec:Spectrum.Data.SpectralAxis.Order
      spec:Spectrum.Data.FluxAxis.Accuracy.LowerLimit
      spec.Spectrum.Data.FluxAxis.Accuracy.UpperLimit

      The toolkit uses these new utypes to plot orders (as found in e.g. Spitzer spectra) as separate traces, and to plot upper and lower limits as arrows (as found in e.g. SEDs). These changes are currently in test and will be released in May. Upon release, these features will be available in IRSA, Rubin, and NED interfaces, all of which are built upon the open-source Firefly toolkit.

      In a future release, Firefly will recognize the spec:Spectrum.Data.SpectralAxis.RelOrder utype proposed in SDMv1.2. When Firefly encounters a spectrum with only RelOrder and not Order, it will automatically plot each unique RelOrder as a different trace, analogous to how it treats Order. If a spectrum contains both Order and RelOrder, Firefly will automatically plot traces based on Order but will provide the user a toggle to plot RelOrder."
Changed:
<
<

<a name="Implementations Validators"></a>Implementations Validators

>
>

Implementations Validators

  There is no dedicated validator for Spectrum data model instances to update for this enhancement.

Since this is not a VO-DML compliant model, the mechanisms for

  • validating the data model itself
  • generating and validating example serializations in VOTable, XML, etc.
  • generating model based code
are not available.

Changed:
<
<

<a name="Comments from the IVOA Community"></a>Comments from the IVOA Community during RFC/TCG review period: 2023-09-11 - 2023-10-23

>
>

Comments from the IVOA Community during RFC/TCG review period: 2023-09-11 - 2023-10-23

  The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Comment by JesusSalgado

This is a preliminary comment: Version 1.0 had a java library implemented by Kelly McCusker (not in IVOA anymore) and Jonathan McDowell that allowed parsing and writing of spectral data compatible with 1.0. It would be great to obtain a version of this library and adapt it for these minor changes.

This library was used by some tools (e.g. VOSpec)

JesusSalgado - 2023-09-22

Changed:
<
<

<a name="Comments from TCG member during"></a>Comments from TCG member during the RFC/TCG Review Period: 2023-09-11 - 2023-10-23

>
>

Comments from TCG member during the RFC/TCG Review Period: 2023-09-11 - 2023-10-23

  WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

Changed:
<
<

<a name="TCG Chair Vice Chair"></a>TCG Chair & Vice Chair

>
>

TCG Chair & Vice Chair

 
Changed:
<
<

<a name="Applications Working Group"></a> Applications Working Group

>
>

Applications Working Group

 
Changed:
<
<

<a name="Data Access Layer Working Group"></a> Data Access Layer Working Group

>
>

Data Access Layer Working Group

 
Changed:
<
<

<a name="Data Model Working Group"></a> Data Model Working Group

>
>

Data Model Working Group

 
Changed:
<
<

<a name="Grid Web Services Working Group"></a> Grid & Web Services Working Group

>
>

Grid & Web Services Working Group

 
Changed:
<
<

<a name="Registry Working Group"></a> Registry Working Group

>
>

Registry Working Group

 
Changed:
<
<

<a name="Semantics Working Group"></a> Semantics Working Group

>
>

Semantics Working Group

 
Changed:
<
<

<a name="Data Curation Preservation Inte"></a> Data Curation & Preservation Interest Group

>
>

Data Curation & Preservation Interest Group

 
Changed:
<
<

<a name="Education Interest Group"></a> Education Interest Group

>
>

Education Interest Group

 
Changed:
<
<

<a name="Knowledge Discovery Interest Gro"></a> Knowledge Discovery Interest Group

>
>

Knowledge Discovery Interest Group

 
Changed:
<
<

<a name="Operations Interest Group"></a> Operations Interest Group

>
>

Operations Interest Group

 
Changed:
<
<

<a name="Radio Astronomy Interest Group"></a> Radio Astronomy Interest Group

>
>

Radio Astronomy Interest Group

 
Changed:
<
<

<a name="Solar System Interest Group"></a> Solar System Interest Group

>
>

Solar System Interest Group

 
Changed:
<
<

<a name="Theory Interest Group"></a> Theory Interest Group

>
>

Theory Interest Group

 
Changed:
<
<

<a name="Time Domain Interest Group"></a> Time Domain Interest Group

>
>

Time Domain Interest Group

 
Changed:
<
<

<a name="Standards and Processes Committe"></a> Standards and Processes Committee

>
>

Standards and Processes Committee

 
Changed:
<
<

<a name="TCG Vote : Vote_start_date - Vot"></a>TCG Vote : Vote_start_date - Vote_end_date

>
>

TCG Vote : Vote_start_date - Vote_end_date

  If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.
Changed:
<
<
Group Yes No Abstain Comments
TCG
Apps
DAL
DM
GWS
Registry
Semantics
DCP
Edu
KDIG
Ops
Radio
SSIG
Theory
TD
StdProc
>
>
Group Yes No Abstain Comments
Added:
>
>
TCG        
Apps        
DAL        
DM *      
GWS        
Registry        
Semantics        
DCP        
Edu        
KDIG        
Ops        
Radio        
SSIG        
Theory        
TD        
StdProc        
 

Revision 32023-09-22 - JesusSalgado

 
META TOPICPARENT name="IvoaDataModel"

Spectrum 1.2 Proposed Recommendation: Request for Comments


Introduction

The Spectrum Data Model describes the structure of spectrophotometric datasets with spectral and temporal coordinates and associated metadata.

This update satisfies an enhancement request by IPAC to add support for the following use cases:

  • 1-D spectra from Spitzer’s Infrared Spectrograph have multiple spectral orders
    • Spectral orders can overlap in wavelength
    • Plotting a Spitzer spectrum without accounting for orders gives you a mess
  • SEDs often include limits on measurements
    • these can be upper or lower limits
    • Plotters need to indicate limits clearly to avoid scientific misunderstanding
    • SDM1.1 has an ad-hoc representation for upper limits (only) which may be usable, but is conceptually incorrect.
The main changes in the v1.2 document are:
  • Section 4.1.1: addition of 'order' and 'relorder' attributes
  • Section 4.6.3: addition of 'UpperLimit' and 'LowerLimit' to explicitly express these concepts
  • Corrected UTypes in VOTable example #1
  • Updated diagrams to proper UML with improved consistency with the schema and text.
Latest version of SpectrumDM can be found at: The GitHub repository for issues and model development can be found at: The IVOA Twiki page for the project can be found at:
Changed:
<
<

Reference Interoperable Implementations

>
>

<a name="Reference Interoperable Implemen"></a>Reference Interoperable Implementations

  The implementation level for this RFE is not as comprehensive as we would typically look for in a REC.

The DM working group consulted with the TCG prior to the start of this RFC and it was agreed that given

  • this is a minor revision based on a specific institution request
  • the request is targeted to serve a very specific purpose
  • the model is not VO-DML compliant, so the level of effort required to produce additional implementations, serializations, validators is out of proportion to the impact of the RFE.
the level of implementation for this enhancement is acceptable.

Data Providers:

Applications:
  • IPAC Firefly Archive toolkit:
    • "For some time, IPAC's Firefly archive GUI toolkit has taken advantage of IVOA Spectral Data Model (SDM) v1.1 to provide users with automatic plots of spectra that are the result of a query or have been uploaded to the tool. The plotting capability uses the SDM to identify which data to plot on the x and y axes and to change units. Firefly has recently been updated to recognize the new utypes specified in the proposed Spectral Data Model v1.2, namely:

      spec:Spectrum.Data.SpectralAxis.Order
      spec:Spectrum.Data.FluxAxis.Accuracy.LowerLimit
      spec.Spectrum.Data.FluxAxis.Accuracy.UpperLimit

      The toolkit uses these new utypes to plot orders (as found in e.g. Spitzer spectra) as separate traces, and to plot upper and lower limits as arrows (as found in e.g. SEDs). These changes are currently in test and will be released in May. Upon release, these features will be available in IRSA, Rubin, and NED interfaces, all of which are built upon the open-source Firefly toolkit.

      In a future release, Firefly will recognize the spec:Spectrum.Data.SpectralAxis.RelOrder utype proposed in SDMv1.2. When Firefly encounters a spectrum with only RelOrder and not Order, it will automatically plot each unique RelOrder as a different trace, analogous to how it treats Order. If a spectrum contains both Order and RelOrder, Firefly will automatically plot traces based on Order but will provide the user a toggle to plot RelOrder."
Changed:
<
<

Implementations Validators

>
>

<a name="Implementations Validators"></a>Implementations Validators

  There is no dedicated validator for Spectrum data model instances to update for this enhancement.

Since this is not a VO-DML compliant model, the mechanisms for

  • validating the data model itself
  • generating and validating example serializations in VOTable, XML, etc.
  • generating model based code
are not available.

Changed:
<
<

Comments from the IVOA Community during RFC/TCG review period: 2023-09-11 - 2023-10-23

>
>

<a name="Comments from the IVOA Community"></a>Comments from the IVOA Community during RFC/TCG review period: 2023-09-11 - 2023-10-23

  The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Added:
>
>
Comment by JesusSalgado

This is a preliminary comment: Version 1.0 had a java library implemented by Kelly McCusker (not in IVOA anymore) and Jonathan McDowell that allowed parsing and writing of spectral data compatible with 1.0. It would be great to obtain a version of this library and adapt it for these minor changes.

This library was used by some tools (e.g. VOSpec)

JesusSalgado - 2023-09-22

 
Changed:
<
<

Comments from TCG member during the RFC/TCG Review Period: 2023-09-11 - 2023-10-23

>
>

<a name="Comments from TCG member during"></a>Comments from TCG member during the RFC/TCG Review Period: 2023-09-11 - 2023-10-23

  WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

Changed:
<
<

TCG Chair & Vice Chair

>
>

<a name="TCG Chair Vice Chair"></a>TCG Chair & Vice Chair

 
Changed:
<
<

Applications Working Group

>
>

<a name="Applications Working Group"></a> Applications Working Group

 
Changed:
<
<

Data Access Layer Working Group

>
>

<a name="Data Access Layer Working Group"></a> Data Access Layer Working Group

 
Changed:
<
<

Data Model Working Group

>
>

<a name="Data Model Working Group"></a> Data Model Working Group

 
Changed:
<
<

Grid & Web Services Working Group

>
>

<a name="Grid Web Services Working Group"></a> Grid & Web Services Working Group

 
Changed:
<
<

Registry Working Group

>
>

<a name="Registry Working Group"></a> Registry Working Group

 
Changed:
<
<

Semantics Working Group

>
>

<a name="Semantics Working Group"></a> Semantics Working Group

 
Changed:
<
<

Data Curation & Preservation Interest Group

>
>

<a name="Data Curation Preservation Inte"></a> Data Curation & Preservation Interest Group

 
Changed:
<
<

Education Interest Group

>
>

<a name="Education Interest Group"></a> Education Interest Group

 
Changed:
<
<

Knowledge Discovery Interest Group

>
>

<a name="Knowledge Discovery Interest Gro"></a> Knowledge Discovery Interest Group

 
Changed:
<
<

Operations Interest Group

>
>

<a name="Operations Interest Group"></a> Operations Interest Group

 
Changed:
<
<

Radio Astronomy Interest Group

>
>

<a name="Radio Astronomy Interest Group"></a> Radio Astronomy Interest Group

 
Changed:
<
<

Solar System Interest Group

>
>

<a name="Solar System Interest Group"></a> Solar System Interest Group

 
Changed:
<
<

Theory Interest Group

>
>

<a name="Theory Interest Group"></a> Theory Interest Group

 
Changed:
<
<

Time Domain Interest Group

>
>

<a name="Time Domain Interest Group"></a> Time Domain Interest Group

 
Changed:
<
<

Standards and Processes Committee

>
>

<a name="Standards and Processes Committe"></a> Standards and Processes Committee

 
Changed:
<
<

TCG Vote : Vote_start_date - Vote_end_date

>
>

<a name="TCG Vote : Vote_start_date - Vot"></a>TCG Vote : Vote_start_date - Vote_end_date

  If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.
Changed:
<
<
Group Yes No Abstain Comments
TCG
Apps
DAL
DM
GWS
Registry
Semantics
DCP
Edu
KDIG
Ops
Radio
SSIG
Theory
TD
StdProc
>
>
Group Yes No Abstain Comments
TCG
Apps
DAL
DM
GWS
Registry
Semantics
DCP
Edu
KDIG
Ops
Radio
SSIG
Theory
TD
StdProc
 

Revision 22023-09-11 - MarkCresitelloDittmar

 
META TOPICPARENT name="IvoaDataModel"
Changed:
<
<

Spectrum 1.2 Proposed Recommendation: Request for Comments

>
>

Spectrum 1.2 Proposed Recommendation: Request for Comments

 
Changed:
<
<

>
>
Deleted:
<
<
 

Introduction

The Spectrum Data Model describes the structure of spectrophotometric datasets with spectral and temporal coordinates and associated metadata.

This update satisfies an enhancement request by IPAC to add support for the following use cases:

Changed:
<
<
  • 1-D spectra from Spitzer’s Infrared Spectrograph have multiple spectral orders
>
>
  • 1-D spectra from Spitzer’s Infrared Spectrograph have multiple spectral orders
 
    • Spectral orders can overlap in wavelength
    • Plotting a Spitzer spectrum without accounting for orders gives you a mess
Changed:
<
<
  • SEDs often include limits on measurements
>
>
  • SEDs often include limits on measurements
 
    • these can be upper or lower limits
    • Plotters need to indicate limits clearly to avoid scientific misunderstanding
    • SDM1.1 has an ad-hoc representation for upper limits (only) which may be usable, but is conceptually incorrect.
The main changes in the v1.2 document are:
  • Section 4.1.1: addition of 'order' and 'relorder' attributes
  • Section 4.6.3: addition of 'UpperLimit' and 'LowerLimit' to explicitly express these concepts
  • Corrected UTypes in VOTable example #1
  • Updated diagrams to proper UML with improved consistency with the schema and text.
Changed:
<
<		Latest version of SpectrumDM can be found at:
  • <URL to document in ivoa.net>
>
>		Latest version of SpectrumDM can be found at:
Added:
>
>
 The GitHub repository for issues and model development can be found at: The IVOA Twiki page for the project can be found at:
Changed:
<
<

Reference Interoperable Implementations

>
>

Reference Interoperable Implementations

  The implementation level for this RFE is not as comprehensive as we would typically look for in a REC.

The DM working group consulted with the TCG prior to the start of this RFC and it was agreed that given

  • this is a minor revision based on a specific institution request
  • the request is targeted to serve a very specific purpose
  • the model is not VO-DML compliant, so the level of effort required to produce additional implementations, serializations, validators is out of proportion to the impact of the RFE.
the level of implementation for this enhancement is acceptable.
Changed:
<
<		Data Providers:
  • GAVO: Updated SSA Server to
>
>		Data Providers:
  • GAVO: Updated SSA Server to
 
    • include a PARAM with UTYPE="spec:Spectrum.Data.SpectralAxis.order" into each Spectrum Table.
Changed:
<
<
    • also includes other changes/corrections from a review of the output against the Spectrum model document.
>
>
    • also includes other changes/corrections from a review of the output against the Spectrum model document.
 
Changed:
<
<		Applications:
  • IPAC Firefly Archive toolkit:
>
>		Applications:
  • IPAC Firefly Archive toolkit:
 
    • "For some time, IPAC's Firefly archive GUI toolkit has taken advantage of IVOA Spectral Data Model (SDM) v1.1 to provide users with automatic plots of spectra that are the result of a query or have been uploaded to the tool. The plotting capability uses the SDM to identify which data to plot on the x and y axes and to change units. Firefly has recently been updated to recognize the new utypes specified in the proposed Spectral Data Model v1.2, namely:

      spec:Spectrum.Data.SpectralAxis.Order
      spec:Spectrum.Data.FluxAxis.Accuracy.LowerLimit
      spec.Spectrum.Data.FluxAxis.Accuracy.UpperLimit

      The toolkit uses these new utypes to plot orders (as found in e.g. Spitzer spectra) as separate traces, and to plot upper and lower limits as arrows (as found in e.g. SEDs). These changes are currently in test and will be released in May. Upon release, these features will be available in IRSA, Rubin, and NED interfaces, all of which are built upon the open-source Firefly toolkit.

      In a future release, Firefly will recognize the spec:Spectrum.Data.SpectralAxis.RelOrder utype proposed in SDMv1.2. When Firefly encounters a spectrum with only RelOrder and not Order, it will automatically plot each unique RelOrder as a different trace, analogous to how it treats Order. If a spectrum contains both Order and RelOrder, Firefly will automatically plot traces based on Order but will provide the user a toggle to plot RelOrder."
Changed:
<
<

Implementations Validators

>
>

Implementations Validators

  There is no dedicated validator for Spectrum data model instances to update for this enhancement.

Since this is not a VO-DML compliant model, the mechanisms for

  • validating the data model itself
  • generating and validating example serializations in VOTable, XML, etc.
Changed:
<
<
  • generating model based code
>
>
  • generating model based code
 are not available.

Comments from the IVOA Community during RFC/TCG review period: 2023-09-11 - 2023-10-23

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Comments from TCG member during the RFC/TCG Review Period: 2023-09-11 - 2023-10-23

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

Changed:
<
<

TCG Chair & Vice Chair

>
>

TCG Chair & Vice Chair

 

Applications Working Group

Data Access Layer Working Group

Data Model Working Group

Grid & Web Services Working Group

Registry Working Group

Semantics Working Group

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Operations Interest Group

Radio Astronomy Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Standards and Processes Committee

TCG Vote : Vote_start_date - Vote_end_date

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Changed:
<
<
Group Yes No Abstain Comments
TCG
Apps
DAL
DM
GWS
Registry
Semantics
DCP
Edu
KDIG
Ops
Radio
SSIG
Theory
TD
StdProc
>
>
Group Yes No Abstain Comments
TCG
Apps
DAL
DM
GWS
Registry
Semantics
DCP
Edu
KDIG
Ops
Radio
SSIG
Theory
TD
StdProc
 

Revision 12023-09-07 - MarkCresitelloDittmar

 
META TOPICPARENT name="IvoaDataModel"

Spectrum 1.2 Proposed Recommendation: Request for Comments


Introduction

The Spectrum Data Model describes the structure of spectrophotometric datasets with spectral and temporal coordinates and associated metadata.

This update satisfies an enhancement request by IPAC to add support for the following use cases:

  • 1-D spectra from Spitzer’s Infrared Spectrograph have multiple spectral orders
    • Spectral orders can overlap in wavelength
    • Plotting a Spitzer spectrum without accounting for orders gives you a mess
  • SEDs often include limits on measurements
    • these can be upper or lower limits
    • Plotters need to indicate limits clearly to avoid scientific misunderstanding
    • SDM1.1 has an ad-hoc representation for upper limits (only) which may be usable, but is conceptually incorrect.
The main changes in the v1.2 document are:
  • Section 4.1.1: addition of 'order' and 'relorder' attributes
  • Section 4.6.3: addition of 'UpperLimit' and 'LowerLimit' to explicitly express these concepts
  • Corrected UTypes in VOTable example #1
  • Updated diagrams to proper UML with improved consistency with the schema and text.
Latest version of SpectrumDM can be found at:
  • <URL to document in ivoa.net>
The GitHub repository for issues and model development can be found at: The IVOA Twiki page for the project can be found at:

Reference Interoperable Implementations

The implementation level for this RFE is not as comprehensive as we would typically look for in a REC.

The DM working group consulted with the TCG prior to the start of this RFC and it was agreed that given

  • this is a minor revision based on a specific institution request
  • the request is targeted to serve a very specific purpose
  • the model is not VO-DML compliant, so the level of effort required to produce additional implementations, serializations, validators is out of proportion to the impact of the RFE.
the level of implementation for this enhancement is acceptable.

Data Providers:

Applications:
  • IPAC Firefly Archive toolkit:
    • "For some time, IPAC's Firefly archive GUI toolkit has taken advantage of IVOA Spectral Data Model (SDM) v1.1 to provide users with automatic plots of spectra that are the result of a query or have been uploaded to the tool. The plotting capability uses the SDM to identify which data to plot on the x and y axes and to change units. Firefly has recently been updated to recognize the new utypes specified in the proposed Spectral Data Model v1.2, namely:

      spec:Spectrum.Data.SpectralAxis.Order
      spec:Spectrum.Data.FluxAxis.Accuracy.LowerLimit
      spec.Spectrum.Data.FluxAxis.Accuracy.UpperLimit

      The toolkit uses these new utypes to plot orders (as found in e.g. Spitzer spectra) as separate traces, and to plot upper and lower limits as arrows (as found in e.g. SEDs). These changes are currently in test and will be released in May. Upon release, these features will be available in IRSA, Rubin, and NED interfaces, all of which are built upon the open-source Firefly toolkit.

      In a future release, Firefly will recognize the spec:Spectrum.Data.SpectralAxis.RelOrder utype proposed in SDMv1.2. When Firefly encounters a spectrum with only RelOrder and not Order, it will automatically plot each unique RelOrder as a different trace, analogous to how it treats Order. If a spectrum contains both Order and RelOrder, Firefly will automatically plot traces based on Order but will provide the user a toggle to plot RelOrder."

Implementations Validators

There is no dedicated validator for Spectrum data model instances to update for this enhancement.

Since this is not a VO-DML compliant model, the mechanisms for

  • validating the data model itself
  • generating and validating example serializations in VOTable, XML, etc.
  • generating model based code
are not available.

Comments from the IVOA Community during RFC/TCG review period: 2023-09-11 - 2023-10-23

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Comments from TCG member during the RFC/TCG Review Period: 2023-09-11 - 2023-10-23

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

Data Model Working Group

Grid & Web Services Working Group

Registry Working Group

Semantics Working Group

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Operations Interest Group

Radio Astronomy Interest Group

Solar System Interest Group

Theory Interest Group

Time Domain Interest Group

Standards and Processes Committee

TCG Vote : Vote_start_date - Vote_end_date

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG
Apps
DAL
DM
GWS
Registry
Semantics
DCP
Edu
KDIG
Ops
Radio
SSIG
Theory
TD
StdProc
View topic | History: r19 < r18 < r17 < r16 | More topic actions...
 
This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback