Spectrum Data Model: version 1.2

Goals

This update is to satisfy requested enhancements to the v1.1 model document.

Presentations by Vandana Desai

• Nov. 2020 Interop: Implementation of the IVOA Spectral Data Model at IPAC” assessment of V1.1 Recommendation
• May 2021 Interop: "IVOA Spectral Models and Access in the Era of Big Data” enhancement request to support IPAC use cases

Participants

data modeler: MarkCresitelloDittmar

editor(s): MarkCresitelloDittmar

implementer(s):

• Firefly:
  • Spectral plotter
• CASSIS: ???
  • Spectral analyser
• GAVO:
  • data provider

Use Cases

The requests stem from visualization use cases:

Discussion Threads

Preliminary meetings:

• Interop DAL/DM Session - May 27, 2021
• DAL/DM Running meeting - June 30, 2021
• IVOA - Spectrum Data Model change request - Oct. 15, 2021

Planning meetings:

• DM Meeting 20220622: Notes

• UCDs assigment for new elements - Jul 21, 2022
• UTypes in VOTable examples - Aug 18, 2022
• Invalid UCDs in standard - Mar. 20, 2023

Data Model Updates

The document is being updated in the IVOA Git repository

SpectrumDM: branch wd-v1.2: (https://github.com/ivoa-std/SpectrumDM/tree/wd-v1.2)

• Migration to Git - preparatory work
  • port Spectrum 1.1 REC document files to model repository
    • tag release (REC-V1.1-20111020)
  • create branch for RFE
  • restructure repository content to match other models
  • migrate model diagrams to proper UML
    • improve consistency with schema and description
• List of Changes:
  • 2022 Jun 22: Document reset to WD for enhancement
  • 2022 Jul 19: Add spectral order elements
  • 2022 Jul 20: Add upper/lower limit elements
  • 2023 Mar 17: Correct case for order and relorder attributes to match spec.

• Open Issues:

• • #7: Invalid UCD in standard
    • Review and discussion of DaCHs serializations identified several UCDs in the current REC which are invalid.
    • Discussion for Issue #5 identified some other actions which could be taken to improve the existing UCDs
      • Update UCD Vocabulary Reference is Section 3.4
      • Add tags which enable validation of UCDs contained in teh document (during build)
      • There is a UCD RFE documented on this wiki page; it is partially resolved in V1.1 and should be closed out.

Implementations

Data Providers:

• GAVO: Updated SSA Server to
  • include a PARAM with UTYPE="spec:Spectrum.Data.SpectralAxis.order" into each Spectrum Table.
  • also includes other changes/corrections from a review of the output against the Spectrum model document.
    • example file: http://dc.zah.uni-heidelberg.de/getproduct/flashheros/data_raw/ca98/blue/n0393.mt
    • server (sample query): TBD

Applications:

• IPAC Firefly Archive toolkit:
  • "For some time, IPAC's Firefly archive GUI toolkit has taken advantage of IVOA Spectral Data Model (SDM) v1.1 to provide users with automatic plots of spectra that are the result of a query or have been uploaded to the tool. The plotting capability uses the SDM to identify which data to plot on the x and y axes and to change units. Firefly has recently been updated to recognize the new utypes specified in the proposed Spectral Data Model v1.2, namely:
    
    spec:Spectrum.Data.SpectralAxis.Order
    spec:Spectrum.Data.FluxAxis.Accuracy.LowerLimit
    spec.Spectrum.Data.FluxAxis.Accuracy.UpperLimit
    
    The toolkit uses these new utypes to plot orders (as found in e.g. Spitzer spectra) as separate traces, and to plot upper and lower limits as arrows (as found in e.g. SEDs). These changes are currently in test and will be released in May. Upon release, these features will be available in IRSA, Rubin, and NED interfaces, all of which are built upon the open-source Firefly toolkit.
    In a future release, Firefly will recognize the spec:Spectrum.Data.SpectralAxis.RelOrder utype proposed in SDMv1.2. When Firefly encounters a spectrum with only RelOrder and not Order, it will automatically plot each unique RelOrder as a different trace, analogous to how it treats Order. If a spectrum contains both Order and RelOrder, Firefly will automatically plot traces based on Order but will provide the user a toggle to plot RelOrder."

<!--
* Set ALLOWTOPICRENAME = TWikiAdminGroup
-->

Spectrum Data Model: version 1.2

Goals

This update is to satisfy requested enhancements to the v1.1 model document.

Presentations by Vandana Desai

• Nov. 2020 Interop: Implementation of the IVOA Spectral Data Model at IPAC” assessment of V1.1 Recommendation
• May 2021 Interop: "IVOA Spectral Models and Access in the Era of Big Data” enhancement request to support IPAC use cases

Participants

data modeler: MarkCresitelloDittmar

editor(s): MarkCresitelloDittmar

implementer(s):

• Firefly:
  • Spectral plotter
• CASSIS: ???
  • Spectral analyser
• GAVO:
  • data provider

Use Cases

The requests stem from visualization use cases:

Discussion Threads

Preliminary meetings:

• Interop DAL/DM Session - May 27, 2021
• DAL/DM Running meeting - June 30, 2021
• IVOA - Spectrum Data Model change request - Oct. 15, 2021

Planning meetings:

• DM Meeting 20220622: Notes

• UCDs assigment for new elements - Jul 21, 2022
• UTypes in VOTable examples - Aug 18, 2022
• Invalid UCDs in standard - Mar. 20, 2023

Data Model Updates

The document is being updated in the IVOA Git repository

SpectrumDM: branch wd-v1.2: (https://github.com/ivoa-std/SpectrumDM/tree/wd-v1.2)

• Migration to Git - preparatory work
  • port Spectrum 1.1 REC document files to model repository
    • tag release (REC-V1.1-20111020)
  • create branch for RFE
  • restructure repository content to match other models
  • migrate model diagrams to proper UML
    • improve consistency with schema and description
• List of Changes:
  • 2022 Jun 22: Document reset to WD for enhancement
  • 2022 Jul 19: Add spectral order elements
  • 2022 Jul 20: Add upper/lower limit elements
  • 2023 Mar 17: Correct case for order and relorder attributes to match spec.
  • 2023 Apr 27: Correct UTypes in VOTable example 1; assign UCDs for upper/lower limits
• Open Issues:
  • #5: UCD assignment (resolved?)
    • selection of UCDs for the new elements had no clear resolution within the working group
    • last question of the discussion thread really sums it up
      • "are we talking about the upper/lower limits of the instrument or the source? Both are related, but the interpretation is different"
    • based on this discussion I have (tentatively) employed the following definitions
      • lower limit: "The minimum flux of the source due to instrument saturation" ==> instr.saturation;phot.flux.density;stat.min"
      • upper limit: "The maximum flux of the source to remain undetected due to instrument sensitivity" ==> instr.sensitivity;phot.flux.density;stat.max"
  • #7: Invalid UCD in standard
    • Review and discussion of DaCHs serializations identified several UCDs in the current REC which are invalid.
    • Discussion for Issue #5 identified some other actions which could be taken to improve the existing UCDs
      • Update UCD Vocabulary Reference is Section 3.4
      • Add tags which enable validation of UCDs contained in teh document (during build)
      • There is a UCD RFE documented on this wiki page; it is partially resolved in V1.1 and should be closed out.

Implementations

Data Providers:

• GAVO: Updated SSA Server to
  • include a PARAM with UTYPE="spec:Spectrum.Data.SpectralAxis.order" into each Spectrum Table.
  • also includes other changes/corrections from a review of the output against the Spectrum model document.

• IPAC Firefly Archive toolkit:
  • "For some time, IPAC's Firefly archive GUI toolkit has taken advantage of IVOA Spectral Data Model (SDM) v1.1 to provide users with automatic plots of spectra that are the result of a query or have been uploaded to the tool. The plotting capability uses the SDM to identify which data to plot on the x and y axes and to change units. Firefly has recently been updated to recognize the new utypes specified in the proposed Spectral Data Model v1.2, namely:
    
    spec:Spectrum.Data.SpectralAxis.Order
    spec:Spectrum.Data.FluxAxis.Accuracy.LowerLimit
    spec.Spectrum.Data.FluxAxis.Accuracy.UpperLimit
    
    The toolkit uses these new utypes to plot orders (as found in e.g. Spitzer spectra) as separate traces, and to plot upper and lower limits as arrows (as found in e.g. SEDs). These changes are currently in test and will be released in May. Upon release, these features will be available in IRSA, Rubin, and NED interfaces, all of which are built upon the open-source Firefly toolkit.
    In a future release, Firefly will recognize the spec:Spectrum.Data.SpectralAxis.RelOrder utype proposed in SDMv1.2. When Firefly encounters a spectrum with only RelOrder and not Order, it will automatically plot each unique RelOrder as a different trace, analogous to how it treats Order. If a spectrum contains both Order and RelOrder, Firefly will automatically plot traces based on Order but will provide the user a toggle to plot RelOrder."

Spectrum Data Model: version 1.2

Goals

This update is to satisfy requested enhancements to the v1.1 model document.

Presentations by Vandana Desai

• Nov. 2020 Interop: Implementation of the IVOA Spectral Data Model at IPAC” assessment of V1.1 Recommendation
• May 2021 Interop: "IVOA Spectral Models and Access in the Era of Big Data” enhancement request to support IPAC use cases

Participants

data modeler: MarkCresitelloDittmar

editor(s): MarkCresitelloDittmar

implementer(s):

• • Spectral plotter
• CASSIS: ???
  • Spectral analyser

• • data provider

Use Cases

The requests stem from visualization use cases:

Discussion Threads

Preliminary meetings:

• Interop DAL/DM Session - May 27, 2021
• DAL/DM Running meeting - June 30, 2021
• IVOA - Spectrum Data Model change request - Oct. 15, 2021

Planning meetings:

• DM Meeting 20220622: Notes

• UCDs assigment for new elements - Jul 21, 2022
• UTypes in VOTable examples - Aug 18, 2022
• Invalid UCDs in standard - Mar. 20, 2023

Data Model Updates

The document is being updated in the IVOA Git repository

• Migration to Git - preparatory work
  • port Spectrum 1.1 REC document files to model repository
    • tag release (REC-V1.1-20111020)
  • create branch for RFE
  • restructure repository content to match other models
  • migrate model diagrams to proper UML
    • improve consistency with schema and description
• List of Changes:
  • 2022 Jun 22: Document reset to WD for enhancement
  • 2022 Jul 19: Add spectral order elements
  • 2022 Jul 20: Add upper/lower limit elements
  • 2023 Mar 17: Correct case for order and relorder attributes to match spec.
  • 2023 Apr 27: Correct UTypes in VOTable example 1; assign UCDs for upper/lower limits
• Open Issues:

• • • selection of UCDs for the new elements had no clear resolution within the working group
    • last question of the discussion thread really sums it up
      • "are we talking about the upper/lower limits of the instrument or the source? Both are related, but the interpretation is different"
    • based on this discussion I have (tentatively) employed the following definitions
      • lower limit: "The minimum flux of the source due to instrument saturation" ==> instr.saturation;phot.flux.density;stat.min"
      • upper limit: "The maximum flux of the source to remain undetected due to instrument sensitivity" ==> instr.sensitivity;phot.flux.density;stat.max"

• • • Discussion for Issue #5 identified some other actions which could be taken to improve the existing UCDs
      • Update UCD Vocabulary Reference is Section 3.4
      • Add tags which enable validation of UCDs contained in teh document (during build)
      • There is a UCD RFE documented on this wiki page; it is partially resolved in V1.1 and should be closed out.

Implementations

Spectrum Data Model: version 1.2

Goals

This update is to satisfy requested enhancements to the v1.1 model document.

Presentations by Vandana Desai

Participants

data modeler: MarkCresitelloDittmar

editor(s): MarkCresitelloDittmar

implementer(s):

• Firefly: ???
  • Spectral plotter
• CASSIS: ???
  • Spectral analyser
• GAVO: ???
  • data provider

Use Cases

The requests stem from visualization use cases:

• • Spectral orders can overlap in wavelength
  • Plotting a Spitzer spectrum without accounting for orders gives you a mess
• SEDs often include limits on measurements
  • these can be upper or lower limits
  • Plotters need to indicate limits clearly to avoid scientific misunderstanding
  • SDM1.1 has an ad-hoc representation for upper limits (only) which may be usable, but is conceptually incorrect.

Discussion Threads

Preliminary meetings:

• Interop DAL/DM Session - May 27, 2021
• DAL/DM Running meeting - June 30, 2021
• IVOA - Spectrum Data Model change request - Oct. 15, 2021

Planning meetings:

• DM Meeting 20220622: Notes

Data Model Updates

The document is being updated in the IVOA Git repository

Implementations

TBD

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->