SSA11RFC < IVOA

IVOA Web>IvoaDAL>SSA11RFC (2011-10-20, PatrickDowler) (raw view)
---++ SSA-1.1 and Spectrum-1.1 Proposed Recommendation: Request for Comments

This document serves as the RFC center for the Proposed Recommendation entitled "Simple Spectral Access, Version 1.1". 
At the same time the Spectrum Data model document has been updated 
in order to correct minor inconsitencies in notation with respect to the SSA protocol standard document ( Utypes mispell, UCD, Fits keywords completed.)

The latest version of SSA after TCG review is:
   * [[%ATTACHURL%/PR-SSA-1.1-20111018.pdf][PR-SSA-1.1-20111018]]

The version reviewed by the TCG is:

   * http://www.ivoa.net/Documents/SSA/20110706/

Latest version of Spectrum Data Model can be found at:
   * http://wiki.ivoa.net/internal/IVOA/IVOADMSpectraWP/PR-SpectrumDM-1.1-20110318.pdf

Additional discussion about any of the comments or responses can be conducted on the Data Access Layer mailing list, dal@ivoa.net and on the Data Model mailing list, dm@ivoa.net However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

---+++ Reference Interoperable Implementations
---+++ Implementations Validators

---+++ RFC Review Period: 20 Apr 2011 - 17 May 2011
---+++ TCG Review Period: 01 June 2011 - 28 June 2011

<hr/>
<hr/>

---++ Comments from the IVOA Community during RFC period: 20 Apr 2011 - 17 May 2011

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document 

   * Sample comment by TWiki.WikiName
      * Response (by TWiki.WikiName)

<hr/>
<hr/>

---++ Comments from TCG member during the TCG Review Period: 01 June 2011 - 28 June 2011

WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

---+++ TCG Chair & Vice Chair (<i>Christophe Arviset, Séverin Gaudet</i>)

Comments on SpectrumDM 1.1 - IVOA.ChristopheArviset 2011-06-07

All my comments are mainly editorial or related to document formatting. I'll be happy to approve SpectrumDM 1.1 once they've been addressed.

[Section 1 Introduction and Motivation]
This section should already clearly indicate the reason for SpectrumDM 1.1, which is mainly for corrections coming from feedback implementation and does not represent any fundamental changes wrt the existing Recommended SpectrumDM 1.03 released on 29 October 2007. It should also indicate its "pairing" with SSAP 1.1.

  * 2011-10-20 response (by OmarLaurino) - I added a paragraph to the text stating the scope of the changes. However, I can't state that they do not represent fundamental changes since they break the existing validators for SpectrumDM1.03. I would also suggest that Spectrum DM doesn't refers to SSAP, since it doesn't depend on it in any way (while the opposite makes sense to me, since SSAP "uses" SpectrumDM).


[Section 1.1 Change log]
This section should also clearly indicate the Recommended SpectrumDM 1.03 released on 29 October 2007

  * 2011-10-20 response (by OmarLaurino) - Done.

[Section 1.2 IVOA Architecture]
Last paragraph should either be rewritten to better reflect the relationship between SpectrumDM 1.03, SpectrumDM 1.1 and the upcoming SpectrumDM 1.2 or should be removed (one could argue why we need a reference to SpectrumDM 1.2 in the SpectrumDM 1.1 document?).

  * 2011-10-20 response (by OmarLaurino) - Agreed, I dropped the paragraph.

[References] 
The references section should be udpated with latest version of standards and should contain all IVOA standards introduced in Section 1.2

  * 2011-10-20 response (by OmarLaurino) - Done.

Various formatting issues:
   * Blank pages that should probably be removed: 18, 22, 66, 81
   * Landscape pages that are cut on the border: 19, 20, 21, 23, 24, 25, 26
   * Pages cut on the border due to too long text: 40, 41, 55, 57



Comments on SSAP 1.1 - IVOA.ChristopheArviset 2011-06-07

All my comments are mainly editorial. I'll be happy to approve SSAP 1.1 once they've been addressed.

[Page 2] Reference to VOSI should be updated as it has now become a REC

 * 2011-10-20 - IVOA.PatrickDowler - done

[Section 1] Introduction
As agreed, together with the IVOA Architecture Diagram, some text should explain the relationship between SSAP and the other IVOA Standards present on that diagram.

The SSAP text used for the IVOA Architecture document could probably be inserted as is:

Simple Spectra Access Protocol (SSAP) defines a uniform interface to remotely discover and access one dimensional spectrum coming from data and metadata collections. SSAP is based on the Spectrum Data Model (itself making reference to the CharDM) that is capable of describing most tabular spectrophotometric data, including time series and spectral energy distributions (SEDs) as well as 1-D spectra. SSAP can be used from VO applications to access the associated spectrum resources.
As with most of the IVOA Data Access Protocols, SSAP makes use of VOTable for metadata exchange, STC, UCD, Utypes and Units for metadata description.
An SSAP service is to be registered into the VO Registry, using the associated registry standards (in particular Resource Metadata, VOResource and SimpleDALRegExt specifically for Data Access Protocols such as SSAP). Once registered, an SSAP service will get a unique IVOA Resource Identifier. Furthermore, an SSAP service should be registered with its supported interfaces through the VOSI standard.

 * 2011-10-20 - IVOA.PatrickDowler - done

---+++ Applications Working Group (<i>Tom Mcglynn, Mark Taylor</i>)

Approved - IVOA.TomMcGlynn 2011-06-12

(but fix or remove reference in SSA Section 8.10.2 which contains the text "see also section 8.10.2" -- IVOA.MarkTaylor - 04 Jul 2011)

 * 2011-10-20 - IVOA.PatrickDowler - done

---+++ Data Access Layer Working Group (<i>Patrick Dowler, Mike Fitzpatrick</i>)

Approved - IVOA.PatrickDowler 2011-06-01

---+++ Data Model Working Group (<i>Jesus Salgado, Omar Laurino</i>)

Approved - IVOA.JesusSalgado 2011-07-01

---+++ Grid & Web Services Working Group (<i>Andreas Wicenec,      Andre Schaaff </i>)

Approved - IVOA.AndreasWicenec

---+++ Registry Working Group (<i>Gretchen Greene, Pierre Le Sidaner</i>)

Approved with comments - IVOA.GretchenGreene 2011-09-19

1.3.2 Parameters
Suggest version should be defined via VOSI not in the protocol definition (twice describe and moreover it's not mandatory)

 * 2011-10-20 - IVOA.PatrickDowler - not changed: the VERSION param is for protocol version negotiation, not metadata about the service

2.1 Dataset and Data Collection 
possbily add link to the VODataService document where all voservice concepts defined 
same comment on 4.1.2.14 COLLECTION 

 * 2011-10-20 - IVOA.PatrickDowler - not changed: such a change w.r.t. SSA-1.0 could imply a change that is out of scope for a bug fix revision.

2.4 Virtual data
do not understand this sentence, it is a capability of cutout. This should be decribe in vosi or regex
It's more important in the case of cutout or slice cut of spectral cube.

 * 2011-10-20 - IVOA.PatrickDowler - not changed: virtual data generation is a service implementation detail (decision) not controlled by users; it is not not a capability per se in SSA-1.1

2.5.1 Data Source 
artificial :  Artificial or simulated data. This is similar to theory data but need not be based 
on a physical model, and is often used for testing purposes. 
do not understand the interest of this item : the main purpose is the input parameter that make a 
difference between simulated and observational data. Why don't we simply keep simulated or theoretical data 

 * 2011-10-20 - IVOA.PatrickDowler - not changed: such a change w.r.t. SSA-1.0 could imply a change that is out of scope for a bug fix revision.

2.7 Services, Interfaces, and Protocols
is the meaning of this paragraph that it's a HTTP protocol ?

 * 2011-10-20 - IVOA.PatrickDowler - not changed: yes, this is not entirely clear, but changes w.r.t. SSA-1.0 could imply a change that is out of scope for a bug fix revision.

2.8 Dataset Identifiers
Why reinventing the wheel ?
There is an IVOA docume "IVOA Identifiers" If it need some evolution. Then it should be in this document not in SSA.

 * 2011-10-20 - IVOA.PatrickDowler - not changed: not reinventing identifiers, do not want to imply a change by changing language here.

2.9 Provenance
No mention of DM provenance or VOResource

 * 2011-10-20 - IVOA.PatrickDowler - not fixed: there is not Provenance REC to refer to. No specific need to refer to VOResource.

2.10 Data Association
Should be described in the registry,  same for all protocols

 * 2011-10-20 - IVOA.PatrickDowler - not changed: association here is a fine-grained association of multipel records in the response.

3.2 Methods & Protocols 
Does anyone want to implement SSAP in  SOAP ?
May be say also REST ?
the 3.3.1 GetCapabilities and 3.3.3 GetAvailability can mention that this is under definition in the VOSI document 
and make a ref at the end to the proposed recommendation version at least 
ref http://www.ivoa.net/Documents/VOSI/20101206/PR-VOSI-1.0-20101206.pdf 
instead of http://wiki.ivoa.net/internal/IVOA/IvoaGridAndWebServices/VOSupport 
InterfacesMandatory-0.26.pdf 
at the end of the document 

 * 2011-10-20 - IVOA.PatrickDowler - reference changed to VOSI-1.0 even though getCapabilities remains un-specified here. Implementors may optionally do the right thing.

3.3.2 StageData 
you mention " is not planned for SSA V1.0", may be it's 1.1 

 * 2011-10-20 - IVOA.PatrickDowler - was out of scope

4.1.1.5 FORMAT
"If FORMAT is omitted, FORMAT=ALL should be assumed" don't you think 
it verbosity will increase really the size of the output.  What not default to votable ?

 * 2011-10-20 - IVOA.PatrickDowler - the FORMAT param here is a query param and not a format (of the response) request (as in TAP). This sort of inconsistency will be removed in SSA-2.0 where incompatible changes are possible.

8.10.2 Overflow 
Previous point made, ? understanding the new changes 
in case of overflow no client yet redo the query with maxrec 
don't you think the service should answer Overflow an produce a query response 
containing the maximum number of records. Then the client could at once give to the user the first N response and inform that there is an overflow. 
is it still compliant with 
  4.1.2.16 MAXREC 
  " A service should typically have a 
  fairly small default MAXREC, provided to improve the query response time, and a large 
  upper limit on MAXREC, provided to enable large queries." 
  that mean that the service will all the time answer an overflow and will not provide data ? 

 * 2011-10-20 - IVOA.PatrickDowler - MAXREC accomplished the limited goals possible in SSA. Services still return up to MAXREC rows, which could be unlimited (service implementation choice).

---+++ Semantics Working Group (<i>Sebastien Derriere, Norman Gray</i>)

_Spectral DM 1.1 Approved with comments. SD 2011.10.05_
Some minor typographic corrections, but also many tedious small fixes to UCDs and units that should improve consistency with other standards.

p15 : Unified Content Descriptors (not Uniform)
Table 1 2nd line of table p19 has shift in columns (UCD1+ column should be empty)
    same on p21 for Char.<nop>FluxAxis.Calibration

    I would have written p19 ucd="arith.zp;time" for <nop>CoordSys.TimeFrame.Zero

    p23 Char.<nop>SpatialAxis.SamplingPrecision.SampleExtent has rather
    ucd="phys.angSize;instr.pixel"

    p23 the 3 ucds for filling factors should be stat.filling (not
    stat.fill). The last one has an extra "time" should be stat.filling;time

    p24 the mandatory Data.<nop>FluxAxis.value and Data.<nop>SpectralAxis.value utypes
    could have a ucd "see sec.4.1 (or table 3)" and "see sec. 4.2 (or table 2)"

    p26 the <nop>BinLow/High utypes can have ucds time.start;time.interval
    and time.end;time.interval
    

    In all table 1, utypes ending in .UCD or .ucd can have the UCD
    column set to "meta.ucd". The utypes in .unit can have UCD set
    to "meta.unit".

In  Table 2 on p27, Angstrom (twice) should be written in lower case "angstrom",
    following OGIP use, and as in Table 3.
    (same remark in section 7.2, multiple occurrences of Angstrom)

p32 "arith.ratio" is a UCD secondary word, so to be used as suffix not
prefix: should be prefixed -> should be suffixed

p34 in section 5.1.3, last paragraph: "the the" -> the

p41:    "of an dataset" -> of a dataset
   Last paragraph: pectral -> spectral


The JHU SDSS VOTable example uses bad units in some cases :
"A" is the symbol for Ampere, but means Angstrom : use "angstrom" instead
(11 occurrences, as "A" or "10**(-17) erg/cm**2/s/A").
"yr" must be used instead of "y" for year

Many UCDs used in the XML and VOTable examples are not valid or
inconsistent with the ones given in table 1. Here is a list of suggested
changes to improve the examples :

*In 7.2*

Extent ucd="pos.region.diameter" -> ucd="pos.angDistance;instr.fov"

<nop>BinLow ucd="stat.min;em.wavelength" -> ucd="em.wl;stat.min"

<nop>BinHigh ucd="stat.max;em.wavelength" -> ucd="em.wl;stat.max"

<nop>FluxAxis ucd="phot.flux.density;em.wavelength" -> ucd="phot.flux.density;em.wl"

<nop>SpectralFrame ucd="em.wavelength" -> ucd="em.wl"

<nop>GenericCoordFrame ucd="phot.fludens;em.wl" -> ucd="phot.flux.density;em.wl"

Publisher ucd="meta.organization;meta.curation" -> ucd="meta.curation"

<nop>PublisherID ucd="meta.curation.pubid" -> ucd="meta.ref.url;meta.curation"

Name ucd="meta.human;meta.curation" -> ucd="meta.bib.author;meta.curation"

<nop>DataID / Creator ucd="meta.curation.creator" -> ucd="meta.curation" 

Date ucd="time;soft.dataset;meta.curation" -> ucd="time.epoch;meta.dataset"

Version ucd="soft.dataset.version;meta.curation" -> ucd="meta.version;meta.dataset"

Instrument ucd="inst.id" -> ucd="meta.id;instr"

Logo ucd="meta.curation.logo" -> ucd="meta.ref.url"

*In 8.2*

PARAM name="System" utype="spec:CoordSys.SpaceFrame.Name" ucd="frame.pos.system" 
  -> ucd="pos.frame"

PARAM name="TimeFrame" utype="spec:CoordSys.TimeFrame.Name" ucd="frame.time.scale" 
  -> ucd="sdm:time.frame"

PARAM name="SpectralFrame" utype="spec:CoordSys.SpectralFrame.RefPos" ucd="frame.em.system"  -> ucd="sdm:spect.frame"

PARAM name="SkyExtent" utype="Char.SpatialAxis.Coverage.Extent" ucd="pos.region.diameter"
  -> ucd="pos.angDistance;instr.fov"

PARAM name="TimeObs" utype="Char.TimeAxis.Coverage.Location.Value" ucd="time.obs"
  -> ucd="time.epoch;obs"

PARAM name="TimeStart" utype="Char.TimeAxis.Coverage.Bounds.Start" ucd="time" 
  -> ucd="time.start"

PARAM name="TimeStop" utype="Char.TimeAxis.Coverage.Bounds.Stop" ucd="time"
  -> ucd="time.end"
(time.start and time.end can be used in both Coverage.Bounds and Coverage.Support)

PARAM name="Publisher" utype="spec:Curation.Publisher" ucd="meta.organization;meta.curation" -> ucd="meta.curation"

PARAM name="PubID" utype="spec:Curation.PublisherID" ucd="meta.curation.pubid"
   -> ucd="meta.ref.url;meta.curation"

PARAM name="Contact" utype="spec:Curation.Contact.Name" ucd="meta.human;meta.curation"
  -> ucd="meta.bib.author;meta.curation"

PARAM name="Creator" utype="spec:Segment.DataID.Creator" ucd="meta.curation.creator"
  -> ucd="meta.curation"

PARAM name="DataDate" utype="spec:DataID.Date" ucd="time;soft.dataset;meta.curation"
 -> ucd="time.epoch;meta.dataset"

PARAM name="Version" utype="spec:DataID.Version" ucd="soft.dataset.version;meta.curation"
  -> ucd="meta.version;meta.dataset"

PARAM name="Instrument" utype="spec:DataID.Instrument" ucd="inst.id" datatype="char"
  -> ucd="meta.id;instr"

PARAM name="Filter" utype="spec:DataID.Collection" ucd="inst.filter.id"
  -> ucd="instr.bandpass"

PARAM name="Logo" utype="spec:DataID.Logo" ucd="meta.curation.logo"
  -> ucd="meta.ref.url"

PARAM name="Resolution" utype="spec:Data.SpectralAxis.Resolution"
ucd="spect.resolution;em.wave" -> ucd="spect.resolution;em.wl"

FIELD name="Coord" ID="Coord" utype="spec:Data.SpectralAxis.Value" ucd="em.wavelength" -> ucd="em.wl"

FIELD name="BinLow" ID="BinLow" utype="spec:Data.SpectralAxis.BinLow"
ucd="stat.min;em.wavelength" -> ucd="em.wl;stat.min"

FIELD name="BinHigh" ID="BinHigh" utype="spec:Data.SpectralAxis.BinHigh"
ucd="stat.max;em.wavelength" -> ucd="em.wl;stat.max"

FIELD name="Flux" ID="Flux1" utype="spec:Data.FluxAxis.value" ucd="phot.flux.density;em.wavelength" -> ucd="phot.flux.density;em.wl"

*In JHU SDSS output* :

FIELD ID="DataSpectralBinLow" name="DataSpectralBinLow" datatype="double" ucd="em.wl" -> ucd="em.wl;stat.min"

FIELD ID="DataSpectralBinHigh" name="DataSpectralBinHigh" datatype="double" ucd="em.wl" -> ucd="em.wl;stat.max"

FIELD ID="DataFluxValue" name="DataFluxValue" datatype="double" ucd="phot.fluDens;em.wl" -> ucd="phot.flux.density;em.wl"

FIELD ID="DataFluxStatErrLow" name="DataFluxStatErrLow" datatype="double" ucd="phot.fluDens;em.wl" -> ucd="phot.flux.density;em.wl"

FIELD ID="DataFluxStatErrHigh" name="DataFluxStatErrHigh" datatype="double" ucd="phot.fluDens;em.wl" -> ucd="phot.flux.density;em.wl"

PARAM ID="TimeFrameName" datatype="char" name="TimeFrameName" ucd="time.scale" -> ucd="sdm:time.frame"

PARAM ID="SpectralLocation" datatype="double" name="SpectralLocation" ucd="em.wl;bandpass" -> ucd="instr.bandpass"

PARAM ID="SpectralStart" datatype="double" name="SpectralStart" ucd="em;stat.min" -> ucd="em.wl;stat.min"

PARAM ID="SpectralStop" datatype="double" name="SpectralStop" ucd="em;stat.max" -> ucd="em.wl;stat.max"

PARAM ID="SpectralBinSize" datatype="double" name="SpectralBinSize" ucd="em;spec.binSize" -> ucd="spect.binSize"

PARAM ID="TimeExtent" datatype="double" name="TimeExtent" ucd="time.expo" -> ucd="time.duration;obs.exposure"

PARAM ID="TimeStart" datatype="double" name="TimeStart" ucd="time.expo.start" -> ucd="time.start;obs.exposure"

PARAM ID="TimeStop" datatype="double" name="TimeStop" ucd="time.expo.end" -> ucd="time.end;obs.exposure"

PARAM ID="FluxAxisUcd" datatype="char" name="FluxAxisUcd" utype="spec:Spectrum.Char.FluxAxis.Ucd" value="phot.fluDens;em.wl" -> ucd="phot.flux.density;em.wl"

PARAM ID="DataFluxUcd" datatype="char" name="DataFluxUcd" utype="spec:Spectrum.Data.FluxAxis.Ucd" value="phot.fluDens;em.wl" -> ucd="phot.flux.density;em.wl"

Note that time.frame and spect.frame do not exist in the current list 
but have been requested and will likely be included in the next list.
Meanwhile, I added a sdm: namespace above, as is allowed by the UCD standard.

_SSAP 1.1 Approved with comments. SD 2011.10.07_


p23, the value given as unit for SPECRP is not really the unit,
  but an explanation of how the quantity is obtained (this is
  clear from 4.1.2.2, so could be removed from the table) 

p37 section 4.2.5.9, replace "phot.fluDens;em.wl" by "phot.flux.density;em.wl"

p52, in the VOTable snippet, the third attribute of the first param should be unit, not units.

p54, according to <nop>SpectralDM table 1 p19
   for param Publisher, ucd="meta.curation" 
   for <nop>PubID, ucd="meta.ref.url;meta.curation"
   for FIELD vturb, ucd="phys.veloc.microTurb" (also on p.56)

p55, ucd="VOX:Spectrum_units" can be replaced by ucd="meta.unit" (also on p.56)

p56, utype for FIELD <nop>FluxAxis is misspelled : change <nop>FluxAsix -> <nop>FluxAxis

p62, PARAM ID="DisplayRef" should have ucd="meta.ref.url", and there is
an extra ssa: in the utype

p63 ucd="time.expo" -> ucd="time.duration;obs.exposure" 
    ucd="time.expo.start" -> ucd="time.start;obs.exposure" 
    ucd="time.expo.end" -> ucd="time.end;obs.exposure" 

p66 stat.fill;em -> stat.filling;em

p67 time.stop;obs.exposure -> time.end;obs.exposure

    time;stat.fill;time -> stat.filling;time

The unit for logg, if expressed as per the OGIP convention, should be
unit="log(cm/s**2)" (p54, 56)

p59 for <nop>DatasetSize" unit="bytes" -> unit="byte"
    There is a missing opening bracket for one PARAM

p63 for <nop>SpaceFrameEquinox unit="y" -> unit="yr" ( unit should really be "yr" - 18 Oct 2011)

And minor remark, the & in the two TD of appendix A p57, and in the 
webtest URL p59 will make
an invalid XML without encoding...

-- 16 Oct 2011 IVOA.JesusSalgado
After discussion at TCG level these are the conclusions:

   * Impact on implementations has been considered. UCDs change do not look to break client applications. Servers will be published with a different versioning metadata. The main impact expected would be in validators
   * UCDs will be updated into the document plus all minor editorial comments suggested on the TCG review. This will done as soon as possible.
   * Major comments will be incorporated in !SpectrumDM 1.2

-- 2011-10-18 IVOA.PatrickDowler

The SSA-1.1 document has been updated to fix typos, units, ucds, and XML correctness issues.

---+++ VOEvent Working Group (<i>Matthew Graham, Roy Williams</i>)

I approve this document.
-- IVOA.MatthewGraham - 13 Oct 2011

---+++ Data Curation & Preservation Interest Group (<i>Alberto Accomazzi</i>)

---+++ Knowledge Discovery in Databases Interest Group (<i>Giuseppe Longo</i>)

---+++ Theory Interest Group (<i>Herve Wozniak, Franck Le Petit</i>)

Approved - IVOA.HerveWozniak 2011-07-04

---+++ Standards and Processes Committee (<i>Francoise Genova</i>)


<hr/>

<br/>
<!--
      * Set ALLOWTOPICRENAME = IVOA.TWikiAdminGroup
-->

   * [[%ATTACHURL%/PR-SSA-1.1-20111018.pdf][PR-SSA-1.1-20111018.pdf]]: PR-SSA-1.1-20111018
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
pdf	PR-SSA-1.1-20111018.pdf	r2 r1	manage	800.3 K	2011-10-19 - 06:10	DougTody	PR-SSA-1.1-20111018
pdf	PR-SpectrumDM-1.1-20111017.pdf	r1	manage	417.6 K	2011-10-20 - 03:37	OmarLaurino	PR-SpectrumDM-1.1-20111017
pdf	SpectrumDM-20111017.pdf	r1	manage	414.1 K	2011-10-19 - 07:48	OmarLaurino
Topic revision: r32 - 2011-10-20 - PatrickDowler
IVOA
Log in or Register
IVOA.net
Wiki Home
WebChanges
WebTopicList
WebStatistics
Twiki Meta & Help
IVOA
Know
Main
Sandbox
TWiki
TWiki intro
TWiki tutorial
User registration
Notify me
Working Groups
Interest Groups
Committees
Stds&Procs
www.ivoa.net
Documents
Events
Members
XML Schema