Spectrum Data Model RFCThis document will act as RFC centre for the Spectrum Data Model 1.01 Proposed Recommendation. Review period: 16 May 2007 to 12 Jun 2007 (still open) In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the data model mailing list, dm@ivoa.net.Comments
Concept My new proposal Your proposal Energy flux vs energy phot.flux.density;em.energy phot.flux.density;em.energy Number flux vs energy phot.flux.density;em.energy,phys.photon phot.flux.density;em.energy or phot.flux.number;em.energy Flux density (vs wl) phot.flux.density;em.wl phot.flux.density;em.wl Polarized flux (vs wl) phot.flux.density;em.wl,phys.polarization phot.flux.density;em.wl Flux density (vs wl) phot.flux.density;em.wl phot.flux.density;em.wl Source intensity phys.luminosity; em.wl,phys.angArea phot.flux.density;em.wl Flux density phot.flux.density phot.flux.density Brightness temperature phot.flux.density; phys.temperature phot.flux.density Aperture area phys.area phys.area Effective area phys.area;phys.transmission phys.area Continuum-only vs wl phot.flux.density;em.wl,spect.continuum phot.flux.density,spect.continuum Continuum-only vs freq phot.flux.density;em.freq,spect.continuum phot.flux.density,spect.continuumCan you (1) explain what is wrong with my proposals (ok the first one uses a so-far-nonexistent UCD I agree, the others seem legal to me) and (2) provide alternatives that do not lose these distinctions? I have a few other issues with your list vs mine: Why is src.var.amplitude;arith.ratio not acceptable for the variability as fraction of mean? To say that this should have the same UCD as the actual amplitude is losing valuable discriminatory power. What is wrong with meta.ucd and meta.unit for e.g. SpatialAxis.ucd, unit? With e.g. Spectrum.Char.SpectralAxis.Accuracy.StatError you suggest "stat.error;em" while I suggested "stat.error;em.wl" or "stat.error;em.freq" etc. Isn't it better to specify here, so that software can use the UCD to tell which is being given? Why don't you like: em.wl;obs.atmos for Air Wavelength? I would really like to be able to distinguish it from vacuum wavelength. How can I tell the client (user) that air wavelengths are being used? - thanks, Jonathan
* JonathanMcDowell -23 Jun 2007 Andrea, I guess I don't feel that phot.flux.density;em.wl,stat.max (and similar) are 'cumbersome' - I find them much easier to understand than extra custom-crafted UCDs like ..perWave. I feel that em.wl, em.freq, etc. are naturally adjectives as well as nouns; since they are 'Q' and not 'P', what I have done seems perfectly legal, and I guess I would like a reason based on the standard that it's not right, rather than just 'Andrea doesn't like it'... I accept the proposal to add stat.error.lower, stat.error.upper, em.wl.air as new UCDs. But I'm a bit concerned that you give me all this input now at/after the end of the RFC period, we were hoping to get approval for Spectrum at the July 15 exec - if we need to add all these new UCDs, does this mean we have to wait for a new update to the words list to be approved before Spectrum can go to approval? (Plus, of course, the time required for all the implementers to change their implementation, since we've published essentially the current list of Spectrum UCDs over a year ago and everyone is using them - this mostly is important for the Flux.Value cases). For polarized flux: yes, a data provider could choose to separate it out into (total flux vs wavelength) and (fraction of polarized flux vs wavelength), but I believe it's fairly common (certainly in AGN work) to show the product of these two things as the 'polarized flux', and I want a UCD for that. Since phys.polarization is an 'E' word, I believe that phot.flux.density;em.wl,phys.polarization is a reasonably unambiguous UCD for this? For em.*: in the context of the doc, this is a shorthand for (EITHER em.wl OR em.freq OR em.energy). - Jonathan * FrancoisBonnarel - 25 Jun 2007 Jonathan, Reviewing the document, I discover two little points which may be a problem: A) Page 8 the document claims that " In section 4 we elaborate these concepts in detail, including some complications that we explicitly do not attempt to handle in this version. The data model fields and possible values arelisted. We distinguish between optional and required fields in the text, as well as via a "Req" column in the tables which has values of R (Required) and O (Optional). Where appropriate we list those values of the physical units which interoperable implementations are required to recognize." But actually at page 13 and in the following table there is no more Required and Optional but "MANDATORY" abreviated as MAN and "OPTIONAL" abreviated as "OPT". In addition we have "REC" for recommanded. I think i remember the statement at page 8 was what was planned at the beginning... B ) Page 17 for the Spectrum.Char.*.Calibration utypes I think the meaning should be "coord calibration status" instead of Type of Coord Calibartion. Type can be interpreted as "Data Type" for calibration, no ? (Or is that due to French being my mother language?) François Bonnarel * DougTody - 8 Jul 2007 The first item is a typo (left over from an earlier version) and will be fixed in the next version. The reference to "section 4" is also incorrect. Regarding the "type of calibration", this could be expressed either way, but I think "type" is better than "status", so long as the allowable values are documented. * FrancoisBonnarel - 25 Jun 2007 Page 56 you write: "How can this be generalized to mapping an arbitrary data model schema to VOTABLE? The only tricky parts are " Spotting where the tabledata parts are. We could require any DM schema that maps to VOTABLE to include elements called FIELDS and Data (perhaps ROWS would be a better name), otherwise you would get a VOTABLE with no data section. " Spotting where to start the main TABLE (i.e. the fact that SPECTRUM is special). We could change the schema to have an explicit attribute, annotation or other marker to tell us this. These issues will require further discussion for future models." We started a discussion on that in Moscow... (yes Moscow, not Beijing !!!) I think the formulation here is a little too restrictive. We may imagine any elemnt with a simple content as a "FIELD" in some context, as long as you have more than one representative of this element. It is not the case here but what when we will get several segments in a main SED or Echelle spectrum ? Will we have each segment as a separate Spectrum serialisation and all this concatenated ? or redefine the PARAMS with several representatives as FIELDS... An index FIELD can be defined to hook information in the "metadata" TABLE to information in the data TABLE. This is the way we try to implement data models in the DAL Extension mechanism (see SSA 1.01 and references therin), and I think we need to let that open. So I will rephrase: "map all elements with simple content to PARAM" as "map all elements with simple content and one single representative to PARAM" and add a third "spotting". "Spotting if and where we need secondary TABLES with Metadata FIELDS and how we hook the information therin to the main TABLE rows" François Bonnarel | ||||||||
Added: | ||||||||
> > | * DougTody - 8 Jul 2007 None of the analysis in 8.1 is actually necessary for SSA V1.0 as the VOTable serialization is explicitly defined, hence we don't require a general scheme for mapping arbitrary data models to VOTable. It is not necessary to resolve this now; as the text says "these issues will require further discussion for future models". That said, my recollection from the old SED model is that each segment mapped to a separate TABLE element. Within each TABLE, one can have both PARAM and FIELD elements, depending upon whether the element has multiple values. What is most needed for a general mapping from a data model to VOTable is probably a way to identify what is a TABLE in the VOTable representation. Within a TABLE, PARAM and FIELD are the same except that FIELD is used if the element has multiple instances. Outside TABLE, only PARAM is used. We should revisit this later when we consider how to handle multi-segment spectra or time series, and SEDs. | |||||||
* FelixStoehr - 02 Jul 2007 Dear all, Currently, the SpectrumDM states under section 5.3 Derived Data fields, Signal-to-noise: "A suitable method, used by the STScI MAST group, is to define the noise by the median absolute value of the difference between adjacent independent flux values in the spectrum. (The MAST definition multiplies this noise value by a 1.048 correction factor for precise applications)." The STScI(MAST)/ST-ECF/CADC Spectral Container Working Group has worked quite a bit on the algorithm and tested several possibilities. The final "best" algorithm is presented in the issue #42 of the ST-ECF Newsletter http://www.spacetelescope.org/about/further_information/newsletters/html/newsletter_42.html Implementations of the new algorithm can be obtained from: http://stecf.org/software/ASTROsoft/DER_SNR/ As a suggestion, we propose to change the sentences cited above by: "A suitable method, set forth by the STScI/ST-ECF/CADC Spectral Container Working Group, is to define the signal as the median of the flux values in the spectrum and the noise as the median absolute third-order difference of flux values spaced two pixels apart. This noise value is then multiplied by 1.482602 / sqrt(6). Padded zeros in the flux values are skipped. A detailed description and discussion of the algorithm can be found in the issue #42 of the ST-ECF newsletter at www.spacetelescope.org/about/further_information/newsletters/html/newsletter_42.html. Implementations of the algorithm can be obtained from www.stecf.org/software/ASTROsoft/DER_SNR. We recommend to use this algorithm where suitable to allow meaningful cross-instrument searches." Felix Stoehr | ||||||||
Changed: | ||||||||
< < | Adding a reference to a recommended method for computing comparable SNRs is an excellent idea. | |||||||
> > | Adding a reference to a recommended method for computing comparable SNRs is an excellent idea. - MarkusDolensky | |||||||
Changed: | ||||||||
< < | MarkusDolensky | |||||||
> > | For SNR, the essential point is that this value represents an overall SNR for the entire spectrum; the SNR of individual spectral features will | |||||||
Added: | ||||||||
> > | vary of course. There are various ways in which this could be computed. I am not sure we are ready yet to standardize on a method for computing the overall SNR, but it is reasonable to suggest a technique such as that mentioned above, so long as we have a good reference to some external paper describing the technique. - DougTody | |||||||
Comments from the Working Group Chairs and Interest Group ChairsChairs should add their comments under their name.Mark Allen (Applications WG)Approved The Spectrum Data Model appears to contain all the pieces necessary to fully describe spectra as is needed in the VO. The examples are especially useful as they will help guide usage in services and applications.Christophe Arviset (TCG vice Chair)Matthew Graham (Grid & Web Services WG)Bob Hanisch (Data Curation & Preservation IG)Gerard Lemson (Theory IG)Mireille Louys (Data Models WG)Keith Noddle (Data Access Layer WG)Francois Ochsenbein (VOTable WG)Pedro Osuna (VOQL WG)Ray Plante (Resource Registry WG)Andrea Preite-Martinez (Semantics WG)I approve the document for Recommendation. My comments are expressed above.Roy Williams (VOEvent WG)This is a well-written, well-thought out definition of a spectrum, and the authors are to be congratulated. I approve it for Recommendation. The examples may be all that a reader sees. They may read NONE of the text! Therefore these are important. After looking carefully, I find only two things worthy of note: -- In the Characterisation section of example 7.2, the flux units are difficult to see, it says name="Flux density" ucd="phot.flux;em.wavelength" unit="erg cm**(-2) s**(-1) Angstrom**(-1)", and this is still not enough for me to see which kind of flux you mean. Perhaps a comment from Table 3 would help, it explains simply: "Flux density per unit wave", and now I am clear. -- In the VOTable example 8.2, there are no schema locations for either VOTable or SpectrumModel. This means that simply copying the example does not allow XML validation.<--
|