Difference: SpectrumDataModelRFC (11 vs. 12)

Spectrum Data Model RFC

This document will act as RFC centre for the Spectrum Data Model 1.01 Proposed Recommendation.

Review period: 16 May 2007 to 12 Jun 2007 (still open)

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Discussion about any of the comments or responses should be conducted on the data model mailing list, dm@ivoa.net.

Comments

• First sample comment (by BrunoRino): ...
  • Response (by authorname): ...

• AndreaPreiteMartinez - 19 Jun 2007
  • I have revised the UCDs in Tables 1, 2 and 3. The revised tables can be found here. Note that rows with changed UCDs are in bold+italic.
  • Pag.23: erase sentence "My solution for brightness... "
  • Pag.23 last line: change with "Note: we propose for the next version of the IVOA REC UCD-list to (a) change the flag of pos.eq from S to Q, (b) add two new words em.start, em.end to indicate the lower/start and upper/end boundaries of a spectral interval (as already done for the time axis)"

• JonathanMcDowell -20 Jun 2007

Andrea,

I don't like the fact that you propose the same UCD for energy flux density and photon number flux density, and the same UCD for polarized flux and total flux, and the same UCD for observed flux and for source intensity per unit emission solid angle. These are different physical concepts, in some cases they have the same units and I REALLY need different UCDs to describe them to users - this is exactly what UCDs are for. I want to avoid using units to distinguish since this is of limited reliability ( e.g. source intensity as described below has the same units as a surface brightness but is a very different thing) and since it's a difference in CONCEPT not a difference in UNIT that is key here.

Concept					 My new proposal								  Your proposal

Energy flux vs energy  phot.flux.density;em.energy					phot.flux.density;em.energy
Number flux vs energy  phot.flux.density;em.energy,phys.photon	phot.flux.density;em.energy
						  or phot.flux.number;em.energy

Flux density (vs wl)	phot.flux.density;em.wl						 phot.flux.density;em.wl
Polarized flux (vs wl) phot.flux.density;em.wl,phys.polarization phot.flux.density;em.wl

Flux density (vs wl)	phot.flux.density;em.wl						 phot.flux.density;em.wl
Source intensity		 phys.luminosity; em.wl,phys.angArea		 phot.flux.density;em.wl

Flux density			  phot.flux.density								 phot.flux.density
Brightness temperature phot.flux.density; phys.temperature		 phot.flux.density

Aperture area			 phys.area											phys.area
Effective area			phys.area;phys.transmission					phys.area

Continuum-only vs wl	phot.flux.density;em.wl,spect.continuum	phot.flux.density,spect.continuum
Continuum-only vs freq phot.flux.density;em.freq,spect.continuum phot.flux.density,spect.continuum

Can you (1) explain what is wrong with my proposals (ok the first one uses a so-far-nonexistent UCD I agree, the others seem legal to me) and (2) provide alternatives that do not lose these distinctions?

I have a few other issues with your list vs mine:

Why is src.var.amplitude;arith.ratio not acceptable for the variability as fraction of mean? To say that this should have the same UCD as the actual amplitude is losing valuable discriminatory power.

What is wrong with meta.ucd and meta.unit for e.g. SpatialAxis.ucd, unit?

With e.g. StatError you suggest "stat.error;em" while I suggested "stat.error;em.wl" or "stat.error;em.freq" etc. Isn't it better to specify here, so that software can use the UCD to tell which is being given?

Why don't you like: em.wl;obs.atmos for Air Wavelength? I would really like to be able to distinguish it from vacuum wavelength. How can I tell the client (user) that air wavelengths are being used?

- thanks, Jonathan

• GaryFuller - 19 Jun 2007
  • Does the model allow for double sideband spectra? That is spectra for which each pixel can correspond to two different frequencies or wavelengths? This is a fairly common situation in the mm and submm.
  • More generally what about the issue of describing how to convert between two different spectral axes, for example in the dsb case the upper and lower sideband frequency?
  • Is there support for non-uniform spectral (and temporal) axes?
  • In derived data fields signal-to-noise ratio seems a poorly defined quantity as it depends on what signal, spectral feature, is of interest to a particular user. A value for the rms noise level on the other hand would be very useful but doesn't seem to be in the model.

• JonathanMcDowell -20 Jun 2007

Gary:

We discussed the double sideband spectrum case but decided that it complicated the model too much for version 1. We'll revisit this in future, and for now you just have to pick one sideband as the nominal frequency, maybe providing two Spectrum instances, one for each sideband. We acknowledge this is a limitation, as is the inability to represent two different spectra axes in a single instance.

The S/N is a bit vague, but was requested by some data providers for who a single target object is the norm. There is no rms noise, but there is a 'typical (1 sigma) error' in StatError which does much the same thing. We may add rms in a later rev but not this one.

We do, however, support non-uniform axes, since each spectral coordinate is listed explicitly.

• AndreaPreiteMartinez - 22 Jun 2007

Jonathan,

1) first of all, a general consideration: if we miss an ucd-word for expressing a given quantity, then let’s create the corresponding new ucd-word.

2) as an alternative, we can agree that e.g. phot.flux.density;em.wl (or freq or energy) indicates the flux density per unit wavelength (or frequency, or energy) : no syntax problem (valid syntax ), but cumbersome if you want to add other modifiers/qualifiers (em.opt, stat.max,...). What I do not like in the form phot.flux.density;em.wl is the use of a quantity (em.wl) as a modifier/qualifier of the first quantity (phot.flux.density). In this case I prefer going to point (1) above, and re-present the proposal (see Victoria 2006) of adding:

phot.flux.density.perFreq, phot.flux.density.perWave, phot.flux.density.perEnergy, .. phot.flux.photon.per*, .. phot.flux.brTemperature, .. phot.intensity.perWave ..

So, phot.flux.density;em.wl;spect.continuum will become phot.flux.density.perWave;spect.continuum

About Polarized Flux: I don’t understand the problem. I would have added an additional attribute for the degree of polarization or the fraction of polarized flux.

the same concern applies to phys.area;phys.transmission: let’s go to phys.area.effective

src.amplitude;arith.ratio : you are right

SpatialAxis.ucd, unit: I took out meta.ucd, meta.unit for uniformity with the other axes. You can replace them, adding them also to TimeAxis .

stat.error;em : yes, if a * is legal in your context, you can write stat.error;em* (but not meaning all the em-word branch! )

em.wl;obs.atmos: I prefer em.wl.air.

3) in order to avoid confusion in multi-word ucds, I suggest also to introduce

stat.error.lower instead of stat.error;stat.min

stat.error.upper instead of stat.error;stat.max

Ciao, Andrea

Andrea,

I guess I don't feel that phot.flux.density;em.wl,stat.max (and similar) are 'cumbersome' - I find them much easier to understand than extra custom-crafted UCDs like ..perWave. I feel that em.wl, em.freq, etc. are naturally adjectives as well as nouns; since they are 'Q' and not 'P', what I have done seems perfectly legal, and I guess I would like a reason based on the standard that it's not right, rather than just 'Andrea doesn't like it'...

I accept the proposal to add stat.error.lower, stat.error.upper, em.wl.air as new UCDs. But I'm a bit concerned that you give me all this input now at/after the end of the RFC period, we were hoping to get approval for Spectrum at the July 15 exec - if we need to add all these new UCDs, does this mean we have to wait for a new update to the words list to be approved before Spectrum can go to approval? (Plus, of course, the time required for all the implementers to change their implementation, since we've published essentially the current list of Spectrum UCDs over a year ago and everyone is using them - this mostly is important for the Flux.Value cases).

For polarized flux: yes, a data provider could choose to separate it out into (total flux vs wavelength) and (fraction of polarized flux vs wavelength), but I believe it's fairly common (certainly in AGN work) to show the product of these two things as the 'polarized flux', and I want a UCD for that. Since phys.polarization is an 'E' word, I believe that phot.flux.density;em.wl,phys.polarization is a reasonably unambiguous UCD for this?

For em.*: in the context of the doc, this is a shorthand for (EITHER em.wl OR em.freq OR em.energy). - Jonathan

Jonathan, Reviewing the document, I discover two little points which may be a problem:

A) Page 8 the document claims that " In section 4 we elaborate these concepts in detail, including some complications that we explicitly do not attempt to handle in this version. The data model fields and possible values arelisted. We distinguish between optional and required fields in the text, as well as via a ”Req” column in the tables which has values of R (Required) and O (Optional). Where appropriate we list those values of the physical units which interoperable implementations are required to recognize." But actually at page 13 and in the following table there is no more Required and Optional but "MANDATORY" abreviated as MAN and "OPTIONAL" abreviated as "OPT". In addition we have "REC" for recommanded. I think i remember the statement at page 8 was what was planned at the beginning...

B ) Page 17 for the Spectrum.Char.*.Calibration utypes I think the meaning should be "coord calibration status" instead of Type of Coord Calibartion. Type can be interpreted as "Data Type" for calibration, no ? (Or is that due to French being my mother language?)

François Bonnarel

Page 56 you write:

"How can this be generalized to mapping an arbitrary data model schema to VOTABLE? The only tricky parts are • Spotting where the tabledata parts are. We could require any DM schema that maps to VOTABLE to include elements called FIELDS and Data (perhaps ROWS would be a better name), otherwise you would get a VOTABLE with no data section. • Spotting where to start the main TABLE (i.e. the fact that SPECTRUM is special). We could change the schema to have an explicit attribute, annotation or other marker to tell us this. These issues will require further discussion for future models."

We started a discussion on that in Moscow... (yes Moscow, not Beijing !!!)

I think the formulation here is a little too restrictive. We may imagine any elemnt with a simple content as a "FIELD" in some context, as long as you have more than one representative of this element. It is not the case here but what when we will get several segments in a main SED or Echelle spectrum ? Will we have each segment as a separate Spectrum serialisation and all this concatenated ? or redefine the PARAMS with several representatives as FIELDS... An index FIELD can be defined to hook information in the "metadata" TABLE to information in the data TABLE. This is the way we try to implement data models in the DAL Extension mechanism (see SSA 1.01 and references therin), and I think we need to let that open. So I will rephrase: "map all elements with simple content to PARAM" as "map all elements with simple content and one single representative to PARAM"

and add a third "spotting".

"Spotting if and where we need secondary TABLES with Metadata FIELDS and how we hook the information therin to the main TABLE rows"

François Bonnarel

Comments from the Working Group Chairs and Interest Group Chairs

Chairs should add their comments under their name.

Marc Allen (Applications WG)

Christophe Arviset (TCG vice Chair)

Matthew Graham (Grid & Web Services WG)

Bob Hanisch (Data Curation & Preservation IG)

Gerard Lemson (Theory IG)

Mireille Louys (Data Models WG)

Keith Noddle (Data Access Layer WG)

Francois Ochsenbein (VOTable WG)

Pedro Osuna (VOQL WG)

Ray Plante (Resource Registry WG)

Roy Williams (VOEvent WG)

This is a well-written, well-thought out definition of a spectrum, and the authors are to be congratulated. I approve it for Recommendation.

The examples may be all that a reader sees. They may read NONE of the text! Therefore these are important. After looking carefully, I find only two things worthy of note:

-- In the Characterisation section of example 7.2, the flux units are difficult to see, it says name="Flux density" ucd="phot.flux;em.wavelength" unit="erg cm**(-2) s**(-1) Angstrom**(-1)", and this is still not enough for me to see which kind of flux you mean. Perhaps a comment from Table 3 would help, it explains simply: "Flux density per unit wave", and now I am clear.

-- In the VOTable example 8.2, there are no schema locations for either VOTable or SpectrumModel. This means that simply copying the example does not allow XML validation.

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->