Observation Data model Core Components and ObsTAP implementation:

+ Request for Comments

This wiki page document will act as RFC center for the Proposed Recommendation entitled " Observation Data Model Core Components and its Implementation in the Table Access Protocol, version 1.0 ".

The specification can be found below as attached files and on the IVOA Document page: http://www.ivoa.net/Documents/index.html

+ Review period: 03 May 2011 to 03 June 2011*

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the Data Model and Dal mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.

Comments from the IVOA Community during RFC period: 03 May 2011 to 03 June 2011*

An Implementation Note is currently in preparation to list and illustrate how an ObsTAP service can be set up based on the current ObsCore/TAP specification described in this proposed specification .

-- MireilleLouys - 04 May 2011

Thank for adding your comments below...


(1) In Figure 1, shouldn't SimpleDALRegExt rather be TAPRegExt?

-- Response: MireilleLouys - 04 July 2011 The figure has been updated with TAPRegExt highlighted in red.

(2) For Section 5, I'd suggest the following wording:

The standard URI of ObsCore is ivo://ivoa.net/std/ObsCore-1.0. Thus, to declare that a TAP service implements the ObsCore model, include

  <dataModel ivo-id="ivo://ivoa.net/std/ObsCore-1.0">ObsCore 1.0</dataModel>
  

in the service's capabilities element as included in the registry record and available on the service's capabilities endpoint. The details are discussed in [TAPRegExt].

To accommodate naive user queries, it is recommended that TAP services implementing the ObsCore model give ``ObsCore'' as a subject in the registry record.

Rationale: TAP services must be registered anyway (so "should be registered" is in conflict with the TAP REC). The ObsCore "keyword" is a nice and lightweight idea, but in registry language "keyword" is "subject".

-- Response: MireilleLouys - 04 July 2011 This has been partly included in the updated Section 5 thanks to pat Dowler.

(3) For Section 6 -- in case it matters for the note, I have ObsCore support in my data center software DaCHS, and it's live on http://dc.g-vo.org/tap. For the record, I think reference implementations (and, if at all possible, validators) should be part of the REC rather than being deferred to a note.

-- MarkusDemleitner - 06 May 2011

-- Response: MireilleLouys - 04 July 2011 Due to the long specification and discussion of the ObsCore Data model, implementation examples will be described in a separate note.


Comment: Please make the pol_states keyword mandatory

Reason: There are four types of information an electro-magnetic wave can be described with but only three of them are mapped into ObsCore. Section B.6.6. already describes how polarization can be mapped in detail. In my opinion this information should be mandatory to (1) make the description of the measurements complete and to (2) render all ObsTAP services symmetric.

It is important to note, that this request will not put any additional burden on service providers of standard imaging or spectral products as for these the pol_states value will always be constant: "I". In any reference implementation this could be the default value resulting thus in no additional work whatsoever for these providers.

-- FelixStoehr 2011, May 23

----------------------------------------------------------------------

-- Response MireilleLouys 2011,May 30

Whether a data set contains or nor polarised data is expressed in the mandatory field : o_ucd . If this string field contains 'polarization' then we know polarised data is present. If not , then it is not necessary to consider this data set in a search for polarised data. By this way , we think we use less metadata to describe observations recording one type of flux . pol_states is useful in the specific use-cases where we search which kind of polarisation is present after o_ucd has been checked.

-- Response : complements MireilleLouys 2011, June 08

This has been discussed during the telco on Monday June 06. See the minutes attached below. It was agreed to have pol_states mandatory, with NULL value when not applicable. The combination of constraints on o_ucd and pol_states in an ADQL query requires both column names to be defined in some data base systems. Therefore pol_states needs to be defined and then to be mandatory.


Comment: typos and clarification on cresolution

1) table 6 typos: for em_min and em_max utypes I assume the last terms should be LoLim,!HiLim, not LoLlim and HiLlim.

2) The s_resolution is listed as "Char.SpatialAxis.Resolution.refval.cresolution"

What does the "cresolution" refer to? Also the resolving power is "Char.SpectralAxis.Resolution.resolpower.refval".

Should it be refval.resolpower to be consistent?

-- Randy Thompson 2011, Jun 02

-- Response: MireilleLouys - 04 July 2011

Typos on LoLim, HiLim corrected. CResolution is an STC substitution type and has been removed. The correct Utype now is "Char.SpatialAxis.Resolution.refval"


A lot of typos found: I am referring to pdf version so some objections given below may be valid only for this - but it has to be checked anyway

1. Introduction last sentence - the global data discoverability and accessibility is in very strange hand-written font. Similar but different font in Section 2, 2nd sentence

4.12 4th paragraph 1st sentence - redundant "the" in "of the observed the region"

Sec.5 at the end [REF] - probably the reference is expected

Sec. 6. - comment [add url] twice - should be filled. The question mark to be removed.

Appendix A.1.2 Use case 1.2 What does it mean LIST=SERVICE REQ and AND=SERVREQ Explanation needed.

In addition in use case III - bad range 5000-9000 (missing zero)

A.1.3 in listing:

3rd line - AND s_resolution< 0.3 (missing underscore)

A.1.4. III Spatial Resolution < 0.7 arcseconds (double dot instead of period 0:7) A1.5 III - again strange comment SERVIC REQ + NEEDS ANOTHER SERVICE (CATALOGUE) and missing right parenthesis

A1.6. LIST=SERVREQ

and in last line - is the absolute value really called by vertical line in TAP query ?

A.2.3

SERVICEREQ+NEEDS ANOTHER SERVICE

A.3.3 listing AND t_exptime> 3600 ? (redundant question mark)

A.5.2 listing last line s_region (missing underscore)

A.6.2 sentence For the quasars, give me high resolution (<0.5") the double dot in 0:5

In general - in appendix the capitalizaed comments like SERVICE REQ are cryptic and inconsistently written (although I understand they were copied from some notes - however document like ObsCore should be precise even here ... (the missing underscores may confuse people i f they try to call service using the cut and paste (in the future).

B.2.1 in sentence "So the users could be warned" (it is only "be warn")

B.4.1 The last sentence begins in different font (The same dataset - it is smaller - is it an intention ?

B.4.4 - defines rights "proprietary" but in B4.5 last sentence An observation with a NULL value in the releaseDate attribute is private by definition.

Is it meant the "proprietary" as a reference to B.4.4 rights ??

B.6.1.1. last sentence strange reference to STC verbosely "CITATION STC \I 1036" - probably some remnant of citation tool

B.6.2 last sentence - again strange citation remnant

B.6.2.3. after the case a) (FWHM of the LSF) -- is it the Line Spread Function ? (probably not so commonly known abbreviation to anybody ...)

after case b) ... resolution power stored as - there is period power.stored

after case c) Char.spectralAxis.Resolution.resolPower LoLim seems to be the white space before !.LoLim

B.6.3.1 strange reference

B.6.5.1 ... more complex combinations such as: phot.flux.density;phys.polarization.stokes.I

is it correct the stokes.I ? (or Stokes.I ?)

last sentence should be probably parentheses (See examples in the table)

B.6.5.2 strange citation

B6.6.

2nd paragraph phot.flux.density;phys.polarisation.Stokes.I (written Stockes) and missing period - is it OK ?

Tables 6 and 7: What does mean principal, Index and Std ? The index TBD - or zero - - what is a meaning of it? and what sometimes there are question mark ? It is a standard and everything should be clear to the reader wink

In addition to the typos and clarification needed: There is a lot of places referring to SDM or SSAP in obsolete versions. I suppose the SSA 1.0 et least (or 1.1.) should be referred to (in 3.3, B.4.3, B.5, B.6.5.2, ) and SDM probably the 1.1 (in 3., B.3.7, B.6.2, )

One inconsistency in table 5 and definition in B.6.2.1

there is no the em_calib_status in the table, but in B.6.2.1 it is defined. as {calibrated,uncalibrated,relative and normalized}.

I think it should be the level of wavelength calibration (not flux) so the usage of "normalized" is questionable (in SSA we have normalized only for fluxcalib). But it would break the symmetry in DM (same calibration for all axes)

However, I think, the introduction of observable axis o_calib_status in B.6.5.2 is the correct place but it is {absolute,relative,normalized, any} B.6.5.2. refers to SSA (it is a value of FLUXCALIB query parameter)

So in query I can use "any" abut if it is describing characterization (data model ) - it is not directly corresponding to query, however absolute, relative is a both state and query ....

I feel the columns from ObsCore should be mainly used for queries - so the SSA value of any is correct then the SDM.

I think the whole em_calib_status should be referring to SSA query params .... and the value of "normalized " kept for symmetry in SDM (maybe I can normalize the wavelength as well to get relative value of distance from given line in e.g. percents...)

The same inconsistency in table 5 and B.6.1.4. the s_calib_status is defined {uncalibrated,raw, calibrated} but in table is NOT CALIBRATED, FINE COARSE....

in t_calib_status (tab 5) is then written "Type of coord calibration" = not defined values and in B.6.3.3 is not defined anything - what values can have time calibration level ?????

I am confused about this - is it a inconsistency between SDM, SSA and CharDM or I do not recognize something important?

-- PetrSkoda 2011, Jun 03

-- Response: MireilleLouys - 04 July 2011 Most typos corrected in the last version.Updates for values of calibStatus, modified for each axis.


There does not appear to be any space for comments during the normal RFC period by the working groups. This represents the applications take.

While I don't see any major issues with the proposal technically, the proposal seems to contain a lot of ancillary justification that isn't necessary for the standard and which could get in the way of someone trying to read it with the goal of implementing it. A lengthy discussion of the justification for the standard, use cases, implementations and such does not belong in the standard itself (though they may be needed in the RFC process).

My one technical concern is that the inclusion of the region field implies some capability of handling ADQL SQL extensions substantially beyond what a minimal TAP service need support, but this is mitigated by this field being nullable.

-- TomMcGlynn June 13, 2011

Comments from TCG member during the TCG Review Period: 07-July-2011 - 07-August-2011

WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair (Christophe Arviset, Séverin Gaudet)

Applications Working Group (Tom Mcglynn, Mark Taylor)

Data Access Layer Working Group (Patrick Dowler, Mike Fitzpatrick)

Data Model Working Group (Jesus Salgado, Omar Laurino)

Grid & Web Services Working Group (Andreas Wicenec, Andre Schaaff )

Registry Working Group (Gretchen Greene, Pierre Le Sidaner)

Semantics Working Group (Sebastien Derriere, Norman Gray)

VOEvent Working Group (Matthew Graham, Roy Williams)

Data Curation & Preservation Interest Group (Alberto Accomazzi)

Knowledge Discovery in Databases Interest Group (Giuseppe Longo)

Theory Interest Group (Herve Wozniak, Franck Le Petit)

Standards and Processes Committee (Francoise Genova)




Topic attachments
I Attachment History Action Size Date Who Comment
Unknown file formatdocx PR-ObsCore-v1.0-20110502.docx r1 manage 323.5 K 2011-05-03 - 10:49 MireilleLouys ObscoreDM Proposed recommendation
PDFpdf PR-ObsCore-v1.0-20110502.pdf r1 manage 1886.5 K 2011-05-03 - 10:50 MireilleLouys ObscoreDM Proposed recommendation (pdf)
Unknown file formatdocx PR-ObsCore-v1.0-20110712.docx r1 manage 342.1 K 2011-07-12 - 21:56 MireilleLouys last version of proposed recommendation
PDFpdf teleconJune06.pdf r2 r1 manage 180.7 K 2011-06-08 - 12:59 MireilleLouys Telecon June 06 :last comments on ObsCore updated
Edit | Attach | Watch | Print version | History: r54 | r29 < r28 < r27 < r26 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r27 - 2011-07-12 - MireilleLouys
 
This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback