ObsCoreRFC < IVOA

---+ Observation Data model Core Components and !ObsTAP implementation:  

   + Request for Comments

This wiki page document will act as RFC center for the 
Proposed Recommendation entitled " *Observation Data Model Core Components and its Implementation in the Table Access Protocol, version 1.0* ". 
 
The specification can be found below as attached files and on the IVOA Document page:
http://www.ivoa.net/Documents/index.html

   + Review period:  03 May 2011 to %RED%03 June 2011*%ENDCOLOR% %BR% 

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your <nop>WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the Data Model and Dal mailing list.
  However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.

---++ Comments from the IVOA Community during RFC period: 03 May 2011 to %RED%03 June 2011*%ENDCOLOR% %BR% 

An Implementation Note is currently in preparation to list and illustrate how an !ObsTAP service can be set up based on the current !ObsCore/TAP specification described in this proposed specification .

-- IVOA.MireilleLouys - 04 May 2011

Thank for adding your comments below... 

-----------

(1) In Figure 1, shouldn't SimpleDALRegExt rather be TAPRegExt?


-- Response: IVOA.MireilleLouys - 04 July 2011
_The figure has been updated with TAPRegExt highlighted in red._

(2) For Section 5, I'd suggest the following wording:
<blockquote>
  The standard URI of !ObsCore is ivo://ivoa.net/std/ObsCore-1.0.  Thus,
  to declare that a TAP service implements the !ObsCore model, include

  <verbatim>
  <dataModel ivo-id="ivo://ivoa.net/std/ObsCore-1.0">ObsCore 1.0</dataModel>
  </verbatim>

  in the service's capabilities element as included in the registry
  record and available on the service's capabilities endpoint.  The
  details are discussed in [TAPRegExt].

  To accommodate naive user queries, it is recommended that TAP services
  implementing the !ObsCore model give ``ObsCore'' as a subject in the
  registry record.  
</blockquote>

Rationale: TAP services must be registered anyway (so "should be registered" is in conflict with the TAP REC).  The !ObsCore "keyword" is a nice and lightweight idea, but in registry language "keyword" is "subject".  

-- Response: IVOA.MireilleLouys - 04 July 2011
_This has been partly included in the updated Section 5 thanks to pat Dowler._

(3) For Section 6 -- in case it matters for the note, I have !ObsCore support in my data center software [[http://soft.g-vo.org/dachs][DaCHS]], and it's live on http://dc.g-vo.org/tap.  For the record, I think reference implementations (and, if at all possible, validators) should be part of the REC rather than being deferred to a note.

-- IVOA.MarkusDemleitner - 06 May 2011

-- Response: IVOA.MireilleLouys - 04 July 2011
_Due to the long specification and discussion of the !ObsCore Data model, implementation examples will be described in a separate note._
----------------------------------------------------------------------
Comment: Please make the pol_states keyword mandatory

Reason: There are four types of information an electro-magnetic wave can
be described with but only three of them are mapped into !ObsCore. Section B.6.6. already describes how polarization can be mapped in detail. In my opinion this information should be mandatory to 
   (1) make the description of the measurements complete and to 
   (2) render all !ObsTAP services symmetric.

It is important to note, that this request will *not* put any additional burden on service providers of standard imaging or spectral products as for these the pol_states value will always be constant: "I". In any reference implementation this could be the default value resulting thus in no additional work whatsoever for these providers.

-- FelixStoehr 2011, May 23

%BLUE%----------------------------------------------------------------------%ENDCOLOR%

-- Response MireilleLouys 2011,May 30

Whether a data set contains or nor polarised data is expressed in the mandatory field : _o_ucd_ .
If this string field contains 'polarization' then we know polarised data is present.
If not , then it is not necessary to consider this data set in a search for polarised data.
By this way , we think we use less metadata to describe observations recording one type of flux .
_pol_states_ is useful in the specific use-cases where we search which kind of polarisation is present after __o_ucd__ has been checked.

-- Response : complements MireilleLouys 2011, June 08

_This has been discussed during the telco on Monday June 06. See the minutes attached below. It was agreed to have __pol_states__ mandatory, with NULL value when not applicable. The combination of constraints on __o_ucd__ and __pol_states__ in an ADQL query requires both column names to be defined in some data base systems. Therefore_  __pol_states__ _needs to be defined and then to be mandatory._

----------------------------------------------------------------------
Comment: typos and clarification on cresolution

1) table 6 typos: for em_min and em_max utypes I assume the last terms
should be !LoLim,!HiLim, not !LoLlim and !HiLlim.


2) The s_resolution is listed as "Char.SpatialAxis.Resolution.refval.cresolution"

What does the "cresolution" refer to?  
Also the
resolving power is "Char.SpectralAxis.Resolution.resolpower.refval". 

Should it be refval.resolpower to be consistent?

-- Randy Thompson 2011, Jun 02

-- Response: IVOA.MireilleLouys - 04 July 2011

_Typos on !LoLim, !HiLim corrected. CResolution is an STC substitution type and has been removed. The correct Utype now is "Char.SpatialAxis.Resolution.refval"_ 

----------------------------------------------------------------------
A lot of typos found:
I am referring to pdf version so some objections given below may be valid only for this - but it has to be checked anyway

1. Introduction last sentence - the global data discoverability and accessibility is in very strange hand-written font. Similar but different font in Section 2, 2nd sentence

4.12 4th paragraph 1st sentence - redundant "the" in "of the observed the region"

Sec.5 at the end [REF] - probably the reference is expected 

Sec. 6. - comment [add url] twice - should be filled. The question mark to be removed.

Appendix A.1.2 Use case 1.2
What does it mean LIST=SERVICE REQ and AND=SERVREQ 
Explanation needed.

In addition in use case III - bad range 5000-9000 (missing zero)

A.1.3 in listing:

3rd line - AND s_resolution< 0.3 (missing underscore)

A.1.4. 
III  Spatial Resolution < 0.7 arcseconds   (double dot instead of period 0:7)
A1.5 
III - again strange comment SERVIC REQ + NEEDS ANOTHER SERVICE (CATALOGUE)  and missing right parenthesis 

A1.6. 
LIST=SERVREQ

and in last line - is the absolute value really called by vertical line in TAP query ?

A.2.3

SERVICEREQ+NEEDS ANOTHER SERVICE

A.3.3 listing 
AND t_exptime> 3600 ?    (redundant question mark)

A.5.2  listing last line
s_region (missing underscore)

A.6.2  sentence For the quasars, give me high resolution (<0.5") 
the double dot in 0:5

In general - in appendix the capitalizaed comments like SERVICE REQ are cryptic and inconsistently written (although I understand they were copied from some notes - however document like !ObsCore should be 
precise even here ... (the missing underscores may confuse people i f they try to call service using the cut and paste (in the future).

B.2.1 in sentence "So the users could be warned"
(it is only "be warn")

B.4.1  The last sentence begins in different font (The same dataset - it is smaller - is it an intention ?

B.4.4 - defines rights "proprietary"  but in B4.5 last sentence
An observation with a NULL value in the !releaseDate attribute is private by definition.

Is it meant the "proprietary" as a reference to B.4.4 rights ??

B.6.1.1. last sentence   strange reference to STC 
verbosely "CITATION STC \I 1036" - probably some remnant of citation tool


B.6.2 last sentence - again strange citation remnant

B.6.2.3.  after the case a)
(FWHM of the LSF) -- is it the Line Spread Function ? (probably not so commonly known abbreviation to anybody ...) 

after case b)
 ... resolution power stored as   - there is period power.stored

after case c) 
Char.spectralAxis.Resolution.resolPower !LoLim
seems to be the white space before !.LoLim

B.6.3.1
strange reference

B.6.5.1 
... more complex combinations such as:
phot.flux.density;phys.polarization.stokes.I 

is it correct  the stokes.I ? (or Stokes.I ?)

last sentence
should be probably parentheses (See examples in the table) 

B.6.5.2
strange citation

B6.6.

2nd paragraph  
phot.flux.density;phys.polarisation.Stokes.I   (written Stockes) and missing period - is it OK ?


Tables 6 and 7:
What does mean principal, Index and Std ?
The index TBD - or zero - - what is a meaning of it?
and what sometimes there are question mark ? It is a standard and everything should be clear to the reader ;-)

In addition to the typos and clarification needed:
There is a lot of places referring to SDM or SSAP in obsolete versions.
I suppose the SSA 1.0 et least (or 1.1.) should be referred to 
(in 3.3, B.4.3, B.5, B.6.5.2, ) and SDM probably the 1.1
(in 3., B.3.7, B.6.2, )

One inconsistency in table 5 and definition in B.6.2.1

there is no the em_calib_status in the table, but in B.6.2.1 it is defined. as {calibrated,uncalibrated,relative and normalized}.

I think it should be the level of wavelength calibration (not flux)
so the usage of "normalized" is questionable  (in SSA we have normalized only for fluxcalib). But it would break the symmetry in DM (same calibration for all axes)

However, I think, the introduction of observable axis o_calib_status in B.6.5.2 is the correct place but it is {absolute,relative,normalized, any}
B.6.5.2. refers to SSA (it is a value of FLUXCALIB query parameter)

So in query I can use "any" abut if it is describing characterization (data model ) - it is not directly corresponding to query,
however absolute, relative is a both state and query ....

I feel the columns from !ObsCore should be mainly used for queries - so the SSA value of any is correct then the SDM.

I  think the whole em_calib_status should be referring to SSA query params .... and the value of "normalized " kept for symmetry in SDM (maybe I can normalize the wavelength as well  to get relative value of distance from given line in e.g. percents...)

The same inconsistency in table 5 and B.6.1.4.
the s_calib_status is defined {uncalibrated,raw, calibrated} but in table is NOT CALIBRATED, FINE COARSE....

in t_calib_status (tab 5) is then written "Type of coord calibration" = not defined values and in B.6.3.3 is not defined anything - what values can have time calibration level ?????

I am confused about this - is it a inconsistency between SDM, SSA and 
!CharDM or I do not recognize something important?


-- PetrSkoda 2011, Jun 03

-- Response: MireilleLouys - 04 July 2011 _Most typos corrected in the last version.Updates for values of !calibStatus, modified for each axis._ 
-------
There does not appear to be any space for comments during the normal RFC period by the working groups.  This represents the applications take.  

While I don't see any major issues with the proposal technically, the proposal seems to contain a lot of ancillary justification that isn't necessary for the standard and which could get in the way of someone trying to read it with the goal of implementing it. A lengthy discussion of the justification for the standard, use cases, implementations and such does not belong in the standard itself (though they may be needed in the RFC process).  

My one technical concern is that the inclusion of the region field implies some capability of handling ADQL SQL extensions substantially beyond what a minimal TAP service need support, but this is mitigated by this field being nullable.

-- TomMcGlynn June 13, 2011

---++ Comments from TCG member during the TCG Review Period: 07-July-2011 - 07-August-2011

WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

---+++ TCG Chair & Vice Chair (<i>Christophe Arviset, Séverin Gaudet</i>)


---+++ Applications Working Group (<i>Tom Mcglynn, Mark Taylor</i>)

---+++ Data Access Layer Working Group (<i>Patrick Dowler, Mike Fitzpatrick</i>)

---+++ Data Model Working Group (<i>Jesus Salgado, Omar Laurino</i>)

---+++ Grid & Web Services Working Group (<i>Andreas Wicenec,      Andre Schaaff </i>)

---+++ Registry Working Group (<i>Gretchen Greene, Pierre Le Sidaner</i>)

---+++ Semantics Working Group (<i>Sebastien Derriere, Norman Gray</i>)

---+++ VOEvent Working Group (<i>Matthew Graham, Roy Williams</i>)

---+++ Data Curation & Preservation Interest Group (<i>Alberto Accomazzi</i>)

---+++ Knowledge Discovery in Databases Interest Group (<i>Giuseppe Longo</i>)

---+++ Theory Interest Group (<i>Herve Wozniak, Franck Le Petit</i>)

---+++ Standards and Processes Committee (<i>Francoise Genova</i>)


<hr/>

<br/>
<!--
      * Set ALLOWTOPICRENAME = %MAINWEB%.TWikiAdminGroup
-->

<br/>
<!--
      * Set ALLOWTOPICRENAME = %MAINWEB%.TWikiAdminGroup
-->
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
docx	PR-ObsCore-v1.0-20110502.docx	r1	manage	323.5 K	2011-05-03 - 10:49	MireilleLouys	ObscoreDM Proposed recommendation
pdf	PR-ObsCore-v1.0-20110502.pdf	r1	manage	1886.5 K	2011-05-03 - 10:50	MireilleLouys	ObscoreDM Proposed recommendation (pdf)
docx	PR-ObsCore-v1.0-20110712-final.docx	r2 r1	manage	342.0 K	2011-07-13 - 20:16	MireilleLouys	last version of proposed recommendation
pdf	PR-ObsCore-v1.0-20110712-final.pdf	r2 r1	manage	1837.7 K	2011-07-13 - 20:15	MireilleLouys	ObscoreDM Proposed recommendation July 11(pdf)
docx	PR-ObsCore-v1.0-20110712.docx	r1	manage	342.1 K	2011-07-12 - 21:56	MireilleLouys	last version of proposed recommendation
pdf	teleconJune06.pdf	r2 r1	manage	180.7 K	2011-06-08 - 12:59	MireilleLouys	Telecon June 06 :last comments on ObsCore updated
This topic: IVOA > WebHome > IvoaDataModel > ObsDMCoreComponents > ObsCoreRFC
Topic revision: r29 - 2011-07-13 - MireilleLouys