ConeSearchV10RFC < IVOA

---++ Simple Cone Search Proposed Recommendation: Request for Comments

This document will act as RFC center for the Simple
Cone Search V1.0 Proposed Recommendation.<br/>
The specification can be found at 
<s>http://www.ivoa.net/Documents/PR/DAL/ConeSearch-20060908.html</s>
<a href="http://www.ivoa.net/Documents/PR/DAL/ConeSearch-20070628.html">
http://www.ivoa.net/Documents/PR/DAL/ConeSearch-20070628.html</a> (revised based on RFC comments).

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your <nop>WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.  The deadline for comments is 21 April 2007.

Discussion about any of the comments or responses should be conducted
on the Data Access Layer mailing list, dal@ivoa.net.

---++ Comments from the Community
   * (1) This first comment is a sample comment

-- IVOA.DougTody - 2007-03-21
---

   $ Please Note: Please be aware of an important intention of this version of the Cone Search specification.  From the "Status" section of the preface (that I can imagine that many readers will keep over this expecting the usual stuff about the standards process): 

<blockquote>
It is the intention of this document that *all existing services compliant with the original NVO document will be compliant with this specification.* Similarly, all future service implementations that are based on this specification are expected to be compliant with the spirit of the original NVO document and practices in the VO current at the time of this writing; thus, existing client applications should work with the new implementations without change.
</blockquote>

Comments on this spec are likely to fall into one of three categories:
   1.  Wording changes that clarify the specification
   2.  Changes that propose new optional features that are completely backward compatible and require no changes to existing services or applications.
   3.  Changes may require slight modifications to existing services in order to be compliant, but would not have a significant impact on applications that use the services whether the modifications were made or not.  
   4.  Changes that would require major changes to existing implementations and/or the applications that use them.  

Category 1 should certainly be accommodated.  Changes in Category 4 should be saved for the next version.  Category 2 changes are possible, but we have opted to save these for the next version so that they can be afforded sufficient discussion.  Category 3 is a gray area that we can discuss (which will slow the process).  However, we hope to minimize such items as we do plan a version 1.1 very soon.  This will likely mean accepting some imperfections in order to provide an endorsed document for this defacto standard.

Thanks!

-- IVOA.RayPlante - 20 April 2007
---

   * (1) I suggest EQUINOX should be a mandatory input. J2000 should not be assumed. EPOCH should be recommended. 

-- IVOA.RichardMcMahon - 28 Mar 2007   

<em> IVOA.RayPlante responds:

This is a good suggestion; however, unfortunately, I think it falls into Cat. 4 above as it would require changes to existing implemenations and applications to support a new required input keyword.  (See John's comment about J2000 below.)</em>

---

   * section 2, part 1 : I find there is an ambiguity about the VERB parameter. The document states it is an "optional parameter" without telling if it applies to the client or to the server side. I think it should be made clear that this parameter is optional in the query, and that it is not compulsory for compliant services to support it. 
   * section 2, part 1 : typo in sentence "If a query includes an optional parameter ... the service should must ignore that parameter."

<em> IVOA.RayPlante responds:  Both good; thanks.</em>

   * In the References section :
      * [VOTable 1.0] link points towards version 1.1 of the document, not version 1.0

<em> IVOA.RayPlante responds:  I'll fix this.  CS can support both versions and implementations using each exist today.  I can clarify this. </em>

      * [RM] link is incorrect (final 'l' is missing in the URL)

<em> IVOA.RayPlante responds:  thanks</em>

   * Although I perfectly understand the need to be backward-compatible with existing Cone Search implementations, there are 2 minor points I would find annoying if I was about to develop a new CS service, and willing to conform to the standard :
      * the document compels to use the old UCD1, at least for the 3 compulsory fields (unless I repeat these 3 fields with corresponding UCD1+, which is rather ugly)

<em> IVOA.RayPlante responds:  The clinging to UCD1 is indeed unfortunate.  Note, however, that we expect the emerging TAP standard will have a simple cone search capability; thus, I expect that once TAP is in place we will encourage new implementations of TAP over Cone Search.  I recommend that we live with this one.  </em>

      * I feel uneasy about the ambiguity on the location of the ERROR parameter. If we plan to choose one location over another in a future document, it will mean changes at the service level anyway. Otherwise, the document should state explicitely (not only in the editor's note)that both locations are allowed.

<em> IVOA.RayPlante responds:  I would say that the "Otherwise" part of your comment is appropriate.</em>

-- IVOA.ThomasBoch - 28 Mar 2007
---
   * "Compliance REQUIRES that a web service be maintained responds to...":
      * Needs rewording.

<em> IVOA.RayPlante responds:  OK</em>

   * "RA -- a right-ascension in the J2000 coordinate system..." (also DEC):
      * Coordinate systems are not my thing, but J2000 looks like an epoch to me, rather than a coordinate system.  Shouldn't this say something like "the ICRS coordinate system" or "the FK5(J2000)" coordinate system?  However I am happy to defer to somebody who knows what they're talking about here.

<em> IVOA.RayPlante responds:  You are correct; let's go with ICRS.</em>

<em> IVOA.AnitaRichards comments: I agree with Ray but, just to make sure that misunderstandings don't propagate, here is my understanding: J2000 is an Equinox, defining the origin of a coordinate system.  It is supposed to be epoch-independent (to within a few decades and the level of precision common at the time). The year 2000 is an Epoch. B1950 is also an Equinox but it defines a system which was epoch dependent, hence, every ageing radio astronomer has nightmares about the positions of VLA calibration sources given in B1950, epoch 1979.9, which can be a significant number of resolution elements apart from the epoch 1950 positions.....</em>

   * "There may be other parameters ... , the service should must ignore that parameter":
      * Choose either "should" or "must".

<em> IVOA.RayPlante responds:  thanks</em>

   * "The basic MIME-type of the HTTP response should be text/xml.  The more specific form text/xml;x-votable may optionally be used.":
      * "text/xml;x-votable" is not a legal MIME type.  The generally agreed MIME type for VOTable is "application/x-votable+xml", though "text/x-votable+xml" would make sense as well (see RFC3023).

<em> IVOA.RayPlante responds:  This probably falls in the middle ground of category 3 (above).  I invite other comments on this, particularly on the question of how big an impact would this on applications using services that make this modification or not.</em>

   * "Exactly one FIELD must have ucd="ID_MAIN", with the type string":
      * Technically, "string" is not a VOTable data type.  This should probably say something like "with a string type" or "with a character array type".

<em> IVOA.RayPlante responds:  good</em>

   * "In case of error, the service MUST respond with a VOTable that contains a single parameter called Error":
      * For definiteness, I'd reword as "a single PARAM with name='Error'", or "a single PARAM or INFO with name='Error'", depending on what best matches current practice.

<em> IVOA.RayPlante responds:  good, thanks.</em>

   * Ref [VOTable v1.0]:
      * The target of this link is the VOTable 1.1 spec, not the 1.0 one.

<em> IVOA.RayPlante responds:  see response to IVOA.ThomasBoch above</em>

-- IVOA.MarkTaylor - 28 Mar 2007

---

   * I suggest to add a parameter DATE (or any else wording) to the set of query constraints in order to permit queries with solar system objects. Without this parameter, the cone-search queries will exclude these celestial bodies.

<em> IVOA.RayPlante responds:  This is a good idea, but I think it falls into Category 2.  As a backward compatible change, it would be appropriate for a version 1.1.</em>

-- IVOA.JeromeBerthier - 19 Avr 2007   

---
   * Section 2.1, last bullet starting by "There may be other parameters"... ends with "the service should must ignore that parameter" _should_ or _must_  is the question...

<em> IVOA.RayPlante responds:  thanks.</em>

   * section 2.3 (case of error). It is defined as a PARAM element, but I imagined an INFO element would be simpler. The PARAM element requires the "datatype" attribute while the INFO element does not. And whatever the final resolution is, a couple of examples of valid error returns would help the implementors.

<em> IVOA.RayPlante responds:  Yes, I think you are correct that the INFO would be simpler for encoding errors.  Unfortunately, switching exclusively to using an INFO element would qualify as a Category 4 change.  If alternatively we offered the INFO element as an alternate (but recommended) location to place error messages, however, this could break cone search clients that are already looking for errors.  Perhaps since there is already ambiguity in this regard currently, adding a little more wouldn't make much practical difference.  I invite discussion.

Nevertheless, we can add the examples.</em>

   * section 3, bullet *Waveband* -- it seems to me that one waveband is missing (Millimeter) -- or did the basic VOResource change ?

<em> IVOA.RayPlante responds:  this reflects the transcription from the original document.  Now that the document is in the standarization process, we should go ahead and change it to come in line with current practices using VOResource.  thanks.</em>

   * Usage of UCDs: is it totally irrelevant to propose to accept the more recent equivalents in UCD1+ ? 

<em> IVOA.RayPlante responds:  I think it would be good to save this for v1.1.  Those clients that do look for the required UCDs would be stuck if the service used UCD1+.</em>

   * Examples: one or two examples of conforming results  from a Cone Search server would help the readers and implementors (e.g. in an appendix)

<em> IVOA.RayPlante responds:  good idea--thanks.</em>

-- IVOA.FrancoisOchsenbein - 29 Avr 2007

--- 

   * For reasons of both backwards compatibility, and to "keep simple usage simple" (e.g., to be able to demonstrate a simple cone search query with only a browser) I suggest that we keep the base MIME type of the cone search query response as "text/xml"; this is also consistent with recent WG discussions in the DAL mail list and in Beijing.

   * In the current document we suggest that "text/xml;x-votable" can alternatively be used, using standard MIME type parameterization to more finely specify the content of the XML document.  However it now appears that MIME type parameters should be of the form param=value, hence a more correct parameterized form for the full MIME type might be "text/xml;content=votable".  Limited tests with modern browsers indicate that MIME-type parameterization of this form works fine, allowing the content to be more fully specified while defaulting to a "text/xml" display for the default configuration of a browser.  (For actual VOTable datasets, the form application/x-votable+xml would be used instead).

   * section 2.3 (case of error).  As others have noted, this specifies that a PARAM be used to indicate errors, however an INFO would be preferable, and would be more consistent with usage elsewhere (SIA, SSA).  If we changed this however we would break backwards compatibility, hence I suggest we leave this as a PARAM, and change it only in future versions of the interface.

-- IVOA.DougTody - 29 May 2007
--- 


---++++ Backward compatibility issue  

This document gives a description of the first protocol implemented in the IVOA and as such has to cope with the evolution of the user needs and how the community understands it.
The question of backward compatibility is raised here : when do we have to switch to a new standard version and 
declare the previous one obsolete. I understand that rewriting existing services is very cumbersome and not very gratifiying.

There are 11 keywords defined in Cone search that have conterparts in the VOresource schema. 
It seems to me that encouraging new services to support VOresource schema is really preferable . It would help to reduce keywords redundancy and avoid keywords translation.

Another question to discuss could be: do we let Cone search evolve independantly for an autonomous search function in the VO or do we want it to be a first step in more sophisticated search protocols?

Sorry , I'm new in this topic, may be I missed something.

-- IVOA.MireilleLouys - 2007-07-08

<em> IVOA.RayPlante responds:  As stated in the preface under Status (perhaps a bit hidden amongst the boiler-plate), this version has been written to ensure that services compliant with the previous spec is compliant with this one; thus, no update is needed.  

With regard to VOResource, another aim of this document was to minimize the changes to the substance of the spec; thus the old resource keywords remain.  The next version will certainly replace these keywords for VOResource versions, and appendix B will go away.

The main idea is to cement what is already in place. 
</em>
---



<!-- ============ PLACE COMMENTS ABOVE THIS LINE ============= -->

---++ Comments from the Technical Coordination Group

TCG members should add their comments under their name.
---+++ Mark Allen (Applications WG)

On the whole this is a good document and we approve it.  However there are a number of points which the authors should attend to.

---++++ *Typos*
   * Status of this document:
      * "Minor changes have been made sharpen the specification..." -- insert "to"
   * Sec 2.1:
      * "http://mycone.org/cgi-bin/search?RA=180.567&DEC-30.45&SR=0.0125" -- s/DEC-30/DEC=30/
      * At end of discussion of VERB parameter, "the service should ignore parameter" -- insert "the"
   * Sec 2.2:
      * In discussion of POS_EQ_RA_MAIN/POS_EQ_DEC_MAIN UCDs, it says "representing the J2000 right-ascension/declination (in the ICRS system)".  I *think* this is tautologous and/or incorrect, since J2000 is implied by ICRS.  However I might be wrong (please consult a coordinate system expert).
   * Sec 3:
      * <nop>MaxSR: "If there is no restriction a value of 180.0 if there is no restriction." -- isn't right.
      * "The service will considered..." - insert "be".
   * Appendix C:
      * "References to the original HTML document has been replaced" -- s/has/have/

---++++ VOTable Examples
All the example VOTables have errors.  STILTS [[http://www.starlink.ac.uk/stilts/sun256/votlint.html][votlint]] reports:
   * Example error response 1:
      * ERROR (l.5): Element "VOTABLE" does not allow "PARAM" here.
      * ERROR (l.5): No datatype specified for PARAM Can't parse values
   * Example error response 2:
      * ERROR (l.4): Attribute "about" is not declared for element "DESCRIPTION".
      * ERROR (l.9): No datatype specified for PARAM Can't parse values
   * Sample VOTable response:
      * ERROR (l.13): No datatype specified for PARAM Can't parse values
Notes:
   * PARAM elements _must_ have datatype attributes, just like FIELD elements.
   * VOTable 1.0 does not permit a PARAM as the direct child of a VOTABLE, though VOTable 1.1 does.  It may be worth noting this in the part where it says that a direct child of a VOTable element is an OK place for an error PARAM to appear.

<em> IVOA.RayPlante responds:  will fix--thanks.</em>

---++++ MIME Types
I'm not particularly happy with the recommended MIME type being =text/xml= rather than =application/x-votable+xml= or possibly =text/x-votable+xml=.  However, this was raised on the RFC page and if the authors consider this the best option to maintain compatibility with existing services, I will accept it. I would recommend (but not insist on) at least a mention of the standard MIME type.  I suggest adding one of the following alternative sentences:
   * "The recommended MIME type for VOTable is =application/x-votable+xml=, but the other forms here are allowed for backwards compatibility."
or
   * "The standard MIME type for VOTable, =application/x-votable+xml=, is disallowed/discouraged here for reasons of backward compatibility.

-- IVOA.MarkTaylor (apps vice chair -- comments agreed by IVOA.MarkAllen) - 05 Jul 2007  



---+++ Christophe Arviset (TCG vice Chair)
I approve this document but I want to flag that we agreed in Moscow interop that the SCS would be an "IVOA legacy standard" to reflect the numerous SCS implementations to date. In that context, I don't believe that there should be future evolution of this standard as my understanding is that it will be superceded by other IVOA standards (eg TAP).

---+++ Matthew Graham (Grid &amp; Web Services WG)

I approve this document.

---+++ Bob Hanisch (Data Curation &amp; Preservation IG)

I approve this document.

---+++ Gerard Lemson (Theory IG)

I approve this document.

---+++ Mireille Louys in agreement with Anita Richards (Data Models WG)
This document shows a clear inheritance from a previous standard definition on which first VO services have been elaborated.
The updates from the original version are clearly mentioned but I think new comers like me might be lost between the old and new recommendations while writing a new service.
Examples of good practice emphasising new features of the standard would help.

I feel uncomfortable that Old UCD format is part of the document as many other upcoming standards use UCD 1+ strings. Is it planned to update UCD format for the next version?
Recipes for data and tool providers to insert translation services could be provided (even in between Cone search calls and data, for 
example) so that both can be supported, albeit with some loss of precision - with probably not much ambiguity for the obligatory parameters of cone search, i.e. RA and Dec...

There are situations where data are published 
incorrectly: there are cases where a cone search can appear to work but give wrong results - i.e., with data in the wrong coordinate system or with a wrong assumption about the units of the search radius.

About section 3. The Resource Profile
...
* <nop>SpaceRefFrame <nop>RefSystem:  this should always be ICRS for the positinal columns to be searched in a Cone Search - this would remind the data provider to check, and make it easier for future services to be extended.

Galactic and other coordinates are not at present supported.
Is it planned as part of ADQL or TAP or whatever? This is of huge 
importance to Galactic astronomers.

We approve the document and look forward for the next version.

---+++ Keith Noddle (Data Access Layer WG)

I approve this document.

---+++ Francois Ochsenbein (VOTable WG)
I do not think that this standard should be, in this version, a
Recommendation: it merely shows a lack of coherence with the
VOTable 1.1 standard (cf  Mark Allen's remarks above):

   * about the UCDs: UCD1+ are recommended in VOTable1.1
   * the <FONT color='darkred'>&lt;PARAM name="Error" value="..."/&gt;</FONT> (example of 2.3) is in contradition with the <FONT color='darkred'>PARAM</FONT> definition (<FONT color='darkred'>datatype</FONT> attribute is required)

I could agree with a a document describing a <i>de facto standard</i>  but I would better see a proposed recommendation which would <FONT color='darkred'>not</FONT>  be in contradiction with the other standards! For instance, propose that the resulting VOTable contains fields having as content of the <FONT color='darkred'>ucd</FONT> attributes:

   * either <FONT color='darkred'>POS_EQ_RA_MAIN</FONT>  or <FONT color='darkred'>pos.eq.ra;meta.main</FONT>
   * either <FONT color='darkred'>POS_EQ_DEC_MAIN</FONT>  or <FONT color='darkred'>pos.eq.dec;meta.main</FONT>
   * either <FONT color='darkred'>ID_MAIN</FONT>  or <FONT color='darkred'>meta.id;meta.main</FONT>

For the positional error, the <FONT color='darkred'>OBS_ANG-SIZE</FONT> UCD has not this meaning! Are there ConeSearch services which return this ?? Here the UCD1+ <FONT color='darkred'>stat.error;pos</FONT> is obviously much more appropriate.


---+++ Pedro Osuna (VOQL WG)
It is my understanding that we agreed that this document would be a "refurbishing" of the old NVO document to make it "IVOA-like" and therefore, preserve existing NVO (and others) SCS services for the future, as legacy services which have demonstrated their usability.
In this sense, I don't think we should look too much into details the possible misalignments with the rest of the existing IVOA protocols, as this one should be left as a legacy document, and therefore not evolve further.

Having said this, I approve the document.

Minor comments:

  1. The introduction says "[...]This specification describes how a provider of an astronomical source catalog[...]". The "astronomical source catalog" is a controversial concept, as we have already seen in the DM group on the Source Catalogue Data Model, still under discussion. I would remove reference to that, and simply say:
"[...]This specification describes how a provider of astronomical data[...]"

  2. In the same introduction, two paragraphs below, it explains the assumption that the data provider has already selected a catalog of astronomical sources, that can be represented as a single table, that has several columns. This is too restrictive, and has the flavour of implementation issues, so I would remove the whole paragraph. 

  3. In point 3. "[...]A Cone Search service MUST be described with a Resource Profile[...]"
I understand that it MUST do so _in the Registry_ (no mention is made to possible verbs in the HTTP query to retrieve those data). If my assumption is correct (which is hinted later after the last "BaseURL" parameter explanation) it should be clearly said. Something like:
"[...]A Cone Search service MUST be described _in an IVOA compliant Registry_ with a Resource Profile[...]"

<em> IVOA.RayPlante responds:  These are all good points that I would normally agree should be dealt with, but the statements discussed reflect the understanding of the VO at the time, and I'm not inclined to try to correct this in this version.  W.R.T 1 and 2, it intent of the original authors is that the spec has been designed on the assumption that the data can be represented as a single flat table and that each record represents a source--an emitter of radiation.  Some implementations have stepped out of this assumption for effective purposes (e.g. each row is an image overlapping the cone); this is not disallowed. </em>

That's all. Nice simple but powerful piece of work.

---+++ Ray Plante (Resource Registry WG)

As the editor of this document, I approve it.

---+++ Andrea Preite-Martinez (Semantics WG)

I approve the document, with the condition that the EXEC explicitly invite the authors to immediately open a discussion on a new WD version 1.1 of the protocol.

---+++ Roy Williams (VOEvent WG)

I approve the document.

<br/>
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
html	ConeSearch-20070914.html	r1	manage	44.0 K	2007-09-14 - 22:04	RayPlante
This topic: IVOA > WebHome > IvoaDAL > ConeSearch > ConeSearchV10RFC
Topic revision: r32 - 2007-09-14 - RayPlante