TAP V1.0: Request for CommentsThis document will act as RFC centre for the TAP V1.0 Proposed Recommendation. Review period: 2009 July 10 – 2009 August 21 (6 weeks) In order to add a comment to the document, please edit this page and add your comment to the list below (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussions about any of the comments or responses should be conducted on the DAL mailing list, dal@ivoa.net.Comments from the community
issues from the mailing list just before and during the RFC period, 2009-08-28, collated here by PatrickDowler0. purely editorial issues not listed here 1. sec 2.3.6 arbitrarily limits the VOTable to TABLEDATA format assessment: minor change to remove limitation 2. sec 2.2.2 says "single result" and 2.7.1 says "single table" assessment: minor change to clarify that the query language dictates how many "tables" are produced by a single query; SQL-based languages like ADQL produce a single table 3. sec 2.3.4 specifies the ISO8601 timestamp format without the T separating the date and time parts note: testing showed that some RDBMSs cannot parse the format with embedded T, but T is used elsewhere in IVOA assessment: minor change to require the T; imposes possible work on implementors 4. sec 2.3.8 describes the MTIME request attribute which filters the rows returned from a query, but support is optional so this could be very dangerous note: MTIME is intended to be used in conjunction with MAXREC to harvest content from one service to another, eg for indexing/search engine purposes, already appears in SSA. It seems poorly understood in the TAP context and can be complex underneath... also worries people when mixed with the concept that unsupported request params like MTIME can be silently ignored, leading to large (MAXREC) query results. MAXREC does protect both service and client worst case. assessment: hard to fix (fully specify) at this time; we could specify that queries fail if MTIME is used and not supported (rather than ignored), clarify as much as possible, and promise to release a Note about implementing and using it later... or we could drop it from 1.0 and worry about it again for 1.1 5. sec 2.5 specifies use of xtype="adql:POINT" or xtype="adql:REGION" rather than using xtype='STC-S" from the VOTable 1.2 doc note: ADQL treats point and region slightly differently in a few places and both were recognised as necessary data types; for better interroperability between services, something like xtype="stc:AstroCoords" or xtype="stc:Region" and xtype="stc:ISOTime" would be better and we would have to VOTable-specific or TAP-specific xtypes.... assuming one can extract suitable type-names from STC. assessment: need the distinction, cannot use only VOTable values 6. the static resource structure specified by TAP is rigid assessment: substantial change at this time since we do not have defined capabilities to describe the URLs for these endpoints (see #8) 7. inline table upload uses en element name as the intended (destination) table name and as specified too tightly coupled with http (also from Tom's comments above) note: has inline table upload been implemented in any prototype? is it proven to work or proven to not work? given that uploads have to be VOTable, is this really usable as is anyway? it does avoid the need for a server (http, eg) to serve the uploaded table assessment: substantial change to fix if the current draft is unworkable... 8. VOSI requests are done via the REQUEST parameter and the sync endpoint rather than plain resources off the base URL of the service, thus entangling the mandatory support for sync (and async) with the location of the VOSI requests... proposal was to access VOSI via resources, make both sync and async optional (must have one, of course), and describe the presence and location of sync and async endpoints via capabilities (thus also relieving part of #6 above) note: inconsistent with DAL style as specified in SSA 1, consistent with REST and GWS style as expressed in the VOSpace 2.0 draft assessment: modest work to change VOSI to resources from REQUEST(s), substantial change to make sync and async optional and define capabilities to describe them, but then we get the flexibility asked for in #6 for free
if the tables[...] contain [...] spatial coordinates and the services support spatial querying via the ADQL region construct, (then) the service MUST support INTERSECTS [...], REGION, POINT, BOX [...]That initial if means that REGION support is not mandatory. Correct? (An if...must is quite misleading). If correct, then how to specify that my service does or does not support regions? In my opinion this must be spelt out explicitely in the document. I would claim useful if the overall philosophy on how to construct a query (first access the metadata, to gather column names, units, etc, then formulate a query) could be highlighted in the introduction. I would claim very useful for data providers' uptake if tutorials could be provided on various aspects, for example:
Comments from the TCG during the normal RFC period (2009 July 10 -- 2009 August 21)Applications (Tom McGlynn, Mark Taylor)See comments above (TAM)Data Access Layer (Keith Noddle, Jesus Salgado)Data Model (Mireille Louys, AnitaRichards)Grid&Web Sevices (Matthew Graham, Paul Harrison)Registry (Ray Plante, Aurelien Stebe)Semantics (Sebastien Derriere, Norman Gray)VOEvent (Rob Seaman, Alasdair Allan)VO Query Language (Pedro Osuna, Yuji Shirasaki)VOTable (Francois Ochsenbein, 2009-09-25)My concerns (http://ivoa.net/forum/dal/0907/1406.htm) are included in Pat's summary; the main points related to VTable are:
Standard and Processes (Francoise Genova)Astro RG (Masatoshi Ohishi)Data Curation & Preservation (Bob Hanisch)Theory (Herve Wozniak, Claudio Gheller)TCG (ChristopheArviset, Severin Gaudet)Comments from the TCG (2009 October 09 -- 2009 November 06)Note: A revised PR-TAP-1.0 document, taking into account the above issues and subsequent mailing list discussions, is now available from the documents page: PR-TAP-1.0-20091006Note: something odd happened during html export that put odd characters in various places; will fix so no need to report on that; the PDF should be ok.Applications (Tom McGlynn, Mark Taylor)This document clearly defines a core functionality for generic queries of tables using either ADQL and PQL. This functionality is central to the development of VO applications and with this document as a guide both data providers and data clients should be able to build very powerful services using these core capabilities. Promoting this document to a recommendation should encourage the widespread adoption of the core TAP functionality leading to an immense increase in the fundamental capabilities of the VO. The TAP interface is very complex with with strong interactions with a myriad of other VO protocols. A number of issues in ancillary areas have arisen since the release of the RFC, in TAP metadata, table uploads, VOSI capabilities, version negotiation, geometry support, and the interfaces of TAP to W3C and other IVOA protocols. A few of these have been addressed in changes made to the document between RFC and TCG review. It is unclear if substantive changes of this kind were appropriate at that point, since they were not themselves subject to general review. Overall these issues raise doubts that this document is an adequate guide to building a fully conformant TAP implementation. Given the juxtaposition of these opportunities and concerns, Applications abstains in the approval process, with a recommendation that, regardless of the outcome of this process, work begin immediately on a new version of the document which focuses not on new functionality but on a clarification and resolution of issues in these ancillary areas.Data Access Layer (Keith Noddle, Jesus Salgado)Data Model (Mireille Louys, AnitaRichards)Grid&Web Sevices (Matthew Graham, Paul Harrison)Registry (Ray Plante, Aurelien Stebe)I have two TCG-level comments:
| ||||||||
Changed: | ||||||||
< < | I'll note that overall I was impressed with the level of detail covered in this document. | |||||||
> > | I'll note that overall I was impressed with the level of detail covered in this document. I can approve this document once these issues are addressed. | |||||||
-- RayPlante - 28 Oct 2009
Semantics (Sebastien Derriere, Norman Gray)Comments from NormanGray. There are minimal interactions with the Semantics WG, so the following are comments on exposition and the document itself. I haven't worked through the various comments above, so some of these might be duplicates. Hrumph. I was looking at the 2009 June 7 version of the document. It might be useful to put a link to the current version of the document in the first line of this page, and not a four-month old version. It's not clear to me, therefore, how many of the comments below are still useful. Document as a whole:
VOEvent (Rob Seaman, Alasdair Allan)VO Query Language (Pedro Osuna, Yuji Shirasaki)VOTable (Francois Ochsenbein)(Comments by FrancoisOchsenbein on 2009-11-05 on TAP document dated 2009-10-26, improved a lot compared to the previous version, thank you!)
Standard and Processes (Francoise Genova)Astro RG (Masatoshi Ohishi)I can go along with the document after the suggested changes have been fixed. Let us have the TAP v1.0 as soon as possible.Data Curation & Preservation (Bob Hanisch)These comments are mostly presentation details which are easily fixed, however there are a couple of issues of substance. None of these are show-stoppers in my view, however, and I would be ok going ahead to REC if things are tidied up. Another option would be to hold TAP at PR for a period of time so that we could see a larger number of fully functional, interoperable implementations, and get further feedback from implementers regarding where the specification needs more detail. The labels “normative” and “informative” do not seem to be applied consistently. Only Section 1.4 of the Introduction, for example, is labeled “informative”, but it seems appropriate that all of Section 1 be labeled “informative”. On the other hand, Section 7 has the “must” and “should” language of a standard and would seem to be “normative”. If the point is to recap HTTP rules then it might be better to move this to an Appendix (e.g., content that is explicitly informative) so it is clearer that these rules are about HTTP and not about TAP. Section 1, para 1, both ADQL and PQL should have references here, on their first occurrence in the document. Para 2, “or in some other format” would imply that virtually anything is ok. Better would be something like “or in one of several other formats specified herein.” Section 1.1, sentence is missing an “and” after “metadata queries”. Section 1.1.2, first sentence has awkward phrasing in “a subset and patterned after”. Perhaps “...to standardized tables that are a subset of, and patterned after, information schema in RDBMSs...” Section 1.1.3, first sentence needs to be with “The” and another “the” is needed before “base service”. In last sentence, “IVOA registry” needs a reference. Section 1.2.1, “...will not be usable with other services unless they implement the exact same data model.” This does not seem correct to me. Isn’t the point that the data models need to be sufficiently similar that like elements are comparable? For example, one could certainly compare two TAP services with very diverse contents if both provided the same contents in the area of interest to the user. So I would have said “a sufficiently similar data model” or something along those lines. Section 1.2.2, I am not sure that “PQL is more abstract than ADQL” (actually I might have said ADQL is more abstract than PQL), but it really doesn't matter. I would drop this phrase and just have “PQL can be used in some cases without first querying the metadata tables; PQL parameters carry sufficient meaning to enable...”. In last sentence, it should be “Details...are...” Section 1.3.2, last sentence, insert “are” before “likely to fail”. Section 1.4, first sentence needs “The” at the beginning. Footnote 2 ends with “(see other)”. See other what? Actually, I know that this means that HTTP code 303 stands for “see other”, so it is just the parentheses that make this a little confusing. Writing “HTTP code 303: See Other” would solve the problem. Section 2.2, reference [5] at end of 1st para should be [3] The list of UWS resources is not completely consistent with the UWS specification. In particular, UWS does not define a “termination” URI, and the URI is “parameters”, not “param”. Section 2.3, last sentence, I would have thought that a response to a spurious parameter should at least include a warning rather than being completely ignored. Section 2.3.1, 3rd para, “This is...” should be “These are...” as it us referring to “additional parameters”. Section 2.3.3, “...is ADQL...” should be “...are ADQL...”. The parenthetic descriptions of ADQL and PQL are unnecessary here, as this has been said elsewhere. Section 2.3.4, 4th para, “...and the services supports...” should be “...and the service supports...” Section 2.3.5, 1st para, “...are specified in elsewhere...” ??? 3rd para, “...and the services wants...” should be “...and the service wants...” End of 4th para, suggest rewriting as “Details on how to use table references in PQL are given in the PQL specification document. [ref]” Section 2.3.7, 2nd para, “This value” should not be in italics. Section 2.3.9, “(e.g., scheme is vos or param)” I think needs to be clarified Section 2.5, 3rd sentence, missing “the” before “uploaded table” and missing “a” before “legal ADQL table”. Section 2.5.2, 1st sentence, missing “the” before “caller” In the example, this reader does not grasp the “boundary=AaB03” stuff. Is this obvious to others? Section 2.6, the 1st sentences seems to go around in circles. In skeleton form: “The [] schema defines tables in the SCHEMA schema that contain [] metadata required to use the tables...”. At some level I think it makes sense, but I’m hoping that this can be reworked! Section 2.6.1, “The schema_name values are unique.” Unique within what scope? How is this to be enforced? I have the same question regarding the definition/scope of “unique” in 2.6.2. 2.6.3, and 2.6.4. Section 2.6.3, column_name unit, cites “VO standard format”. We don’t actually have an explicit document on standard units. VOTable refers to SI units and FITS conventions. I think this needs a footnote or something to explain the situation. Section 2.6.4, there is a block of text starting with “Data types and how they map...” which seems out of place here, as this is not to do with Foreign Keys. Does this belong in the general text of 2.6 at the beginning? In the middle of this paragraph “...where the services chose...” should be “...where the service selects..”, and “...which columns are returned by default” would be better as “...those columns that are returned by default.” Section 2.7.3, after bullets, “...service containers an cannot...” should be “...service containers and cannot...” Section 2.8.1, remove hyphen in “version-number” and add reference after last sentence (to IVOA document conventions). Section 2.8.3, what is meant by “...level two”? Section 3, 5th paragraph, “The resource document should include the table metadata...”. I think this is getting a bit far afield from TAP and straying into best practices for registries. I’d rather the “should” were a “may”. The footnote (6) is very wishy-washy and I think can be deleted. Section 5.1 and 5.2, there are several places where there is extraneous space in the example HTTP GET and POST commands, particularly “/tap /sync/” In 5.1, as noted before, the UWS resources are not totally consistent with the UWS specification. In the middle of this section, the sentence that begins “Practically, ...” is a very odd structure. I don’t think one typically starts a sentence with an adverb like that. It would be better as “In practice it is very hard...”. Section 6, rather than “prototyping” I’d suggest “further implementation experience”.Theory (Herve Wozniak, Claudio Gheller)TCG (ChristopheArviset, Severin Gaudet)<--
|