TAPImplementationNotes < IVOA

This page is intended to collect points that should be clarified/fixed in
future versions of the UWS/TAP/ADQL combo of standards.  MarkusDemleitner
suggests we should edit much of this into an IVOA Note ("Implementation notes
for a service implementing UWS, TAP, and ADQL") and fix standards documents
after that as necessary.

Some points (which should eventually all be reflected here) are raised in 
   * http://www.ivoa.net/internal/IVOA/InterOpMay2011GWS/normand_malapert_lesidaner_uwsIVOA2011V2.pdf 
   * http://voparis-srv.obspm.fr/doc/uws-v2.0.txt
   * http://www.ivoa.net/internal/IVOA/InterOpMay2011Applications/tap-gavo.pdf

Please inline comments to existing points as one-level-deeper enumerations (I guess...)

---++ UWS

---+++ From Paul's mail

The following points are discussed in the Mail B36FEF85-E316-436E-AC69-2F92D0E0FC5C@manchester.ac.uk dated 2011-05-23 to the GWS list by PaulHarrison

Paul promised to update the UWS in volute to reflect much of this, but it
certainly wouldn't hurt to have some if it in an "implementation note"-type document.

   * Section 2.1.3 HELD status - whilst this might appear to have little utility in current implementations, in future versions where there might be quotas or priorities in the UWS then HELD is a way of expressing within the UWS that the job is accepted in principle, but will not be run until some action (like freeing up some of the quota) is taken.
   * It is probably not made clear enough that the initial values of the parameters (and certainly the possible parameter names) are all established during the initial POST that creates the job and in most cases this is how the job should be driven - The ability to set an individual parameter after job creation is an additional capability that the UWS may offer - it should not offer the ability to create new parameters nor delete existing parameters - in this way a client that just creates the job with the initial POST does not "miss" out on setting a crucial parameter. We could make this clearer by removing the ability to set the individual parameter, as I believe that it was added as a "would be nice" feature without a strong use case. There is only one guaranteed way to set a parameter that all UWS services must implement - in the initial POST that creates the job.
   * Section 2.2.3.2 & 2.2.3.3, Changing execution duration & destruction time - if a service choses not to implement these features, then the standard is clear that a value of 0 should be returned for the execution duration, but I agree it is not clear what should be returned for the destruction time - in the job schema the !DestructionTime element is nillable, so that would be appropriate representation in the job XML - however for the value returned at the resource URL then I agree that there is no description of what should be returned in the case where the UWS never deletes a job - you could return a value far in the future.
   * a job can be deleted at any time - it is up to the UWS server side to clean up appropriately
   * Although the current wording of the document does not make this clear enough in every case, the intention is that changing the PHASE of the job is a request by the client to the server, and the client sees whether it has been successful by examining the XML returned by the redirect to the URI /{jobs}/(job-id)/. The allowable transitions are shown by the state diagram within the document.  TODO: Decide if invalid transitions should be an error
   * Attempt to update a parameter on a job that's not PENDING: a 403 [Forbidden] status should be returned
   * The text needs updating to say that creating a parameter at any stage other than the initial job creation POST is not allowed.

---+++ Other
   * Section 2.2.3.1 defines the HTTP response code for an accepted job as 303, but does not say what should happen for a rejected job.  It should do (200 plus error document??) -- IVOA.MarkTaylor - 08 Jun 2011
   * The content of the /quote resource is an integer number of seconds (sec 2.1.1), but the content of the uws:quote element is xs:dateTime (schema); this mismatch seems unnecessarily confusing unless there's some rationale I'm missing. -- IVOA.MarkTaylor - 29 Jun 2011

---++ TAP

   * Can we come up with a lightweight way of allowing some sort of (insecure) authentication ("don't publish my queries") while letting people to re-upload remote TAP results? --[[MarkusDemleitner][MD]]
   * UPLOAD parameter spec needs some clarifications. --[[MarkusDemleitner][MD]]
      * Are quoted identifiers allowed as table names? (in !DaCHS, they are not)
      * What should hapen if a URL or table name contains a comma or semicolon? (in !DaCHS, they are effecitvely forbidden in both table names and in URLs, since there is no way to escape them)
      * When people re-post an UPLOAD parameter, should uploads be added or replaced? (in !DaCHS, they are added)
   * xtype=adql:REGION on upload: such columns will usually result in polygons, at least when implementing against pgsphere --[[MarkusDemleitner][MD]]
   * Require a filename header on inline uploads? (this would make it easy to tell them from "regular" parameters without having to parse all UPLOAD parameters first) --[[MarkusDemleitner][MD]]
   * One of the columns in the TAP_SCHEMA.columns table is named "size".  This is an ADQL reserved word, which is unfortunate. Can be got round by quoting the column name in ADQL, but it's a gotcha which might be worth mentioning. -- IVOA.MarkTaylor - 08 Jun 2011
   * UWS 2.1.11 discusses how parameters of an existing job can be updated, and says that it's up to the implementation to define what is permitted. As far as I can see this is not really done by TAP, though some examples in the _Informative_ section 5 provide suggestions.  It should be clarified. -- IVOA.MarkTaylor - 24 Jun 2011
   * Should the table metadata (from /tables endpoint and TAP_SCHEMA tables) include metadata about the TAP_SCHEMA tables themselves?  Should be made explicit in the TAP standard. -- IVOA.MarkTaylor - 28 Jun 2011
      * Agreed; since 2.6, second paragraph, says they should be in TAP_SCHEMA, I'd venture it's pretty much implied they should be in /tables, too.  -- IVOA.MarkusDemleitner - 29 Jun 2011

---++ ADQL

   * The spec omits language that says <separator> (and thus comments) is what actually separates tokens.  Thus, a naive implementation of the grammar only allows comments between parts of split-up string literals.  The spec needs to be improved, but meanwhile saying "<separator> is this grammar's token separator" or so should do. --[[MarkusDemleitner][MD]]
   * Decaying INTERSECTS with point arguments to CONTAINS is a major implementation effort without much benefit.  Can we please just deprecate it? --[[MarkusDemleitner][MD]]
   * Can we recommend a simple positional crossmatch function like <tt>crossmatch(ra1, dec1, ra2, dec2, radius)</tt>, all in degrees?  People use that a lot, and asking them to write that CONTAINS mess all the time is not nice --[[MarkusDemleitner][MD]]
   * There should be some language on what to do with oversized uploads; in the inline case, the server probably should send back a 413 status and just close the connection (which, for common client libraries, will just raise a connection reset exception or so, but there's nothing we can do about this as far as I know) --[[MarkusDemleitner][MD]]






<br/>
<!--
      * Set ALLOWTOPICRENAME = %MAINWEB%.TWikiAdminGroup
-->
This topic: IVOA > WebHome > IvoaDAL > TAPImplementationNotes
Topic revision: r8 - 2011-06-29 - MarkTaylor