Difference: UWS11RFC (11 vs. 12)

UWS v1.1 Proposed Recommendation: Request for Comments

Public discussion page for the IVOA UWS 1.1 Proposed Recommendation.

The latest version of the UWS Specification can be found at:

• http://ivoa.net/documents/UWS/20150907/index.html

Reference Interoperable Implementations

Would the implementors of the UWS 1.1 specification please add details; or, please remove your implementation from this list if not correct.

CDS / GAVO Implementation (Gregory Mantelet)

VO Paris Implementation (Mathieu Servillat)

CADC Implementation (Pat Dowler)

GAVO uws-client Implementation (Kristin Riebe, Adrian Partl)

Implementations Validators

No implementation validators for UWS 1.1, though CADC (and others?) have UWS 1.0 validators that still pass:

• CADC UWS 1.0 Validator: OpenCADC cadcTestUWS

• taplint TAP validator within stilts does non-exhaustive validation of the UWS v1.0 functionality of a TAP service: job submission, lifecycle tracking, deletion, job document parsing etc. See UWS Stage. If someone can point me at a running UWS 1.1-compliant TAP service (minimum: WAIT-blocking and job documents labelled version='1.1' ) I'll try to update the validation for it. -- MarkTaylor - 2015-07-03

RFC Review Period: 2015-06-30 - 2015-08-18

TCG Review Period: 2015-09-09 - 2015-10-07

Comments from the IVOA Community during RFC period: 2015-06-30 - 2015-08-18

Comments from MarkTaylor

(Still) a good clearly written document. One or two minor comments/suggestions:

* Notation: I don't understand the usage of curly and round brackets in referencing URL path elements. For instance in sec 2.1.11 I see: "/{jobs}/{job-id}/parameters/(parameter-name)", and in sec 2.2: "/(jobs)/(jobid)", and in sec 2.2.1.1: "/{jobs}/(job-id)". Are () and {} doing different, ahem, jobs here? If not, can you just pick one or the other?

• • there is no significance in the different types of braces - I have made them all curly.
• Sec 1.3: "an XML document (e.g. ADQL, CEA [harrison05])" - is it anachronistic to refer to ADQL in this context since post-ADQL-2.0 there is no ADQL/X representation?
  • I think you are right - I have removed it
• Sec 2.1: A count indicator ("1") is missing from the Job-Owner link in the UML diagram
  • you are right - and quote should be 0..1
• Sec 2.1.3: The phases "HELD", "SUSPENDED" and "ARCHIVED" do not appear in the diagram.
  • I had not added these in the past as I thought it would complicate the diagram, as the conditions for transition into those states are more complex - you are probably right that they should be included though, and I have produced a version with them all in.
• Sec 2.2: "future versions of this document will add other bindings such as SOAP." Really?
  • highly unlikely - I have made the statement less precise about which binding may be added.
• Sec 2.2: "an non-existant job id" -> "a non-existent job-id"
  • done
• Sec 2.2.1.1: "it should not return a response (i.e. block)" is a bit ambiguous. Prefer "it should not return a response (i.e. it should block)"?
  • done
• Sec 2.2.1.1: "/{jobs}/(job-id)?WAIT&PHASE=QUEUED": I think it has to be "WAIT=<value>" not just "WAIT"?
  • done
• Sec 4.2: "ADQL [1] can serve as a JDL." - bibliography referencing is wrong.
  • corrected
• Sec 4.3: "Call it CEA v2 to distinguish it from CEA v1 as currently maintained by AstroGrid." - it's a bit anachronistic.
  • changed wording to make it clear that AstroGrid is no longer active
• Sec 4.3: "The JDL in CEA v2 is similar to that in CEA v1 It is a formal ..." - missing full stop
  • done
• Sec 4.3: "the emerging XForms technology" - is it still emerging?
  • not really - have updated wording.
• Schema: version attribute to be made optional? (see schema evolution discussion)
  • done in latest version of document and schema

Thanks Paul - fixes all look good. -- MarkTaylor - 2015-09-11

Comments from WalterLandry

Regarding the new job query capability in Sec 2.2.2.1, it feels like
we are creating yet another query syntax with new keywords. To be
concrete, I would change the examples to

• /{jobs}?QUERY=PHASE=EXECUTING
• /{jobs}?QUERY=STARTTIME>2014-09-10T10:01:02.000
• /{jobs}?QUERY=TOP+100

• /{jobs}?QUERY=(PHASE=EXECUTING+OR+PHASE=COMPLETED+OR+PHASE=SUSPENDED)+AND+STARTTIME>2014-09-10+ORDER+BY+ENDTIME

Reply by PaulHarrison

Whilst I would normally be a strong advocate of trying to reuse existing syntax and not do re-invention, in this case the filtering capability is not intended to be a full query language for jobs. I think that your suggestions start to introduce the full query language approach which has the following disadvantages

1. More effort has to be made in defining the query language than the current standard does - e.g. TOP in ADQL does not imply any ordering - by using the "keyword" style filtering that the current document specifies exactly what LAST means - so to be consistent with ADQL the "query language" should be something like /{jobs}?QUERY=TOP+100+ORDER+BY+STARTTIME+DES
2. There is a greater burden on the implementors to do the query parsing.

Comments from TCG members during the TCG Review Period: 2015-09-09 - 2015-10-07

WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or do not approve of the Standard.

IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory.

TCG Chair & Vice Chair ( _Matthew Graham, Pat Dowler )

Applications Working Group ( _Pierre Fernique, Tom Donaldson )

Data Access Layer Working Group ( François Bonnarel, Marco Molinaro )

Data Model Working Group ( _Mark Cresitello-Dittmar, Laurent Michel )

Grid & Web Services Working Group ( Brian Major, Giuliano Taffoni )

Registry Working Group ( _Markus Demleitner, Theresa Dower )

p.7, paragraph "ARCHIVED: At...": I believe "but the metadata associated with the job must be retained" was actually intended to be "but the metadata associated with the job have been retained" (or I don't get what this is intended to say).

p.9, "Indicated in the job XML as a Nill element": I believe this should be "Nil" and that the double l in "nillable" is just morphological reduplication. (Also, Quote one paragraph down has "nil" with one l and in lower case; whatever you choose, it should be consistent)

2.1.11 I thought that during RFC I had ranted somewhere to the effect that one single way to change parameters should be made "canoical" and the other deprecated. This would greatly help the implementation of UWS generic clients. I still believe this would have been a good thing, though I admit introducing such a change during TCG review might be inapproriate.

2.2.1 .../quote now returns an ISO timestamp. This differs from REC-1.0, where it returns an integer number of seconds. Since I'm not aware of any client making good use of quote and few servers actually returning something meaningful there, I'd not consider this serious, 2.0-justifying breaking the standard, but it should certainly be noted in the change log. Incidentally, I'd rather reference DALI here, since the ISO standard actually allows an almost infinite number of serialisations, so standards trolls might be tempted to play games with the current formulation.

On p. 11 it says "This URI should be given as the access URL in the UWS registration," but then no further comment is made on how such a registration might actually look like. To foster registration of UWS services, I would volunteer to contribute an informative appendix with a sample registry record for a UWS service.

Relatedly, I should have intervened a bit earlier because there's actually stuff a special UWS capability should note (support for ARCHIVED, perhaps limits on blocking times, plus there's limits on execution duration and retention time as in TAPRegExt...). Actually, I wonder if we shouldn't extend TAPRegExt for that purpose. This would mean adding a couple of (optional) elements and making language and outputFormat optional. If you agree that'd be a good idea unless bad things result, all that it would take was "UWS services SHOULD use TAPRegExt 1.1 capabilities as those become available" somewhere near the existing registration language.

Has anyone already written a StandardsRegExt record for this standard? If not and no one else steps forward, I volunteer to go ahead and draft one.

Like Mark, I'd like to see a bit more details on the implementations and where they actually run. I'd also feel much better if the blocking mode were demonstrated with an actual client^W^W^WTOPCAT.

-- MarkusDemleitner - 2015-09-25

Semantics Working Group ( _Mireille Louys, Alberto Accomazzi )

Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )

Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick )

Data Curation & Preservation Interest Group ( Françoise Genova )

Operations Interest Group ( _Tom McGlynn, Mark Taylor )

As I noted above, this is basically a good document.

However, the Implementations/Validators part of the RFC page still looks to me rather underpopulated for a document in TCG review.

Three implementations are listed, but the implementation authors have not as requested filled in details, so it's not clear what the status of these implementations is, or where to go to see the code or running UWS 1.1 services. In particular, it's important that the implementations conform to the version determination arrangements described in sec 2.2.1.1, which have been revised during this RFC period.

There are no validators listed that test UWS 1.1-specific functionality, partly (in taplint's case at least) because there are no services to validate (if service providers want to blame validators for this chicken-and-egg problem - OK then let me know and I'll write the validation code first).

Since TAP is currently the most important(?) UWS client standard, before recommending approval I'd really like to see a TAP service whose async endpoint implements the new UWS features (I'm especially personally interested in the WAIT blocking in accordance with sections 2.2.1.1 and 2.2.1.2). Possibly there is one out there, but (despite my request in the Implementations Validators section) it's not clear from the information on this page where it can be found. If no TAP service can be persuaded to do that, I'd at least like to see a running UWS service implementing the new features.

-- MarkTaylor - 2015-09-11

Knowledge Discovery in Databases Interest Group ( George Djorgovski )

Theory Interest Group ( _Franck Le Petit, Carlos Rodrigo )

Standards and Processes Committee ( Françoise Genova)

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->