VOSI v1.1 Proposed Recommendation: Request for Comments

Discussion page for the IVOA VOSI 1.1 Proposed Recommendation.

The lastest VOSI 1.1 Proposed Recommendation can be found at:

• http://www.ivoa.net/documents/VOSI/20161214/

Comments from the IVOA Community RFC period: 2015-04-15 - 2015-05-27

Comments from Mark Taylor

Basically OK. I have a couple of suggestions for change/improvement:

• Schema versioning:
  • Can this standard make use of the XML Schema Versioning Note? In particular, I'd like to see a version="1.1" attribute on tableset elements returned from tables-1.1 services.
    • Response by BrianMajor (2016-12-14): Good point. I've added the attribute version=1.1 in all top-level documents.
• detail parameter:
  • I would like to see, alongside detail=min , a detail=max option. Sometimes the client knows whether it wants/needs the whole tableset at once - it may be that a columnless tableset is no use to it, in which case the current proposal may end up returning a (possibly large) document which it doesn't want. However services (like VizieR) can't be forced to return the whole tableset if it's huge. So I'd suggest:
    • [no detail specification] : service decides whether to include column detail or not (unchanged)
    • ?detail=min : service must not include column detail (unchanged)
    • ?detail=max : service may either include all column detail, or fail with a 403 Forbidden
  • I would make use in topcat of a detail=max option along these lines if it existed.
  • I still (cf my and Markus's email comments 16 Feb) don't see much value in the 303 redirect.
• Response by BrianMajor (2016-12-14): All these points have been addressed as discussed at the Cape Town interop.

And some minor comments on the text:

• Sec 3: a rogue backslash is visible in the output of the verbatim'd example element specification http://some.name.space\#elementName
  • Response by BrianMajor (2016-12-14): Corrected.
• Sec 3.1: in the PDF version, the URL http://www.ivoa.net/xml/VOResource/v1.0 extends off the side of the page (a couple of words elsewhere in the document hang a long way off the right as well).
  • Response by BrianMajor (2016-12-14): Addressed.
• Sec 4: The table mentions both ivo://ivoa.net/std/VOSI#tables and ivo://ivoa.net/std/VOSI#tables-1.1 standardID values, but the text about the tables endpoint just says "... the value of the standardID attribute must be ivo://ivoa.net/std/VOSI#tables ".
  • Response by BrianMajor (2016-12-14): Thanks--the text now says it must be the 1.0 or 1.1 tables URI.
• Sec 4: "(see example given in section 2.1)" - there is no section 2.1.
  • Response by BrianMajor (2016-12-14): Fixed.
• Sec 5: Example 1 contains xsi:type="vs:ParamHTTP" attributes, where the vs namespace is set as xmlns:vs="http://www.ivoa.net/xml/VODataService/v1.0" . But the last sentence in section 4 says "this document recommends using the http://www.ivoa.net/xml/VODataService/v1.1#ParamHTTP interface type". Is that an inconsistency? Examples 2, 3 and 4 do use the v1.1 namespace (with vod: prefix).
  • Response by BrianMajor (2016-12-14): Yes, I think so. All references to VODataService are to v1.1 now. Also, I've updated all examples to use the vs namespace as per the recommendation in VODataService "Use of the vs prefix in compliant instance documents is strongly recommended".

-- MarkTaylor - 2016-04-19

• Response by BrianMajor (2016-12-14): Thanks for your detailed review.

Implementation and Validators

Please note your implementation or validator in this section.

TOPCAT

A pre-release version of TOPCAT that uses a VOSI-1.1-style tables endpoint (PR-20160413) is available at ftp://andromeda.star.bris.ac.uk/pub/star/topcat/pre/topcat-full_vosi11.jar. To use it, open the TAP window, point it at a VOSI-1.1 capable service (e.g. http://www.cadc.hia.nrc.gc.ca/tap) and select from the TAP|Metadata Acquisition menu either "VOSI11-1step" or "VOSI11-2step". It seems to work. -- MarkTaylor - 2016-04-19

• Updated for ?detail=min/max hints as agreed in Cape Town -- MarkTaylor - 2016-07-11
• Seems to work for the CADC TAP service detail=min/max implementation -- MarkTaylor - 2016-10-03

Comments from Tom McGlynn

General

This is a pretty simple standard, but I think it is presented in a way that makes it seem more mysterious and difficult than it really is.

After some standard filler in section 1, section 2 jumps into a discussion of the need to build this as a REST standard then in section 3 we talk about how we can get different kinds of metadata.

However we never say what we are talking about... I think this standard desperately needs a paragraph that says something like:

VOSI provides a standard way to get metadata from VO services which implement it. To use VOSI, a client must first determine a base [VOSI?] URL for the VO service. This is typically provided as a field in the registry entry associated with the service, but other mechanisms may also be used. In this doucument the we use

http:/base.host/base/url/

as the services base URL. The VOSI protocol defines a set of URLs that can be constructed from the base URL as described below and the required responses that must be provided when these URLs are invoked.

I'd then recast the GET examples as sothing like:

GET http://base.host/base/url/availability

where we typographically separate the base URL and the VOSI suffixes. The text would note this typographic convention.

There may be more complexity here than I realized (see below for my comment on mixing CGI and REST), but the current standard is very hard to understand because there is no root that ties it to the actual services.

With this small addition I think the standard would be much more transparent.

• Response by BrianMajor (2016-12-14): The very first paragraph of section 1 does a decent job of explaining that this specification defines a common way obtaining service metadata. However, I agree that the start of section 2 is abrupt. I have added a short introductory paragraph. As for the determination of URLs for the metadata endpoints, it is not as simple as looking for a constant string (such as 'availability') off a website base URL. Services are free to name these endpoints what they like. The discovery of the URLs for the different metadata reports comes from looking up the particular capability (even for the 'capability' endpoint!) from the services' registry entries.

Some technical issues

- What about https? Is this supported? All US Federal government sites are expected to go to HTTPS within the next year or so.

• Response by BrianMajor (2016-12-14): There is no specific mention of HTTP vs HTTPS in this document as it is up to each provider to define the endpoint URLs. I don't think mandating a certain protocol for service access should be a part of this document.

- For existing SIA and Cone search interfaces the base URL is often something like:

http://heasarc.gsfc.nasa.gov/cgi-bin/vo/cone/coneGet.pl?table=rosmaster&

If I want to this to support VOSI what is the proper URL to invoke? How do we composite the construction of VOSI URLs with both CGI and rest components? Can the VOSI base URL be different from the base URL used in the DAL query?

• Response by BrianMajor (2016-12-14):
  • CGI requests within a 'REST' framework are indeed compatible. REST is mainly mentioned to differentiate requests from SOAP web service requests, where service communication is done using XML documents rather than plain HTTP.
  • Yes, the endpoints can be completely different and are defined in the capabilities 'capability' of your service.

Validation

It seems to me that building a generic VOSI validator to handle just the capabilities and availability responses should be easy and should be required before passing to recommendation. Since I think only TAP services use the tables endpoint and that is supported by TOPCAT and TAPLint already, I don't think that a generic validator need consider that endpoint.

• Response by BrianMajor (2016-12-14): It looks like two validators are now available.

-- TomMcGlynn - 2016-05-10

• Response by BrianMajor (2016-12-14): Thanks for your review and comments.

Comments from TCG members during RFC period: 2015-04-15 - 2015-05-27

WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or do not approve of the Standard.

IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory.

TCG Chair & Vice Chair ( _Matthew Graham, Pat Dowler )

Applications Working Group ( _Pierre Fernique, Tom Donaldson )

I approve this document. -- PierreFernique - 2017-03-14

Data Access Layer Working Group ( François Bonnarel, Marco Molinaro )

We approve the document which is a clear and comprehensive specification.

In case the authors still do some little changes we make following optional suggestions

Section 3 = push 3.2 (non - service metadata) as 3. 4 (because it's some addition to "capability", "availaibility" and "tables")

Add an example to section 4 = in order to show how a capability element for VOSI endpoints look like in the registry.

-- FrancoisBonnarel - 2017-05-09

-- MarcoMolinaro - 2017-05-09

• Response: Francois, Marco: In the creation of the REC version of this document, I have moved section 3.2 to 3.4 as you suggest. However, even though it is a good idea, I will not add the example because I don't want to make a mistake at this point. -- BrianMajor - 2017-05-24

Data Model Working Group ( _Mark Cresitello-Dittmar, Laurent Michel )

I read through the 2016-12-14 version of the document and have no objections. I approve this document.

-- MarkCresitelloDittmar - 2017-01-27

Grid & Web Services Working Group ( Brian Major, Giuliano Taffoni )

I approve this document.

--IVOA.BrianMajor - 2016-12-14

Registry Working Group ( _Markus Demleitner, Theresa Dower )

While TomMcGlynn's comment on HTTP/HTTPS was addressed here, section 2 on Interface Bindings still seems to read to me as HTTP-specific. I'd still like to see the HTTP/S agnosticism spelled out if there's an opportunity, but I won't hold up the document on it. Everything looks good to me from a Registry standpoint.

-- TheresaDower - 2016-12-15

Semantics Working Group ( _Mireille Louys, Alberto Accomazzi )

I found no contradictory point with any of the Semantics standards, so I approve the document.

-- MireilleLouys - 2017-05-11

Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )

Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick )

Data Curation & Preservation Interest Group ( Françoise Genova )

Operations Interest Group ( _Tom McGlynn, Mark Taylor )

Knowledge Discovery in Databases Interest Group ( George Djorgovski )

Theory Interest Group ( _Franck Le Petit, Carlos Rodrigo )

Standards and Processes Committee ( Françoise Genova)