Table Access Protocol 1.1 Proposed Recommendation: Request for CommentsTOC The table access protocol (TAP) defines a service protocol for accessing general table data, including astronomical catalogs as well as general database tables. Access is provided for both database and table metadata as well as for actual table data. This is the proposed revision (version 1.1) of the standard, including changes from community feedback, clarifications and a few optional features. Latest version of IVOASTANDARD can be found at: (This RFC was
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Added: | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
> > | A chronological view on the authenticated endpoints in TAP-1.1 (up to the 2018 Fall Interop in College Park) is available here. This chronology was prepared for and then linked into this mail thread summarizing the outcome from College Park Interop's discussion. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Reference Interoperable Implementations and ValidatorsFrom Pat DowlerThe primary CADC TAP service (http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/tap/) is a complete TAP-1.1 implementation. We have implemented TAP-1.1 changes over the whole time that the new version was developed, but I would like to draw attention to these in particular:
From Markus DemleitnerIn Release 1.1, I believe DaCHS (http://soft.g-vo.org/DaCHS) implements all aspects of TAP 1.1 (though of course I also expect that a validator will uncover a couple of issues, which we'll fix as fast as possible). An endpoint to try things out is http://dc.g-vo.org/tap, which also has interoperated nicely with TAP 1.0 clients so far. Pat's notes on tap_schema datatypes, adql:Region, and DALI 1.1 geometries apply here, as well. -- MarkusDemleitner - 2018-04-23From Mark TaylorI have done client implementation, and partial validator updates, for the features in TAP 1.1 that have changed since TAP 1.0. In particular the authenticated service declaration/discovery described in the example in section 2.4 has been tested and covered by validator updates. It works. Some other TAP 1.1-specific validator updates have been made, including tests of required foreign key declarations, "size"/arraysize consistency, removed REQUEST=doQuery parameters. Some other features, including timestamp comparisons, table column name upload behaviour, use of xtype and of DALI-1.1 datatypes, are not tested in taplint yet. In due course I hope to get round to adding validator coverage for these featues. There are just a few changes to the non-validator clients apart from authentication: avoid REQUEST=doQuery, report on new TAP_SCHEMA columns in metadata browser. I don't currently forsee other client code changes elsewhere in TOPCAT/STILTS related to TAP 1.1-specific features. The updated client code (TOPCAT, taplint, other TAP-sensitive STILTS commands) can currently be found at ftp://andromeda.star.bris.ac.uk/pub/star/topcat/pre/topcat-full_tap11.jar. Note however this implementation is provisional, and may be changed or withdrawn depending on future TAP 1.1 discussions. -- MarkTaylor - 2018-10-22Comments from the IVOA Community during RFC/TCG review period: 2018-04-23 - 2018-11-23The comments from the TCG members during the RFC/TCG review should be included in the next section. *Please note a new revision of the PR was released on October 24.* The RFC has been twice extended (see PR details at start of page). In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this documentComments from TCG member during the RFC/TCG Review Period: 2018-04-23 - 2018-11-23*Please note a new revision of the PR was released on October 24.* WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment. IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.TCG Chair & Vice ChairApplications Working GroupData Access Layer Working GroupData Model Working GroupSome minimal comments from my read through.. Laurent may have some more technical comments as well. Section 1.2.6: 'simple key-vale' -> 'simple key-value' Section 2.4: 'the mandatory chilsd' -> 'the mandatory child' Section 2.7: 'Thus, for the asynchronous TAP queries, the parameter requirements must be satisfied (and errors returned if not) only when the query is run in (in the sense of UWS job execution).' Something seems missing.. the query mode maybe? Section 2.7: ' may be created with with no parameters' == Extra 'with' Section 5.1.3: 'URL should lead to a an error' == Extra 'a'Grid & Web Services Working GroupRegistry Working GroupSemantics Working GroupData Curation & Preservation Interest GroupEducation Interest GroupKnowledge Discovery in Databases Interest GroupSolar System Interest GroupTheory Interest GroupTime Domain Interest GroupOperationsComments from Mark Taylor (2018-06-12)I have some concerns relating to the approach taken in Section 2.4 to declaring service endpoints for a variety of security methods. The UWSRegExt extension referenced here has not been previously published, but that is now being addressed by the draft UWSRegExt Note currently in circulation. Some edits to section 2.4 will be required when the content of that document has been agreed. But even with a well-defined UWSRegExt, from a client point of view I see some difficulties with using a capabilities document like the one shown in Section 2.4 of PR-TAP-20180416. The proposal there is fine for clients that know they want to find, say, a TAP sync endpoint using tls-with-certificate authentication; I believe that is the usage scenario for which this arrangement has been tested at CADC. However, a client like TOPCAT has different service discovery requirements: it needs to find a bundle of related endpoints (sync, async, tables, capabilities, examples) which together constitute a TAP service. From that point of view the mixed bag of interfaces-with-securityMethods proposed here is difficult to interpret. I made a short presentation in Shanghai complaining about this. To me, it would make more sense to declare a separate base URL for each securityMethod, with the subordinate endpoints (/sync, /async, /tables etc) hanging off each one in the standard TAP-1.0 pattern. I know Pat is not keen on that, and I admit I'm not sure if it could be made to fit easily with the existing VOResource/VOSI-Capabilities framework. Response: These items from my DAL mailing list post of 2018-06-13 (this thread -- note by MarcoMolinaro) is the primary obstacle to this approach: 2. DALI-1.1 specifies (Section 2) that all endpoints in a service must be siblings of the VOSI-capabilities resource (except VOSI-availability); further, DALI also specifies that the capabilities resource must be named /capabilities (relative to the base). 3. VOSI-1.1 specifies that VOSI-capabilities must be provided with anonymous access and such capabilities must not include a securityMethod So, you have a base URL and a single anonymous child {baseURL}/capabilities and all the TAP endpoints (eg sync and async query execution) have to be siblings of that (i.e. direct children of {baseURL}). That is just a flat list of interfaces and can't be presented as a bundle with current VOSI. Somewhat ironically, before that DALI-1.1 restriction the CADC TAP service used to place the username/password auth endpoints under a "subdirectory", so it used to be /tap/auth/sync and /tap/auth/async and now it is /tap/auth-sync and /tap/auth-async so everything is a sibling of /tap/capabilities... that is quite natural from a service deployment point of view and more or less conforms to the bundle and defaults approach described above. But the community thought it was a good idea to be able to navigate(?) from any endpoint to the capabilities endpoint. Either (flat or bundle) works fine if you use the capabilities the way that our cadc-registry client lib does (it is just cosmetic). But: the only way to go the bundle way is for TAP-1.1 to violate DALI-1.1 and/or retract that bit from DALI (admit mistake; issue erratum). -- PatrickDowler - 2018-08-23 The alternative that does not break any rules and does not require any UWSRegExt is
A TAP service consists of a related set of resource types, as listed in [the table in Section 2] : TAP-sync, TAP-async, VOSI-capabilities, VOSI-availability, and optionally VOSI-tables and DALI-examples. In some cases a client needs to identify a related "bundle" of endpoints providing one each of some or all of these resources in order to orchestrate its interaction with the TAP service. In TAP 1.0, this identification was straightforward, since only one instance of each resource type existed, and these were present at fixed locations relative to the TAP service base URL. In TAP 1.1, the capabilities document may declare multiple variants for some or all of these resource types, for instance to support different authentication contexts, and identifying the related bundle of endpoints that should be used given particular authentication requirements is therefore no longer trivial. Clients are free to combine the declared interfaces in any way they see fit, but the recommended way to interpret a TAP capabilities document as a list of TAP endpoint bundles is as follows:That text is my attempt to make explicit what I think are the implicit assumptions about how the services described by the capabilities example in the PR are supposed to fit together. If I have misread those, it may need some adjustment. Response: Document updated with approximately the above descriptive text. -- PatrickDowler - 2018-08-29 Note however that this text does not address the issue of multiple interface elements within a capability element having the same xsi:type but different version attributes and (the same? different?) securityMethods, which complicates all this further - do we expect to see those?. I can think of ways to adjust the text above to accommodate those possibilities, but it starts to get pretty messy. As far as I know(?), CADC has the only TAP services and the only TAP clients to make use of the scheme described in this PR to work with non-trivial authentication requirements, and I worry that the details of the scheme may be driven by their particular requirements or implementation details. I might be wrong about that, but I would very much like to see other TAP services that need authentication (LSST, IRSA, MAST, CSIRO?) look closely at this proposal, and preferably implement against it, to see whether it fits with their requirements. If it does, I will do my best to get TOPCAT working smoothly with it. Other than that, this document mostly looks good to me. I have some minor comments:
Further Comments from Mark Taylor, regarding PR-TAP-1.1-20180830 (2018-09-12)Note: the following comments concern PR-TAP-1.1-20180830 as written. I'm still arguing that this approach may not be the best one; if the discussion in this thread succeeds in persuading the authors to change the structure of the capabilities document, some of the following comments may be invalidated. Thank you for including my text concerning bundles of services. Given this I think I will be able to write client code to work with this style of capabilities document. However, I still find it more unwieldy than the old services-at- well-known-relative-locations arrangement, and as I hinted in my earlier comments, I can still see a number of unanswered questions that this prescription leaves. Some of these are particularly from the point of view of a validator: it's not clear how much of the content of the example /capabilities document in section 2.4 is intended to be normative (the only way you're allowed to do it) and how much is just personal taste or dictated by the details of the particular usage scenario being illustrated here.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Changed: | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
< < | Response: The base url with type ParamHTTP services to let TAP-1.0 client find the base url and use the service. The explicit anonymous sync and async endpoints can be included and this allows a generic capabilities-reading client that doesn't know anyhting about TAP naming conventions to correctly find endpoint URLs (this is how the cadc-registry java lib works -- without TAP knowledge). So, those can be included, but I didn't think about whether they must or not. If we didn't need backwards compat it would be must and not redundant. Every other service we have explicitly defines anon interfaces and it is only TAP-1.0 compat (where we mistakenly put two endpoints under one interface because of the whole sync vs async battle and not wanting to scare anyone) so I think must is the right solution and should or must include the base ParamHTTP for backwards compat. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
> > | Response: The base url with type ParamHTTP services to let TAP-1.0 client find the base url and use the service. The explicit anonymous sync and async endpoints can be included and this allows a generic capabilities-reading client that doesn't know anyhting about TAP naming conventions to correctly find endpoint URLs (this is how the cadc-registry java lib works -- without TAP knowledge). So, those can be included, but I didn't think about whether they must or not. If we didn't need backwards compat it would be must and not | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Deleted: | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
< < |
Standards and Processes CommitteeTCG Vote : 2018-10-25 - 2018-12-07If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date. Voting period has been extended twice due to new PRs released on August 30 and October 24.
<--
<--
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|