(r7) RegTAP11RFC < IVOA

IVOA Web>IvoaTCG>2019ARoadmap>RegTAP11RFC (revision 7) (raw view)~~EditAttach~~
---+ RegTAP 1.1 Proposed Recommendation: Request for Comments

%TOC{depth="2"}%

---++ Summary

Registries provide a mechanism with which VO applications can discover and select resources - first and foremost data and services - that are relevant for a particular scientific problem. The RegTAP specification defines an interface for searching this resource metadata based on the IVOA's TAP protocol. It specifies a set of tables that comprise a useful subset of the information contained in the registry records, as well as the table's data content in terms of the XML VOResource data model. The general design of the system is geared towards allowing easy authoring of queries.

RegTAP 1.1, as a minor version increment over RegTAP 1.0, is backward compatible. The main differences between 1.1 and 1.0 are as follows:

   * new allowed res_detail values for testQueryStrings
   * more generic type information in schema tables
   * mapped terms in fields for dates and resource relationships to a vocabulary for DataCite compatibility
   * case-insensitive query support from ADQL 2.1
   * new columns for service mirrors, rights, and authentication and authorization of protected data and services
   * new table for alternate identifiers, supporting DOIs, ORCIDs, bibcodes, and future identification schemes
Latest version of RegTAP 1.1 can be found at:
   * http://www.ivoa.net/documents/RegTAP/20190529/index.html

---+++ Reference Interoperable Implementations

Two separate reference implementations of server-side architecture exist at GAVO (and other archives using GAVO's codebase) and STScI

   * GAVO: The machines serving http://reg.g-vo.org/tap implement all proposed changes (on top of DaCHS and Postgres). The Implementation Validators section has a list of queries exercising the new features.
   * STScI: http://vao.stsci.edu/regtap/tapservice.aspx implements all proposed changes minus optional detail insertions for test queries. Runs MSSQL server, C# ASP.NET web service code plus the ADQL translator from the VOLLT project (https://github.com/gmantele/vollt)
The TOPCAT client is aware of !RegTAP 1.1 features and is interoperable with both reference implementations

---+++ Implementations Validators

The !RegTAP validator (currently at http://docs.g-vo.org/regtap-val; should this move into the VCS for the standard?) has been updated to cover the main new features.

For reviewers, here's a set of RegTAP queries exercising the main user-visible new features (TAP access URLs above):

alt_identifiers &ndash; find VO resources and their titles that have DOIs:

<pre>select ivoid, res_title, alt_identifier
from rr.resource
natural join rr.alt_identifier
where alt_identifier like 'doi:%'
</pre>

rights_uri &ndash; find VO resources that have a CC license declared:

<pre>select ivoid, res_title, rights_uri
from rr.resource
where rights_uri like 'http://creativecommons.org/%'
</pre>

or find what license URIs are already in use:

<pre>select distinct rights_uri from rr.resource
</pre>

mirror_url &ndash; find mirrors available for a known access url (in this case, indicating that the service is available through https, too):

<pre>select ivoid, mirrors.mirror_url
from rr.interface as intfs
join rr.interface as mirrors
using (ivoid,intf_index, cap_index)
where intfs.access_url='http://dc.zah.uni-heidelberg.de/antares/q/cone/form'
</pre>

authenticated_only &ndash; find resources unavailable without authentication (note that we do not claim that's enough to actually operate them; the use case at this point is filtering them out with a view to a VO that has more of them):

<pre>select distinct ivoid from rr.interface where authenticated_only=1
</pre>

vocabulary mapping &ndash; use just a single term to find out services of data collections:

<pre>select res_title
from rr.resource as res
natural join rr.relationship as rel
where relationship_type='isservedby'
  and rel.related_id='ivo://nasa.heasarc/services/xamin'
</pre>

---
---

---++ Comments from the IVOA Community during RFC/TCG review period: 2019-06-15 - 2019-07-31

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

   * Sample comment by TWiki.WikiName 
      * Response (by TWiki.WikiName)

---
---

---++ Comments from TCG member during the RFC/TCG Review Period: TCG_start_date - TCG_end_date

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

---+++ TCG Chair & Vice Chair

---+++ [[IvoaApplications][Applications Working Group]]

---+++ [[IvoaDAL][Data Access Layer Working Group]]

---+++ [[IvoaDataModel][Data Model Working Group]]

---+++ [[IvoaGridAndWebServices][Grid & Web Services Working Group]]

---+++ [[IvoaResReg][Registry Working Group]]

---+++ [[IvoaSemantics][Semantics Working Group]]

---+++ [[IvoaDCP][Data Curation & Preservation Interest Group]]

---+++ [[IvoaTheory][Education Interest Group]]

---+++ [[IvoaKDD][Knowledge Discovery Interest Group]]

---+++ [[IvoaSS][Solar System Interest Group]]

---+++ [[IvoaTheory][Theory Interest Group]]

---+++ [[IvoaVOEvent][Time Domain Interest Group]]

---+++ [[IvoaOps][Operations]]

   * As of 2019-06-13, the "latest" version pointed at from this RFC page is http://ivoa.net/documents/RegTAP/20190503/index.html, but the current latest version is actually http://www.ivoa.net/documents/RegTAP/20190529/index.html. It's 20190529 that I'm commenting on here.
   * Fig 1: the caption says _'(tagged with "Relational Registry")'_ . I can't work out what this note means - should it say something like _'(tagged as "RegTAP")'_ ?
   * Appendix D: In this version (20190529) all the changes since REC-1.0 have been condensed into a single section (D.1) to avoid extra content that only serves historical interest of VO archaeologists; I think this is good practice. By the same token, does it makes sense to delete all the sections (D.2-D.9 in 20190529) that detail how it got to REC-1.0? As an issue affecting preparation of all REC-track documents, this is probably something which also ought to be discussed at TCG or SDP level.
   * Appendix D.1: One change in the text which I don't see noted here is changes to the standard_id constraints in the example queries from section 10, e.g. 10.3 in REC-1.0 has =WHERE standard_id='ivo://ivoa.net/std/sia'= , and in PR-1.1-20190529 has =WHERE standard_id LIKE 'ivo://ivoa.net/std/sia%'= . I appreciate this may be a result of changes elsewhere in the standards landscape rather than in !RegTAP itself, but it would be useful for people making use of these example queries if the change log summarised what's changed in recommended practice since REC-RegTAP-1.0. (There's also a correction to the argument order in the ivo_hashlist_has invocation in section 10.3 which should probably be logged for completeness).
   * This RFC page says _"The TOPCAT client is aware of !RegTAP 1.1 features and is interoperable with both reference implementations"_ . Yes it is interoperable, because of the 1.1-&gt;1.0 backward compatibility. However, can you refresh my memory about what 1.1 features it's aware of? I'm not denying it! Just can't remember what I may have claimed.
   * Plus a few typos: 
      * Sec 4.1: _"whereever"_ -&gt; _"wherever"_
      * Sec 6: _"specifiation"_ -&gt; _"specification"_
      * Sec 8.1, vr:organisation: _"is to be references by IVOID"_ -&gt; _"is to be referenced by IVOID"_ ?
      * Sec 8.10: _"slight denormalization of the vr:Relationship type: Whereas..."_ -&gt; _"slight denormalization of the vr:Relationship type: whereas..."_ ?
-- IVOA.MarkTaylor - 2019-06-13

---+++ [[IvoaStdsDocsProc][Standards and Processes Committee]]

---

---++ TCG Vote : Vote_start_date - Vote_end_date

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

| Group | Yes | No | Abstain | Comments |
| TCG | | | | |
| Apps | | | | |
| DAL | | | | |
| DM | | | | |
| GWS | | | | |
| Registry | | | | |
| Semantics | | | | |
| DCP | | | | |
| KDIG | | | | |
| SSIG | | | | |
| Theory | | | | |
| TD | | | | |
| Ops | | | | |
| &lt;nop&gt;StdProc | | | | |
Topic revision: r7 - 2019-07-24 - TheresaDower
IVOA
Log in or Register
IVOA.net
Wiki Home
WebChanges
WebTopicList
WebStatistics
Twiki Meta & Help
IVOA
Know
Main
Sandbox
TWiki
TWiki intro
TWiki tutorial
User registration
Notify me
Working Groups
Interest Groups
Committees
Stds&Procs
www.ivoa.net
Documents
Events
Members
XML Schema