RegTAP 1.2 Proposed Recommendation: Request for Comments

RegTAP, formally known as the IVOA Registry Relational Schema, is the standard way for clients to query the VO Registry. Version 1.2 adds tables to give the coverage in space, time, and spectrum and a tap_table view intended to replace GloTS. To make use of these features, we require a few optional ADQL features and the extra UDF ivo_interval_overlaps.

Latest version of RegTAP can be found at:

• https://ivoa.net/documents/RegTAP/20240124/
• Bleeding edge versions during RFC are found at https://docs.g-vo.org/RegTAP.pdf; prefer these for your review reads, or, even better, build the document from the sources yourself. See https://github.com/ivoa-std/RegTAP and feel free to open bugs or pull requests there at any time.

RegTAP is a fairly long standard. For review, it is probably advisable to only skim the unchanged text. For the purposes of this RFC, it is probably sufficient to closely inspect sects. 4.5, 7, 8.15 through 8.18, 9, and 10 (in particular 10.13). See also appendix E.

Reference Interoperable Implementations

Server-side implementations exist in the TAP services at http://reg.g-vo.org/tap and https://registry.euro-vo.org/regtap/tap.

Client-side code exploiting the new spatial tables is present in pyVO; WIRR also has constraints on coverage in space, time, and spectrum.

Support for tap_tables is less common; however, once the data providers fix their metadata records, the content of tap_tables essentially is equivalent to what we already have in GloTS, and hence the editor would argue that all clients using GloTS (e.g., TOPCAT) count as reference implementations of tap_tables.

A prerelease version of TOPCAT available at https://www.star.bristol.ac.uk/mbt/releases/topcat/pre/topcat-full_regtap12.jar makes use of rr.tap_tables for service discovery: in the TAP window the TAP|Service Discovery submenu has an option "RegTAP 1.2". If this is selected (instead of the default option "GloTS") then searching for services in the "By Table Properties" tab uses the rr.tap_tables table from GAVO DC rather than GloTS. -- MarkTaylor - 2024-02-28

Implementations Validators

A validator is part of the source distribution at https://github.com/ivoa-std/RegTAP. See the validator subdirectory. Note that it requires a registry seeded with the data provided, and hence the suite can only be used by the service operators.

---

---

Comments from the IVOA Community during RFC/TCG review period: 2024-02-01 - 2024-03-20

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Comments by MarkTaylor

This revised standard looks in quite good shape. I have a few minor comments.

• Section 1: "The simplification yields a schema with 14 tables." That's 18 tables now.
• Section 3: "In the tables of columns given below, the X_index columns have '(key)' given for type." In fact they don't (and did not at RegTAP 1.1), the tables in Section 8 list them all with type "integer". That should be changed.
• Section 4.3: "RegTAP 1.0 required that most columns containing values not usually intended for display to be converted to lower case on ingestion." Remove "that" or "to".
• Section 4.3: "When matching against these, queries should use case-insensitive matching, for which this specification offers the ivo_nocasematch user defined function. ADQL 2.1 has an ILIKE operator, which may be used instead." Since RegTAP 1.2 requires ILIKE, this could probably be tidied up a bit.
• Section 4.4: "Note that with VOTable 1.3, non-ASCII in char-typed fields, while supported by most clients in TABLEDATA serialization, is technically illegal..." The reference to VOTable 1.3 should probably say something like "at least up to VOTable 1.5". You may wish to reconsider the optimistic tone of the comment about future VOTable versions later in that paragraph.
• Section 9: "significanly" -> "significantly"
• Section 9.1: The language feature type for COALESCE is ivo://ivoa.net/std/tapregext#features-adql-conditional not ivo://ivoa.net/std/TAPRegExt#features-adql-type
• Section 10.1: "the new authenticated_only column" - "new" should maybe be replaced with "new at RegTAP 1.1" or similar.
• Section 18.16, 17: I tend to agree with Anne that the units for the temporal and spectral values could use some minimal explanation, given that this document may be used for reference by people trying to make queries against these tables. I don't think it would hurt to give a non-normative clue somewhere about what these values are. Perhaps tabulated column descriptions could say "Lower/Upper limit in MJD of a time interval..." and "Lower/Upper limit in Joules of a messenger energy...".
• Various features that are standardised in ADQL 2.1 are mentioned here. Would it make sense to require that RegTAP services are ADQL 2.1 compliant?

I have also added an entry to the Implementations section above.

-- MarkTaylor - 2024-02-28

---

---

Comments from TCG members during the RFC/TCG Review Period: 2024-02-01 - 2024-03-20

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

Data Access Layer Working Group

Data Model Working Group

Grid & Web Services Working Group

Registry Working Group

Semantics Working Group

Data Curation & Preservation Interest Group

Education Interest Group

Knowledge Discovery Interest Group

Operations Interest Group

Radio Astronomy Interest Group

Solar System Interest Group

I focused on the changes documented from the last version. Please note that I have yet to memorize the contents of the entire IVOA document set. Mea culpa.

• Thanks for your detailed review. Would you have a look at https://github.com/ivoa-std/RegTAP/pull/12 to see whether I have addressed all the points I do not otherwise comment on below in there (commit e308e0cd at this point) -- MarkusDemleitner - 2024-02-27

Typo: Heading 9.1: "requried" should be "required" (this also appears in the Table of Contents, of course)

Chapter 4, headings (all): In every other section of the document, section headings are in title case; in this chapter, section headings have only the first word capitalized. This should be made consistent throughout the document.

Page 11, Section 4.5 (Vocabulary Considerations), paragraph 1: The last sentence (At the time of this writing..) states that some document containing needed definitions is still in "Working Draft" state. The only reference corresponding to the citation "Demleitner (2019)" in the reference list is to version 2.0 of the "Vocabularies in the VO" recommendation. Version 2.0 was accepted in May 2021, and then superseded by version 2.1 in February 2023. So the rationale in this paragraph is no longer valid. How does this affect the requirements of this recommendation?

Page 13, Table 1: The table lists two versions of the VODataService recommendation explicitly (versions 1.0 and 1.1) but not the current version (1.2). Why not? What is the point of including minor versions in this list? I would expect that the "v1.x" notation would be widely understood. I did not check the version list for the other standard mentioned that have since been updated, but someone should.

• That's a long and sad story. It was a bad mistake to have minor versions in the namespace URIs, and this now confuses everyone. See https://ivoa.net/documents/Notes/XMLVers for the background. -- MarkusDemleitner - 2024-02-27

Page 19, "vs:datacollection": A reference is made to this resource type being included in VODataResource v1.1 for a specific purpose, but recommends that other resources be used. It would be appropriate, and perhaps compelling, to note that this resource type is, in fact, deprecated in VODataResource v1.2 (or earlier?) and should not be used.

Page 24, Section 8.6 (The res_table Table), paragraph 2: This paragraph makes repeated reference to "VODataSevice v1.0". As far as I can tell, this document is no longer accessible. It should be, for provenance at the least. If this document is lost or otherwise unavailable, the information that an implementor would need in order to follow the instructions in this paragraph needs to be supplied in some other form. Otherwise the requirement is unenforceable and somewhat ridiculous.

• Interesting... you are right, there is no v1.0 specification on the document repository (but the schema is still there: https://ivoa.net/xml/index.html). Digging out the 1.0 internal working draft might be a service to the community at large, but I frankly cannot see myself even trying – I've not been part of the VO back then. Would it help you if I linked to the v1.0 schema? -- MarkusDemleitner - 2024-02-27
• Well... I've made a quick census and I've convinced myself we ought to finally update or remove the few records that still use VODataService 1.0 (the schema). Hence, there's now https://github.com/ivoa-std/RegTAP/pull/16 -- MarkusDemleitner - 2024-03-05

Page 25, paragraph 1 (The table should have...): Suggest adding "also" to the final sentence ("Since table_utype is used in data discovery, it should also be indexed."). This is primarily to catch the attention of people skimming the document for workload estimation.

Page 26, paragraph 1 (The table_column table...): Should the mention of VODataService 1.1 be replaced with the current version (1.2)?

Page 35, paragraph 2 (The details of how the MOC-valued...): First, DALI 1.2 was accepted in July 2023, so the future tense is no longer appropriate. Second, I do not see that that document does provide the information indicated (how MOC coverage will be entered, specifically). This may be my misunderstanding, but I see no indication other than "It's a string value" in DALI 1.2, which falls far short of what I would need to know as an implementor. Should this instead reference some specific section of the MOC standard? If not, then some additional explanation of what is expected in this string field is in order.

• Hu... No, for all I know DALI 1.2 is still under review – we don't even have a PR, have we? Anyway, I have reformatted that part a bit. -- MarkusDemleitner - 2024-02-27

Page 35, last sentence (The rows for time_start and...): The statement that these columns "MUST have 'd' in their unit column" is completely mysterious to me. Why? What unit is "d"? What document would I have to read to translate 'd' into something meaningful? And why on earth would you require a potential implementor to chase down some other document to find the word that would have made this sentence sensible?

Page 36, Section 8.17, last sentence (The rows for spectral_start and...): Previous rant | sed 's/d/J/g'.

• That's basic VOUnits and TAP. This may look a bit odd in this context, but unless you insist I'd rather not quote these two standards in this particular place; I'd then logically have to explain quite a bit more in many other places, too -- MarkusDemleitner - 2024-02-27.

Typo: Page 36, Section 8.18, paragraph1: The second m-dash should not be followed by a comma.

Page 36, Section 8.18, most of it: I'm having difficulty understanding what the bottom line is for the description in this section. Some reasons:

• There is a reference to "version 1.2", but I'm not clear on which of the various recommendations in play here that version attaches to.
• In the numbered list, items 1, 2, and 4 are criteria that seem to depend on intrinsic properties of the table - things I would expect to find within a single row, although that is NOT how they are presented; while item 3 is a criterion based on a relationship between rows.
• Consequently, and for other more grammatical reasons, I cannot confidently interpret the sentence that begins with "Hence" and ends with the double full stop at the end of item 4.

I was going to offer a suggestion for rewriting, but I couldn't untangle it enough to make a decent effort. If you can explain it to me, I'd be happy to help wordsmith.

• Thanks for the offer. This is again patching up things we've messed up in the past. For the background, you can go back to 2015: http://wiki.ivoa.net/internal/IVOA/InteropOct2015Reg/auxcap.pdf. For now, I'd hope Mark Taylor helps cleaning this up -- MarkusDemleitner - 2024-02-27.

Page 39, Chapter 10, paragraph 3: There is a reference to "today's (2019) registry". When I checked my calendar, "today" was in 2024. If the intention is to reference a specific version of a recommendation, then please do so formally. Relative dates like "today" are problematic "tomorrow".

Page 46, "accessURL (!)": The word "legacy" is not sufficiently specific. Ideally, this should indicate a specific recommendation and version number, and also whether that is the point where some significant thing was deprecated, superseded, or no longer supported. Since this is listed as a required table entry, this needs to be clarified. Also, it seems odd that a section called "XPAths for res_detail" indicates that this "accessURL" is required, but then immediately states that is must NOT be in res_detail. So why is it listed as required for res_detail?

Page 52, Appendix C, last line of code: There appears to be mangled text on the last line, which reads, "SELECT * FROM fromes) q)". At the very least, the parentheses in the entire code block are unbalanced.

• Thanks again! -- MarkusDemleitner - 2024-02-27

-- Anne Raugh, 2024-02-26

Theory Interest Group

Time Domain Interest Group

Standards and Processes Committee

---

TCG Vote : Bambi eyes - more Bambi eyes

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group	Yes	No	Abstain	Comments
TCG
Apps
DAL
DM
GWS
Registry
Semantics
DCP
Edu
KDIG
Ops
Radio
SSIG
Theory
TD
<nop>StdProc