Request for Comment: RegTAP v1.0

This document serves as the RFC center for the Proposed Recommendation entitled "Registry Relational Schema, Version 1.0". The version reviewed during the RFC can be found at http://www.ivoa.net/documents/RegTAP/20140227/index.html.

Additional discussion about any of the comments or responses can be conducted on the Resource Registry mailing list, registry@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Notes on Implementations and Validaters

A provisional validation suite as available at http://docs.g-vo.org/regtapval-2014-06.tar.gz

An implementation of this is available on the TAP services at http://dc.g-vo.org/tap and http://gavo.aip.de/tap (the basic underlying software is identical for those).

Another implementation (out of date with respect to XPath/utypes and some DB fields) is available as a TAP service at http://ia2-vo.oats.inaf.it:8080/registry . Even if not fully compliant, the IA2 team (VObs.it) plans to update it to the REC version and fix some other compliance problems to keep it at least as an independent mirror of the GAVO service. This implementation uses both a programming language (Java) and RDBMS (MySQL) different from the GAVO one.

Comments from the community

• Sample comment by WikiName
  • Response (by WikiName)

Comments from MarkTaylor

Great job Markus et al. A few minor comments:

• Sec 1.1: 'The entire standard is now known as "IVOA relational registry schema"' - the title of the document has "IVOA Registry Relational Schema" (words in a different order) - is this an inconsistency?
• Figure 1 caption: "(tagged with RegTAP)" - the box in the figure is labelled "Relational Registry" not "RegTAP", so I think this is wrong.
• Sec 4: The prefix->namespace table does not appear to be in any discernable order - order it alphabetically either by prefix or by URI?
• Sec 7.1: The description of short_name contains the text "Applications may use to refer to this resource in a compact display." - words missing?
• Sec 7.1: It would be useful for reference if the the information about field formatting presented in the text of this section could be repeated in the descriptions column of the table. Specifically:
  • content_level, content_type, rights: note values are hash-separated (this is already done for waveband)
  • creator_seq: note names are semicolon-separated
• Sec 7.7: "The table_column table models the content of VOResource's column element" - I might be wrong, but I think that should read "VODataService's column element".
• Sec 7.7: "hence, this column will contain one of NULL, vs:TAPType, vs:SimpleDataType and vs:VOTableType" - since these will be lowercased by the time they appear in the RegTAP table, would it be less confusing to mention them in lower case in this text?
• Sec 7.8: "analoguos" -> "analogous"; "adviced" -> "advised".
• Sec 7.9: In the param_description description, "column's contents" should read "parameter's contents".
• Sec 7.10: "resoure" - "resource".
• Sec 8: The first parameter of the ivo_hashlist_has UDF is declared with the name "haslist"; I think that should read "hashlist".
• Appendix A: "suffient" -> "sufficient".
• Appendix A: This is a long list, and the possibility of typos etc is not negligable. Would it be a good idea to add at the start that in the case of a discrepancy between the xpaths in this list and the XML schemas defined by the relevant standards, one or other of those should be taken as normative?
• Appendix A: I see a /capability/testQuery/pos/lat for SIA, but no corresponding /capability/testQuery/pos/long (SSAP has both).

Thanks for the useful comments. I hope they're all considered in volute rev. 2497. On the question of precedence of xpaths, I'm now declaring XML schema xpaths as normative and promise Errata in case contradictions go unnoticed. -- MarkusDemleitner - 2014-04-01

Late comments

Some technically late comments:

• section 4 seems to be missing the slap prefix and namespace from SimpleDALRegExt.
• while I appreciate (like even) the fact that all the reasonable queries can be performed by natural join, I really don't like multi-column primary and foreign keys. This is especially painful when the extra column to provide the necessary uniqueness is only a smallint that has to be carefully generated while inserting content. Have you considered using the concept of surrogate primary keys? Or at least allowing the implementer to chose the form and datatype of PKs (and hence FKs) so they could implement in that way? For a pure implementation of surrogate PK the ivoid columns would have to be renamed and the implementer would simply be required to provide keys suitable for a natural join. However, just allowing the implementer to chose the datatype for that uniqueness column would enable a variety of simpler techniques to populate it and make the natural joins work correctly. Renaming the ivoid column so it was not used in the natural join would not strictly be needed for correctness, just index efficiency (probably not a concern), so that could be kept as-is... is a known/predicable datatype actually needed?
• I now always use surrogate primary keys and recently moved to using 128-bit UUIDs, for which there are RDBMS functions (for default) and plenty of libraries with well-known algorithms... sure, 128-bit is complete overkill in the registry, but I'm not sure that smallint is actually enough for simple implementations like an RDBMS sequence or identity column (which I have come to despise for other reasons). With all those long varchar columns, the size of the uniqueness column isn't worth skimping on.

We did consider surrogate keys in the very earliest relational registry data models - see RelationalRegistryDM - However, it was decided that ease of writing queries was the primary goal of the design, and therefore it was useful to have the IVORN in as many tables as possible - so for certain queries fewer joins would be necessary. The smallint part of the composite key is generally easily generated on ingestion from the child index in its parent of the main modelled XML element for that table.

-- PaulHarrison - 2014-05-23

The slap prefix went in in Volute rev. 2645 -- thanks for catching this.

In Volute rev. 2646, I dropped requirements on the artificial foreign keys. Pat (and others interested in this) -- please review http://volute.googlecode.com/svn/trunk/projects/registry/regtap/RegTAP-fmt.html#primarykeys and the diff (this necessitated changes in several places). Does this fix things as far as you are concerned?

-- MarkusDemleitner - 2014-06-03

Yes, making the type for the _index columns implementation specific is sufficient. Thanks.

-- PatrickDowler - 2014-07-24

Comments from TCG members during the TCG Review Period:

WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham )

Applications Working Group ( _Mark Taylor, Pierre Fernique )

Data Access Layer Working Group ( Patrick Dowler, Mike Fitzpatrick )

Data Model Working Group ( _Jesus Salgado, Omar Laurino )

Grid & Web Services Working Group ( Andreas Wicenec, Andre Schaaff )

Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)

Semantics Working Group ( _Norman Gray, Mireille Louys)

Data Curation & Preservation Interest Group ( Alberto Accomazzi, Françoise Genova )

Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )

Knowledge Discovery in Databases Interest Group ( George Djorgovski )

Theory Interest Group ( _Franck Le Petit, Rick Wagner )

Time Domain Interest Group ( _Matthew Graham, John Swinbank )

Standards and Processes Committee ( Françoise Genova )

<!--
* Set ALLOWTOPICRENAME = TWikiAdminGroup
-->