UDF catalogue Proposed Endorsed Note: Request for CommentsThis PEN proposes a mechanism for listing ADQL extension functions that are requried to work the same way across different data centers. It is a supplement to the ADQL standard, and currently there's text in ADQL 2.1 referencing it. The latest version of the UDF catalogue can be found at: http://ivoa.net/documents/udf-catalogue/ Version control: https://github.com/jontxu/udf-catalogueReference Interoperable ImplementationsAs to the endorsed content, there should always be at least two services carrying the functions. Regrettably, the required RegTAP res_detail table doesn't carry UDF names (there's always a use case exactly for the most exotic feature you can think of), so to find these services, you'll have to rely on volutary extra keys parsed by the services. The RegTAP service at http://dc.g-vo.org/tap parses the extra res_detail. Use a query likeselect access_url from rr.res_detail natural join rr.interface natural join rr.capability where detail_xpath='/capability/language/languageFeatures/feature/form' and detail_value like 'ivo_healpix_index%' and standard_id='ivo://ivoa.net/std/tap'Since people have asked for it, here's how to look at things for the healpix functions. With the above query, you will identify (at this point) 24 services carrying the functions; it's a bit tricky to see which really use different software (see these considerations over at Ops for more on this), but you can trust me that https://gaia.ari.uni-heidelberg.de/tap and http://dc.zah.uni-heidelberg.de/tap are different software (actually: I think VizieR can do it, too, it's just missing from its capabilities). So, fire up your TOPCAT and paste either URL into the TAP URL box in the TAP client. Then check out the PEN to find: While ADQL does not support standalone evaluation of functions, a query like SELECT TOP 1 <example> AS res FROM TAP_SCHEMA.tables will return one row with the function result for simple, non-aggregate func-Armed with this, try, for instance, SELECT TOP 1 ivo_healpix_index(0, 1, 1) AS res FROM TAP_SCHEMA.tables(from the first example in 2.1.1), and see that it actually returns 4. If some volunteer wrote up a little bash/stilts (or pyvo) script to automate that, I'd gladly include it in the spec archive. -- MarkusDemleitner - 2020-09-02 Implementations ValidatorsNot really applicable here. It might be nice to have a few standard queries per function with the intended results, but since ADQL doesn't have queries without from clauses and there is essentially no content in the underlying databases you can rely on, they're really hard to write. Perhaps a case for an ADQL extension: Facilitate writing tests?Comments from the IVOA Community during RFC/TCG review period: 2020-02-15 .. 2020-03-30The comments from the TCG members during the RFC/TCG review should be included in the next section. In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document
Comments from TCG member during the RFC/TCG Review Period: 2020-02-15 .. 2020-03-30WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment. IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.TCG Chair & Vice ChairApplications Working GroupData Access Layer Working GroupDAL is happy with the goal and main content of this Note and will be happy to support its Endorsement provided that the following comments are taken into account and fixed in the document.
Data Model Working Group | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Changed: | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
< < | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
> > |
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Added: | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
> > |
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Grid & Web Services Working GroupRegistry Working GroupI'm in general agreement with Ops that this is a good list to keep somewhere, but unsure whether the need for it to be continually updated works with or is in conflict with the process for a Note. I'll sign off if general consensus is to keep it in a Note by the time other WGs have weighed in. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Changed: | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
< < | I, frankly, am not happy about the sluggish RFC | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
> > | I, frankly, am not happy about the sluggish RFC either. But inventing a new process for the relatively minor thing of reviewing new interoperable ADQL UDFs just seems appears unproportional to me. And, of course, if you want a wide review of something, I doubt that changing the process will make much of a difference in terms of net agility. -- MarkusDemleitner - 2020-10-19 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Deleted: | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
< < | either. But inventing a new process for the relatively minor thing of reviewing new interoperable ADQL UDFs just seems appears unproportional to me. And, of course, if you want a wide review of something, I doubt that changing the process will make much of a difference in terms of net agility. -- MarkusDemleitner - 2020-10-19 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
With my TAP implementer hat on, I do find it useful to see what other data providers are calling their own support functions, and what functions already exist from standards other than ones we have implemented locally, as we add some UDFs to new and existing services. There are some UDF already available in MAST services, not all useful to include (especially under their current names; aliases may fix this), namely ones tightly coupled with other MAST interfaces, allowing for parallel queries to those in CASJOBS, etc. If we keep this a note, I may propose the inclusion of some generically useful existing UDFs from MAST in a future version, particularly UDFs for reference frame/scale conversion and the like.
-- TheresaDower - 2020-10-07
Semantics Working GroupThis Proposed Endorsed Note lists a series of User Defined Functions (UDF), which are now related to ADQL but, as it is recalled in the text, in future may not be naturally associated with a particular standard. For the identified functions, a human readable description is provided. A given description lists the set of input parameters (together with their type and physical meaning) and the output. The goal of the Parameter Description Language is to deal with these descriptions in a standard way (cf. section 2 of http://www.ivoa.net/documents/PDL/20140518/PR-PDL-1.0-20140518.pdf). Indeed one of the primary goals of PDL was to provide the community with a grammar allowing data/service providers to describe the parameters of their functions.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Changed: | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
< < | Carlo and I have discussed this a bit, and we | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
> > | Carlo and I have discussed this a bit, and we figured that while having a uniform system of describing interfaces is highly desirable and thus PDL should be adopted by standards requiring this kind of metadata, this EN probably is not a good place to begin that; for one, the signatures are already implemented and work well for the sort of weak typing we find in SQL. More importantly, the UDF catalogue simply gives descriptions as required by TAPRegExt, and hence PDL would need to be adopted there; adding the PDL descriptions in the UDF catalogue would then probably be relatively trivial. -- MarkusDemleitner - 2020-10-19 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Deleted: | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
< < | figured that while having a uniform system of describing interfaces is highly desirable and thus PDL should be adopted by standards requiring this kind of metadata, this EN probably is not a good place to begin that; for one, the signatures are already implemented and work well for the sort of weak typing we find in SQL. More importantly, the UDF catalogue simply gives descriptions as required by TAPRegExt, and hence PDL would need to be adopted there; adding the PDL descriptions in the UDF catalogue would then probably be relatively trivial. -- MarkusDemleitner - 2020-10-19 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Data Curation & Preservation Interest GroupEducation Interest GroupKnowledge Discovery Interest GroupSolar System Interest GroupTheory Interest GroupTime Domain Interest GroupOperationsGeneral comment: it is useful to have this information in a centralised form, and the Note is generally well-written and comprehensible. I am willing to Endorse it. However, although an EN is more lightweight than a Recommendation-track document, the endorsement process is still typically quite slow. An alternative would be to have something more informal such as a wiki page (it's worked quite well for SampMTypes, which has no enforced restrictions on editing, but has in practice been very stable). One or two specific comments on the text beyond those already noted:
Standards and Processes CommitteeTCG Vote : Vote_start_date - Vote_end_dateIf you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.
<--
|