IVOA Interop April 2022
Operations IG 1: Operations and Standards
Date: Tues April 26 2022, 22:00 UTC
Participants: (35)
Markus Demleitner: Requirements and Validators
LM (too tired to attend) : You are absolutely right to say that checking non null obscore.OBS_ID values is very slow because obscore is a view. The problem is that this assertion might be true for science queries as well, and especially for queries by positions. From my point of view, this could be a real issue for tables designed for data discovery (TapHandle developer speaking).
FB : standardized metadata are "value addition" to your services. Although the client could work without them you are missing something as a user.
With UCD, for example, the simple thing to do is to be able to display a standard definition of your data piece
Mark Taylor: Validation for Standard Authors
LM (too tired to attend) : I've 2 remarks about the validation of XML examples in standards.
- XML examples are usually snippets amd rarely complete documents and snippets cannot validate against a schema, one can just check the XML comformity.
- MT: one approach is to embed a snippet as an external entity into a test (complete) document suitable for validation. Maybe I should try to do this and provide an example e.g. in the ivoatexDoc Note.
- XML schemas are usually developped along with the standard, therefore using them to validate the ongoing standard is a sort of chicken-egg problem
- MT: but for each candidate schema it should be possible to validate a candidate document against it, no?
Notes
Markus Demleitner: Requirements and Validators
presentation is a variation on blog post at:
https://blog.g-vo.org/requirements-and-validators.html
A "MUST" in standard has significant cost. when should it be used?
- example: where a standard says a value must not be null, a validator would have to verify this by querying table for that value, which could be very expensive to run for large tables. this may cause queries to run a long time or time out.
- when a standard includes a "must", validator has to write some test to check it
Valid reasons for MUST:
- internal requirements - necessary to work at all
- functional requirement - rules imposed so the results make sense
- hedging - prevent clever shortcuts, future-proofing
avoid MUST requirements which are really more "advice"
Importance of validating:
- internal: medium
- external (functional?): higher
- hedging agaisnt cleverness - difficult/impossible?
- hedging for future - highest importance - getting it right now, so later client implementation will succeed
The smaller the immediate impact of failing a requirement, the HIGHER the importance of a validator
- larger impact is more likely noticed soon
Mark Taylor - Validation for Authoring VO Standards
Why validate while authoring?
- check it can be implemented, not misleading users, save time later (reduce errata)
- early validation can help avoid errata later
What to validate
- normative text - schemas, UCD's, (VO)Units
Mechanics
- separate validatable text from main document file (rather than inline with standard LATEX
- if possible, integrate examples into document build process
There are VO validators out there for VOTable, UCDs and VOUnits. IVOA has a page with list of validation tools : https://wiki.ivoa.net/twiki/bin/view/IVOA/IvoaValidatorsSummary
Also XML validators out there, not just IVOA
- correct XSD, make sure documents conform to schema
- examples should conform to external schema
- examples should be well formed
IvoaTex Note - in production, github.com/ivoa-std/ivoatexDoc
STILTS manual
summary
- standard should have associated validator, run validation as part of doc build if possible
- if doc defines XML schema, validate XSD, also at least one schema instances, and any examples in the doc text
James Dempsey: example of a standard that does this well?
MD: IvoaTexDoc has a few examples in the new testing section
MT: The EPN-TAP validator checks UCDs
Tom Donaldson: Astropy has a solid VOTable validator built in
Markus Demleitner: attaching validation to doc could introduce software dependencies on the validation tool?
TD: examples, test cases in validator can help clarify the standard they are validating, resolve misunderstandings.