IVOA Validation Capabilities

This page provides a comprehensive summary of the validation capabilities associated with published IVOA standards. Validators include any software which takes an entity purported to be compliant with one or more IVOA standards and tests this compliance. The entity may a string, a file, a library, or an on-line service. Generally validators may produce false positives, entities which seem to pass the validator but which may still be non-compliant. Only limited tests are done and some semantic aspects of a protocol may not be amenable to automated testing. However false negatives, where failure is indicated even though the entity is compliant, should be rare and should be reported to the party responsible for the validator.

The page is given as a table with entries corresponding to each of the entries in the IVOA standards page. IVOA members who develop validators or who know of validation capabilities should update this table as appropriate. In cases where multiple validators exist for a protocol, separate entries are given for each.

Group
Standard
Versions
Status
Discussion
Type
App
SAMP 1.3 Yes

The JSAMP library/toolkit includes a fairly comprehensive set of validation tests (hubtester) for a SAMP Hub. In principle this can test both Standard and Web Profiles, though the web client is faked (it's not done from a browser). No SAMP client validator is included - such a thing wouldn't be able to do much, but could be a useful addition.

Standalone

VOTable 1.4,1.3,1.2,1.10,1.0 Yes

STILTS

STILTS votlint provides comprehensive validation for VOTable 1.0, 1.1, 1.2, 1.3, 1.4, includes schema/DTD validation as well as much more stringent tests. Can be used standalone, but API has hooks for programmatic incorporation into other validators that need to validate VOTables (taplint does this).

You can also validate against the VOTable schema (or for v1.0 the DTD) provided in the standard.

Standalone and library,

External XML validation

1.4,1.3,1.2,1.10 Astropy

In any environment with Astropy installed, there is a volint command. The same output report can be generated programmatically by directly accessing the validate() method, which also optionally allows schema validation.

Notes:

  • volint warns about version 1.0 UCDs even though they are technically legal in VOTables, so the conesearch-required UCDs ID_MAIN, POS_EQ_RA_MAIN and POS_EQ_DEC_MAIN will cause warnings.
  • UCDs are validated with volint, but units are not. See UCD and VOUnits sections below for notes on independent astropy validation of those.
Installing astropy for standalone or library use
MOC 1.0,1.1, 2.0 Yes Moc.jar provides a validator method for MOC 1.0 REC. This validator is remotely usable via the MocServer servlet provided by CDS. Standalone Jar + Web page
ASCII MOC validator for MOC 1.1 Standalone
HiPS 1.0 Yes Checks a supplied HiPS list and all services it references Standalone
PADC (VOParis) provides a Hipslint-based web validator and runs Hipslint daily on registred services to provides validation reports.

Web

Testing Framework

Web API

DAL
DALI 1.0,1.1 None DALI provides a set of basic capabilities that all DAL services are to implement. DALI capabilities are sometimes tested inother validators but a specific separate test for DALI would be desirable.
DataLink 1.0, 1.1 Partial STILTS datalinklint validates the content of a {links}-response table, including any service descriptors. It does not attempt to check link targets or {links} services themselves. Standalone
N/A Partial shomydl ("show my datalinks") displays datalink documents hierarchically; it should help figuring out syntax problems and ways to improve annotation. Web Service
Cone Search 1.03 Multiple Cone search validators test that a service accepts standard inputs and that required fields are found in the VOTable outputs. Validators also test a limited number of error conditions
The HEASARC uses an NCSA-developed package and tests registered services monthly providing a Web site to view results and run tests interactively. Web page, testing framework
PADC (VOParis) provides a web site which allows a user to validate a specific service and to view the results of their daily validation of registered services.

Web

Testing Framework

Web API

ESA enables a user to validate a particular services and to see the results of their daily testing of registered services

Web and testing framework

Astroquery (Astropy affiliated package) contains some kind of cone search validation client.

Standalone

SIA Multiple SIA service validators check that a service accepts standard inputs and provides appropriate error responses in at least some situations. Most validators check only the initial VOTable response by the service.
1.0 (2.0)

For 1.0 uses NCSA developed library for checking results. In addition to checking initial VOTable responses, checks that first data link is valid and of appropriate type. SIA 2.0 uses VO-Paris validator but stores results at the HEASARC.

Users can check individual services or see results of monthly validation.

Web, Testomg framework
1.0, 2.0 PADC (VOParis) provides a web site which allows a user to validate a specific service and to view the results of their daily validation of registered services

Web

Testing Framework

Web API

1.0 ESA enables a user to validate a particular services and to see the results of their daily testing of registered services Web, testing framework
SLA 1.0 Yes PADC (VOParis) provides an SLA validator Web
SSA 1.1,1.04,1.03 Multiple
PADC (VOParis) provides validaton of SSA services in the same fashion as other DAL services

Web

Testing Framework

Web API

ESA provides validation of SSA services in the same fashion as other DAL services Web, testing framework
STC-S 1.0 None
TAP 1.0,1.1 Yes TAPLint provides an extensive set of tests of a TAP implementation including DALI, synchoronous and asynchronous queries and tests of a variety of error conditions. The various methods in which metadata for the TAP service is provided are compared for consistency. Errors, warnings and informative messages are provided. Users may configure the tests to be run. Tests of included ObsCore capabilities may be run. Versions: 1.0, 1.1. Standalone
The HEASARC's VO valdiation tool runs TAPLint on registered services are saves the results with a monthly cadence.

Testing framework

PADC (VOParis) provides a taplint-based web validator and runs taplint daily on registred services to provides validation reports.

Web

Testing Framework

Web API

TAPRegExt 1.0, 1.1 Yes STILTS taplint (CAP stage) checks TAPRegExt 1.0. Standalone
ADQL 1.0, 2.0, 2.1

Gregory Mantelet's ADQL parser library identifies valid ADQL 2.0 and 2.1 queries.

STILTS taplint issues a few ADQL 2.0 queries to TAP services and notes errors, but does not attempt comprehensive feature testing.

(ADQL 1.0 is moribund)

CDS Library (?)

SkyNode 1.01 N/A Obsolete document
SimDAL 1.0 N/A
VOEvent Transport 1.0, 2.0 Missing
SODA 1.0 Yes CASDA's sodalint tests the DALI and SODA endpoints plus the service descriptor Standalone
EPN-TAP 2.0 Yes taplint stage EPN does quite comprehensive validation of EPN-TAP services. Standalone
PADC (VOParis) provides a taplint-based web validator and runs taplint daily on all tables of EPN-TAP registred services to provides validation reports.

Web

Testing Framework

Web API

DaM
Photometry 1.0 N/A
SimDM 1.0
Astronomical Dataset Characterization 1.13,1.12 N/A
Simple Spectral Lines 1.0 N/A
Spectral 1.0, 2.0 N/A
ObsCore 1.0, 1.1 Yes STILTS taplint (OBS stage) does comprehensive testing that mandatory ObsCore elements are present, and mandatory/optional elements are as prescribed by the standard, in running TAP services. Versions: 1.0, 1.1.

Standalone

Characterization: Complements and new features. 1.0 Unknown
VO-DML 1.0

Standard tooling will validate models with the

gradle vodmlValidate

command

https://github.com/ivoa/vo-dml
VO-DML 1.0 Yes CADC Tool Standalone
Dataset Metadata 1.0 N/A

Cube (Image) 1.0 N/A Standalone - link TBD
STC 2.0 Yes

vo-dml schema validation examples pass votable and xml validation

Standalone - link TBD
ObsLocTAP 1.0 Yes STILTS taplint LOC stage. Standalone
MIVOT 1.0
GWS
Parameter Descripton Language 1.0 Missing
SSO 1.01, 2.0 N/A Standard references other non-VO standards
VOSpace 2.0,2.1 Yes A set of integration tests on VOSpace conformance. Library
Credential Delegation 1.0 Missing
UWS 1.0, 1.1 Multiple

STILTS taplint (UWS stage) tests most but not all aspects of UWS 1.0 operation required from TAP services.

Standalone
uws-validator: suite of tests created for testing UWS 1.1 features, but can do basic tests for UWS 1.0 as well (use --tags=-uws1_1 to exclude 1.1 specific tests). May still report false negatives, since it is quite strict. Standalone
VOSI 1.0,1.1 Yes A set of integration tests on VOSI conformance. Library
GMS 1.0
Web Service Basic Profile 1.0 N/A Obsolete document
ReR
IVOA Identifiers 2.0 Yes This service implements the Identifiers 2.0 suggestion for how to resolve an IVOA publisher dataset identifier; that is, it tries to resolve the registry part to Datalink, SSAP, or Obscore capabilities and tries those to find access URLs. If that fails, it repeats this procedure for services that the base resource declares as served-by.

Web,

Standalone

OAI-PMH with IVOA extensions

(non-IVOA standard)

2.0 Yes This Registry of Registries page validates entire registries. It implements an OAI 2.0 validator (using the OAI Explorer by Hussein Suleman) with additional IVOA-specific validation including the referential integrity of managed authorities, as well as validation of individual VOResources in the registry. Web
Registry Interfaces 1.0 Yes The OAI-PMH part is validated by the rofr validator; capability instances can be validated using common XSD tools. Web, XML Schema
Resource Metadata 1.12,1.01 N/A No validatable content in this standard.
StandardsRegExt 1.0 Yes Instance documents can be validated using common XML Schema tools Schema
SimpleDALRegExt 1.0, 1.1, 1.2 Yes Instance documents can be validated using common XML Schema tools SIA, SLAP, SSA, and Cone search schema files
VOResource 1.03,1.1 Yes Instance documents can be validated using common XML Schema tools schema
VODataService 1.1, 1.2 Yes Instance documents can be validated using common XML Schema tools schema

RegTAP:

Registry Relational Schema

1.0, 1.1, 1.2 Yes GAVO has a suite of tests for a relational registry. This can be run on a specially prepared instance of RegTAP software, not on an operational RegTAP service. Supports 1.0, 1.1 and 1.2. Standalone
Sem
VOUnits 1.0 Yes Unity is a C/Java library that parses/validates VOUnit expressions. It comes with a long list of test cases. STILTS harness also available. Library, standalone
Astropy can parse and validate VOUnits, and that capability can be accessed programmatically via the VOUnit class. The parse() method emits warnings for unknown units. Library
UCDs 1.10 N/A See UCD1+ vocabulary
UCD1+ Vocubulary 1.23,1.3,1.11 Yes This web page provide some basic HTTP tools to validate, assign, translate, explain UCD1+. The validation is believed to be comprehensive following some changes in 06/2018. Web
1.4, 1.5

Ucidy is a lightweight Java library aiming to parse and validate UCD. There is also the possibility to research UCD words in different ways among a validated and/or customised list.STILTS harness also available.

Library, standalone
1.4 Astropy provides UCD validation programmatically via the parse_ucd() function which throws an appropriate ValueError() for invalid UCDs. Library
UCD Maintenance 1.20,2.0 N/A?
Vocabularies 1.19, 2.0 Yes 2.0 validator is now part of the standard. Standalone
SDP
IVOA Document Standards 1.2 N/A
VOE
VOEvent 2.0,1.0 Missing
VOEventRegExt 1.0 Missing

Notes:

  • Group: The IVOA Working Group under which the standard is being developed/maintained.
  • Standard: A short version of the name of the standard.
  • Versions: The versions of the standard for which a validator is available. Normally only recommendations and the version of the standard currently in development are included.
  • Status: Do validators exist? Yes/Multiple: Validators exist. N/A (not applicable): Either the standard is not validatable or is obsolete/abandoned. Missing: A validator should be available for this standard but none is known.
  • Discussion: A discussion of what validators do (or might) test. When multiple validators an overall entry for the standard may be given followed with details for each validator on separate rows..
  • Type: What kind of validator exists. Types include
    • Library: Code library that can be incorporated in other applications (including validators). Download link provided.
    • Standalone: A standalone tool that runs on the tester's machine. A download link is provided.
    • Web: A tool that can be run interactively by a user at a web site. Web link provided.
    • Testing framework: A web site that provides links to results of tests of a class of services that is performed autonomously at some cadence. Web link provided.
    • Web API: An Application Programming Interface which provides test results in a programmatic way over the web. API specification link provided.
http://www.star.bris.ac.uk/~mbt/stilts/sun256/sun256.html
Edit | Attach | Watch | Print version | History: r60 < r59 < r58 < r57 < r56 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r60 - 2023-10-03 - TamaraCivera
 
This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback