VODML < IVOA

---+ VO Data Modeling Language Specification

Current REC:
   * _VO-DML: a consistent modelling language for IVOA data models_ ([[http://ivoa.net/documents/VODML/20180910/VO-DML-REC-v1.0.docx][docx]], [[http://ivoa.net/documents/VODML/20180910/VO-DML-REC-v1.0.pdf][pdf]], [[http://ivoa.net/documents/VODML/index.html][IVOA.net]])
The tooling and document source associated with VODML can be found at [[https://github.com/ivoa/vo-dml][GitHub]]

[GL 2014-04-25: I will try to keep track of comments from the mailing list on [[VoDmlWGComments][this page]]

We have made a first superficial pass at porting other IVOA data models to VO-DML, in particular characteriation, stc, simdm. These can be used as starting points for a more formal IVOA DM effort to create VO-DML representations for all models, but such an effort MUST include domain experts for those models, i.e. likely the original modellers.

Both DatasetMetadata and [[https://volute.g-vo.org/svn/trunk/projects/dm/vo-dml/models/STC2/][STC2]] have been ported to the last VO-DML version (Spring 2016).

Many of these models have been created from a UML definition using an XSLT script available in the [[https://volute.g-vo.org/svn/trunk/projects/dm/vo-dml/xslt/][xslt]] sub-folder. For each model an HTML documentation file has been generated from the vo-dml/xml representation using another xslt script. These can be executed using appopriate ttargets in the ant build.xml file. This also contains a target for validating vo-dml/xml files, which is especially useful for such files written by hand.

Please read the [[https://volute.g-vo.org/svn/trunk/projects/dm/vo-dml/README.txt][README.txt]] for a first draft of a documentation of the structure of the vo-dml folder on volute and and hints how to use the ant build.xml script for the various XSLT scripts. This documentation will be copied on this page over time.

A group of IVOA members from SAO, HST, and JHU had a face to face meeting in February 2016 to work on the current drafts and move them towards the RFC stage. Notes from the meeting and blackboard snapshot are [[https://confluence.stsci.edu/display/MastDist/VODML+Meeting][publicly available]]

A model template [[https://volute.g-vo.org/svn/trunk/projects/dm/vo-dml/models/templates/][template]] has been made available for Modelio 3.0 has. It must be first imported in Modelio and then used as a base for the new model. Before to be converted in VO-DML, the model must be exported as an XMI (UML2.4) file. The others steps are described in [[https://volute.g-vo.org/svn/trunk/projects/dm/vo-dml/README.txt][README.txt]]. In some cases the XMI file must be patched by hand (UML namspace 1st row) to pass the VO-DML transformation. This has been tested on Lunix with Modelio 3.0 and on MacOS (El Captain) with Modelion 3.3.
---++ Notes for Proposed Recommendation
---++++ Reference Implementations

Reference Implementations were developed during the WG discussion phase to inform the development of the specification. Feedback from implementations was folded back into the Proposed Recommendation document.

The normative part of the specification defines an XML schema representation for Data Models. Strictly speaking, a reference implementation supports reading or writing of XML documents written according to the schema, covering the entire schema definition, or provides validators for XML instances.

XML documents describing hypothetical or actual data models using all of the specification features also seem to qualify as Reference Implementations.

In a more general sense, however, implementations may map the meta-model defined by VODML to other meta-models, using the XML schema as an exchange format between custom and standard representations of the meta-model and its instances. In this sense the list below includes software implementations or graphical representations that can be used to read or write data model instances in their standard representations or to create new products from them.

In any case, all the implementations below can be used as a "golden standard reference" for the different implementations of the specification.
---+++++ Validator

A Reference Implementation for validators was developed by means of [[http://www.w3schools.com/xsl/][eXtensible Style Sheet Language Transformation]] scripts, schema validation, and [[http://www.schematron.com/][schematron]].

The script is available [[http://volute.g-vo.org/svn/trunk/projects/dm/vo-dml/][on volute]] as part of a bigger suite of implementations, also described in this list. Some [[http://volute.g-vo.org/svn/trunk/projects/dm/vo-dml/README.txt][documentation is available]].

The validator takes care of verifying that instances are compliant with the schema. Also, it checks that instances comply with rules that are harder or impossible to descibe in a schema, by means of a schematron ruleset.

The validator was developed by Gerard Lemson (GaVO and JHU).
---+++++ DatasetMetadata data model

Current document is located in [[http://volute.g-vo.org/svn/trunk/projects/dm/DatasetMetadata-1.0/doc/WD-DatasetDM-1.0-20160412.pdf][volute]].

The model was defined by following VODML prescriptions and has a [[http://volute.g-vo.org/svn/trunk/projects/dm/vo-dml/models/ds/DatasetMetadata-1.0.vo-dml.xml][standard representation]] as a VODML XML file exercising all features of the VODML meta-model.

The model validates against both the schema and the schematron presented above, as shown by the [[http://volute.g-vo.org/svn/trunk/projects/dm/vo-dml/models/ds/DatasetMetadata-1.0.vo-dml.xml.validation-report.txt][validation report]].

A browseable representation of the model is [[http://volute.g-vo.org/svn/trunk/projects/dm/vo-dml/models/ds/DatasetMetadata-1.0.html][also available]].

The implementation demonstrates that the VODML meta-model is sufficient to express a complex real life data model, and that the VODML schema can unambiguously represent such a data model in a standardized fashion.

The model was developed by the DM working group with Mark Cresitello-Dittmar (SAO) as editor.
---+++++ STC2 data model

There is currently no official Working Draft for this effort, however model diagrams are [[http://volute.g-vo.org/svn/trunk/projects/dm/vo-dml/models/STC2/2016-02-19/Diagrams2016-02-19][hosted on volute]]. The model is being developed following VODML prescriptions and has a [[http://volute.g-vo.org/svn/trunk/projects/dm/vo-dml/models/STC2/2016-02-19/VO-DML-STC2.vo-dml.xml][standard representation]] as a VODML XML file exercising all features of the VODML meta-model.

The model validates against both the schema and the schematron presented above, however it makes use of a discouraged feature (attributes of arbitrary multiplicity) as showed by the [[http://volute.g-vo.org/svn/trunk/projects/dm/vo-dml/models/STC2/2016-02-19/VO-DML-STC2.vo-dml.xml.validation-report.txt][validation report]].

A browseable representation of the model is [[http://volute.g-vo.org/svn/trunk/projects/dm/vo-dml/models/STC2/2016-02-19/VO-DML-STC2.html][also available]].

The implementation, although in progress, demonstrates that the VODML meta-model can express highly complex data models, and that the VODML schema can unambiguously represent such a data model in a standardized fashion.

The model was developed by the Arnold Rots (SAO) with contributions from other WG members, and it is currently in the WG discussion phase.
---+++++ ivoa data model

This model represents a collection of basic types (mostrly VODML's PrimitiveType-s) that can be reused by other models to express values in the Models.

It has a [[http://volute.g-vo.org/svn/trunk/projects/dm/vo-dml/models/ivoa/IVOA.vo-dml.xml][standard representation]] as a VODML XML file.

A browseable representation of the model is [[http://volute.g-vo.org/svn/trunk/projects/dm/vo-dml/models/ivoa/IVOA.html][also available]].

The model was developed by Gerard Lemson (GaVO and JHU).
---+++++ <a name="HTML_generator"></a>HTML generator

Valid VODML representations can be used to produce, automatically or semi-automatically, a number of documentation products. One example is the HTML generator included in the [[http://volute.g-vo.org/svn/trunk/projects/dm/vo-dml/][VODML tools suite]].

The tool takes a standard model definition as input and produces a browseable, interactive web page describing the model, with cross-references, figures, and tables of contents. A UML diagram is also automatically generated if the required libraries and applications are installed (some examples have been provided in the above sections for DatasetMetadata and STC2 data models).

The implementation shows that the VODML language is expressive enough to allow the automation of tasks related to the data modeling effort.

The implementation was developed by Gerard Lemson (GaVO and JHU) with contributions from WG members using different UML modelers.
---+++++ UML translation scripts

Data modeling can sometimes start with a UML modeler tool. Although the [[http://www.omg.org/spec/XMI/][XMI]] format was designed as a standardized exchange format for meta-model descriptions, including the serialization of UML models, the standard is complex and the level of interoperability is far from perfect.

In order to allow translation from XMI files produces by widely used UML modelers to standard (and much simpler) VODML model descriptions, a set of translation scripts were included in the aforementioned VODML toolset.

This implementation shows that different meta-models can be mapped to VODML and enable interoperability with other standards. It also shows the simplicity of VODML when compared to other similar solutions with a much wider scope.

The implementation was developed by Gerard Lemson (GaVO and JHU).
---+++++ Jovial software library

The [[https://github.com/olaurino/jovial][jovial library and toolset]] is an ongoing effort to provide a supporting toolbox to IVOA data modelers. At its core it provides a full implementation of the VODML meta-model and schema. It is developed in Groovy but it is compatible with Java.

The library defines a Domain Specific Language for expressing data models and instances of data models, and allows to read standard VODML descriptions into Java objects, as well as to serialize Java/Groovy in-memory representations or DSL descriptions to VODML documents. A data model browser GUI is also currently under development.

The implementation shows that software libraries and applications can be easily implemented against the VODML meta-model employing widely used programming languages. One of the applications of the library is to provide a format-agnostic language to express instances of data models that can be serialized in a number of formats, so far VOTable using the ongoing VODML Mapping in VOTable WD.

The library was used to produce reference DatasetMetadata serializations starting from VODML XML descriptions of the participating Data Models (DatasetMetadata, ivoa, STC2).

The implementation was developed by Omar Laurino (SAO).
---++++ Interoperability of Reference Implementations

It is important to show that reference implementations are actually interoperable, as interoperability is the key requirement for IVOA specifications.

A number of implementations were developed using different technologies and languages in a complex network of interactions. One interesting demonstration of interoperability is the pipeline that leads from abstract UML models in a modeler desktop application to the serialization of comples data model elements to VOTable. Although the serialization to VOTable, as well as support for UML, are not parts of the normative part of the specification, interoperability should be clearly demonstrated:

* Data Models (e.g. DatasetMetadata, STC2 and ivoa basic types) were defined as UML models following VODML prescriptions using Modelio and MagicDraw, two widely used modeling desktop applications.

* DatasetMetadata includes the ivoa model

* Standardized descriptions of DatasetMetadata and ivoa models were produced by translation scripts from the XMI representation of the Model in the application-specific XMI flavor. XML descriptions refer to each other (DatasetMetadata *imports* ivoa and STC2).

* The standardized descriptions can be loaded in a Groovy software library to facilitate users in defining instances of the Data Models and serializing them, e.g. to VOTable. The application can also be used to define Models and save them in standardized VODML XML format to be used by other Models or applications.

* The standardized descriptions can also be used to produce human readable documentation in a common format, i.e. HTML documents with clickable cross-references.

Participants in the pipeline used different operating systems, programming languages, and desktop applications.

<!--
      * Set ALLOWTOPICRENAME = IVOA.TWikiAdminGroup
-->
This topic: IVOA > WebHome > IvoaDataModel > VODML
Topic revision: r10 - 2022-10-12 - PaulHarrison