IVOA Data Model Working Group

Introduction

The role of the Data Modeling group is to provide a framework for the description of metadata attached to observed or simulated data.
The activity of the Data Model WG activity focuses on logical relationships between these metadata, examines how an astronomer wants to retrieve, process and interpret astronomical data, and provides an architecture to handle them. What is defined in this WG can then be re-used in the protocols defined by the DAL WG or in VO aware applications .

Participation

Membership of this working group is informal as people and projects come and go. To join see our page on How to participate

Progress Meetings

The DM working group conducts, with varying levels of regularity, running meetings to discuss the progress of ongoing work and organize our efforts.
The DM Running Meeting Page is available to:

• Find day/time of the next scheduled meeting
• Add discussion items for addition to an upcoming meeting agenda
• Provide an access point to meeting notes.

WG Roadmaps

• See the TCG roadmap

Mailing-list

dm@ivoa.net

• info - subscribe/unsubscribe/edit: listinfo form
• Working group discussions: archive

Standards and Documents

Here is a graphic illustrating the dependencies between data models used in the IVOA framework.

Standards repository

Development repository

Implementation Requirements for Data Models

The "IVOA Document Standards" standard has a broad outline of the implementation requirements for IVOA standards. These requirements fit best into the higher level standards for applications and protocols than for data models themselves. At the Oct 2017 interop in Trieste, the following implementation requirements for Data Model Standards was agreed upon, which allow the models to be vetted against their requirements and use cases, without needing full science use cases to be implemented.

• VO-DML models must validate against schema
• Serializations which touch each entity of the model. These serializations may be 'fake' (ie: not based on actual data files), and are to be produced by the modeler as unit tests/examples.
• Real world serializations covering use cases, produced by others following the model, in a mutually agreed upon format.
• Software which interprets these serializations and demonstrates proper interpretation of the content.

General presentations on the DM modeling effort

• DataModels in the Virtual Observatory Framework

Projects

Active Projects

• Source Data Model:
  • Source and Catalog Requirement survey
  • Source DM and Workshop 2021: see here
  • Project migrates to MANGO data model - providing a generic framework which applies to this project.
• Multi-Dimensional Data Model:
  • The project encompasses the description of Multi-Dimensional data products, from pixelated Images to n-dimensional hypercubes
  • The concept is that this single N-Dimensional Cube model should form the framework from which specialized data products can be defined.
  • As such: it forms the basis for supporting these products-of-interest in the community
    • Spectra (SpectralDM v2.0), including Eschelle spectra
    • TimeSeries
    • Event Lists
    • Pixellated images ( ImageDM v1.0)
  • The Cube model has been demonstrated to satisfy the requirements of these products, but lacks the implemented science oriented usage threads needed to gain general support.
    • We are working closely with the formation of a High Energy Interest Group, which we hope will bring resources and motivation to help see this project to fruition.

Proposed Projects

• IVOA Note for FITS serializations
  • discussion started as question about lightcurve data model, but was concluding with a proposal to develop an IVOA Note with ‘preferred’ serializations for various products in FITS format.
  • see DM mail thread
• Characterisation v2.0
  • Project to create vo-dml compliant version of the model.
  • Currently idle.. pending resource availability
• Proposal Data Model
  • Data model for making Observation Proposals. It is being created as project within The Opticon RadioNet Pilot (https://www.orp-h2020.eu)
  • see git repository
• STC-2.0:
  • Project to create vo-dml compliant version of STC
  • The project has been broken up, with separate models covering portions of the domain space covered by STC-1
    • Coords, Measurements, Transforms - completed or in progress
    • Regions, Intervals, ?? - TBD
• VODML-Mapping:
  • Project to develop an annotation syntax mapping VOTable serializations to VO-DML compliant models.
  • The initial syntax proposal was rejected;
  • An alternative syntax was developed as MIVOT

Topic Pages

• UTYPEs discussion: Closed with decision to adopt VO-DML
• Mapping hands-on session page (May 2018)
• High Energy physics data modeled & serialised with VO tools page
• Running meeting AstrPyVO MIVOT page
• Obscore 1.1-Next wiki page

Archival Project Reference Information

RFC pages for our standards.

Older IVOA Drafts and Related Documents

• Quantity model current drafts (.ps, .pdf)
• Quantity model serialization examples (.html)
• Observation model current drafts (.ps, .pdf)
• 2004 Linear STC: linear (command line) implementation of STC (v1.2).
• 2002 IDHA Data Model for Astronomical Images: modeling of the acquisition process and data products
• 2005 Characterisation_Note_20051129.pdf: Characterization DM draft Note
• spec96d.pdf: Spectrum DM version 0.96 latest (Victoria)
• spec97d.pdf: Alternative Spectrum DM with STC
• SourceDM_Moscow_2006.pdf: Source Catalog Data Model Moscow Sept 2006
• CharacterisationWDversion1.0.pdf: Characterisation Data Model
• update of CharacterisationIvoadraft-v1.1.pdf: Characterisation Model Working draft version 1.1
• ivoa_char_2d_image_tutorial_1.0.pdf: User's guide for Characterisation DM by Alberto Micol
• CharacterisationImplementationv1.1.pdf: Implementations of the Characterisation DM v1.1

Virtual Observatory Projects working on Data models and their implementations should list themselves here. * to fill and update***

• ObservatoireStrasbourg
  • SAADA CharDM and ObsCore

• CDS
  1. Aladin portal
  2. CAMEA, an interactive tool for Characterisation metadata publishing
  3. Vizier + Saada: interface for products associated with Vizier sources
• Lerma- Paris
  1. Aspid-SR CharDM

Meetings

Most of our face-to-face activity takes place during IVOA Interoperabiltiy Meetings. Follow the Data Model Page links to locate agendas, presentation slides, and action items.

Involved People

Please put your name here if you wish ...

Other DM Meetings

• ADASS 2021: BoF TAP and the data models TapandTheDMs
• Check the above Interop pages for DM WG sessions
• ESO/Garching, Jan 2004: Data Model Technical Meeting 4 Draft 0.1 of Observation model
• Baltimore, Dec 2003: Data Model Technical Meeting 3 (no minutes yet; see draft 0.2 of Quantity)
• ESO/Garching on Jan. 23/24, 2003: Data Model Sessions minutes (WG creation and planning)
• Cambridge MA, Oct 2002: Data Model Technical Meeting 1 presentations & minutes

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup, MarkCresitelloDittmar
 
 
-->