Provenance Data Model Legacy

Original concept in Observation data model IVOA note (2005)

The basic concepts of this model have been discussed as early as 2002/2004. It gave birth to the Spectral data model and the characterisation data model. The draft for the full Observation model became an IVOA note in 2005. The concept of provenance was introduced in this note.

Motivation and use cases

The motivation and use cases for a provenance data model have been discussed at most of the Interop meetings between 2006 and 2012 with evolution of the concept:

Interop Content Materials
Victoria 2006 Close to completion of char datamodel, motivations to start full Obs data model including Provenance Spring 2006
Beijing More detailed proposal for the content Spring 2007
Cambridge UK New provenance use cases gathered from a spectroscopy workshop held at ESAC in 2007 Fall 2007
Trieste Looking for new partners with various interests and observational background. Spring 2008
Baltimore Juan de proposes a radio astronomy view of Provenance Fall 2008 / Juan de Santander Vela
Baltimore Mireille proposes a core Provenance container with hooking of specific details in addition Fall 2008 / Mireille Louys
Garching Juan de moved to ESO and proposes with Arancha a provenance view for ESO requirements Fall 2009 / Juan de Santander Vela
Garching Igor is moving to CDS. He proposes provenance use cases for complex datasets Fall 2009 / Igor Chilingarian

There were splinter meetings on provenance in Baltimore, Garching and Nara. No trace has been left about this. But they have been opportunities to discover requirements from newcomers in the game. The same happened at EURO-VO meetings in the same period.

First xml attempts

Attempts were made to serialize provenance details. The goal was to demonstrate the aibility of the project.

Interop Content Materials
Baltimore First serialization examples in xml. Processing is made of succession of processing stages hooking inpout and output datasets Fall 2008 François Bonnarel
Strasbourg An example of xml schema for Observing Configuration has been added Spring 2009 DM
Strasbourg Access to Provenance metadata was seen under the "generic data set" concept which gave birth to ObstAP/ObsCOre and DataLink soon after Spring 2009 DAL
Garching For the first time Provenance metadata are seen as a complement to ObsCore in the general VO architecture Fall 2009 François Bonnarel

You can also look directly at the xml schema and the xml example prototyped in this period. They are not consistent however.

UML diagram

According to IVOA rules at that time a Data Model had to be exposed with an UML schema. In 2010 it was considered time to try to build one.

Interop Content Materials
Victoria 2010 First (and only) attempt to build a provenance UML diagram Spring 2010
Nara In 2010 we were also introducing a use case for VOspace. But we realized progresses were really slow. UML diagram unchanged Fall 2010

Sao Paulo review (October 2012)

In a dedicated splinter meeting at this interop an attempt was made to relaunch the effort. There was a lot of use cases summarized in the Introduction talk as well as a summary of the previous technological attempts. There were talks by Nikos from Marseille (calculator), ArnoldRots and KristinRiebe, but unfortunately they are lost. A the end of the meeting MarkusDemleitner concluded that the approach followed so far was definitely too ambitious and we had to start with revised ambitions and more stable technology. This started at the Heidelberg meeting in May 2013.


Why was all this so slow and eventually "failed"? In FrancoisBonnarel opinion there was a sociological and a technical reason. The sociological one was that the Provenance task was nobody's priority task (including DM "permanents" like Mireille, Anita, Igor or François) just because there was much more critical VO businness for them at the time. Things have changed now. The technical point on the other side was that there were a lot of classes to develop and many ways were possible to describe the metatada. Probably moving to the W3C core architecture helped to structure the efforts around a sound and stable core.

Topic attachments
I Attachment History Action Size Date Who Comment
Unknown file formatxsd Provenance.xsd r1 manage 3.0 K 2016-09-16 - 14:06 FrancoisBonnarel Provenance xml schema
XMLxml provenance.xml r1 manage 3.7 K 2016-09-16 - 15:06 FrancoisBonnarel Provenance xml example document
Edit | Attach | Watch | Print version | History: r6 < r5 < r4 < r3 < r2 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r6 - 2016-10-04 - KristinRiebe
This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback