ProvenanceDataModelLegacy < IVOA

IVOA Web>IvoaDataModel>ObservationProvenanceDataModel>ProvenanceDataModelLegacy (2016-10-04, KristinRiebe) (raw view)

<br /> <!--
      * Set ALLOWTOPICRENAME = IVOA.TWikiAdminGroup
-->
---+ Provenance Data Model Legacy
---++ Original concept in Observation data model IVOA note (2005)

The basic concepts of this model have been discussed as early as 2002/2004. It gave birth to the Spectral data model and the characterisation data model. The draft for the full Observation model became an [[http://www.ivoa.net/documents/latest/DMObs.html][IVOA note]] in 2005. The concept of provenance was introduced in this note.

---++ Motivation and use cases

The motivation and use cases for a provenance data model have been discussed at most of the Interop meetings between 2006 and 2012 with evolution of the concept:

| *Interop* | *Content* | *Materials* |
| Victoria 2006 | Close to completion of char datamodel, motivations to start full Obs data model including Provenance | [[http://wiki.ivoa.net/internal/IVOA/InterOpMay2006DataModel/Observation.ppt][Spring 2006]] |
| Beijing | More detailed proposal for the content | [[http://wiki.ivoa.net/internal/IVOA/InterOpMay2007DataModel/future.pdf][Spring 2007]] |
| Cambridge UK | New provenance use cases gathered from a spectroscopy workshop held at ESAC in 2007 | [[http://wiki.ivoa.net/internal/IVOA/InterOpSep2007DataModel/ObservationUseCase.ppt][Fall 2007]]  |
| Trieste | Looking for new partners with various interests and observational background. | [[http://wiki.ivoa.net/internal/IVOA/InterOpMay2008DataModels/Observation.pdf][Spring 2008]] |
| Baltimore | Juan de proposes a radio astronomy view of Provenance | [[http://wiki.ivoa.net/internal/IVOA/InterOpPct2008DataModels/VODataProvenance.Baltimore.pdf][Fall 2008 / Juan de Santander Vela]] |
| Baltimore | Mireille proposes a core Provenance container with hooking of specific details in addition | [[http://wiki.ivoa.net/internal/IVOA/InterOpPct2008DataModels/obsDMstrategy.pdf][Fall 2008 / Mireille Louys]] |
| Garching | Juan de moved to ESO and proposes with Arancha a provenance view for ESO requirements | [[http://wiki.ivoa.net/internal/IVOA/InterOpNov2009DM/TheCaseForProvenance.pdf][Fall 2009 / Juan de Santander Vela]]  |
| Garching | Igor is moving to CDS. He proposes provenance use cases for complex datasets | [[http://wiki.ivoa.net/internal/IVOA/InterOpNov2009DM/complex_data_chilingarian.pdf][Fall 2009 / Igor Chilingarian]] |

There were splinter meetings on provenance in Baltimore, Garching and Nara. No trace has been left about this. But they have been opportunities to discover requirements from newcomers in the game. The same happened at EURO-VO meetings in the same period.

---++ First xml attempts

Attempts were made to serialize provenance details. The goal was to demonstrate the aibility of the project.

| *Interop* | *Content* | *Materials* |
| Baltimore | First serialization examples in xml. Processing is made of succession of processing stages hooking inpout and output datasets  | [[http://wiki.ivoa.net/internal/IVOA/InterOpPct2008DataModels/ObsProvCharDM.pdf][Fall 2008 François Bonnarel]] |
| Strasbourg | An example of xml schema for Observing Configuration has been added | [[http://wiki.ivoa.net/internal/IVOA/DMInteropStrasbourg/ObsCharProv.pdf][Spring 2009 DM]] |
| Strasbourg |  Access to Provenance metadata was seen under the "generic data set" concept which gave birth to ObstAP/ObsCOre and DataLink soon after  | [[http://wiki.ivoa.net/internal/IVOA/200905DALSessions/GenericDsObs.pdf][Spring 2009 DAL]] |
| Garching | For the first time Provenance metadata are seen as a complement to ObsCore in the general VO architecture | [[http://wiki.ivoa.net/internal/IVOA/InterOpNov2009DM/FullObservationDataModel.pdf][Fall 2009 François Bonnarel]] |

You can also look directly at the [[%ATTACHURL%/Provenance.xsd][xml schema]] and the [[%ATTACHURL%/provenance.xml][xml example]] prototyped in this period. They are not consistent however.

---++ UML diagram

According to IVOA rules at that time a Data Model had to be exposed with an UML schema. In 2010 it was considered time to try to build one.

| *Interop* | *Content* | *Materials* |
| Victoria 2010 | First (and only) attempt to build a provenance UML diagram | [[http://wiki.ivoa.net/internal/IVOA/InterOpMay2010DM/Provenance.pdf][Spring 2010]] |
| Nara | In 2010 we were also introducing a use case for VOspace. But we realized progresses were really slow. UML diagram unchanged | [[http://wiki.ivoa.net/internal/IVOA/InterOpDec2010DataModel/Provenance.pdf][Fall 2010]] |

---++ Sao Paulo review (October 2012)

In a dedicated splinter meeting at this interop an attempt was made to relaunch the effort. There was a lot of use cases summarized in the [[http://wiki.ivoa.net/internal/IVOA/InterOpOct2012Splinters/Provenance.pdf][Introduction talk]] as well as a summary of the previous technological attempts. There were talks by Nikos from Marseille (calculator), IVOA.ArnoldRots and IVOA.KristinRiebe, but unfortunately they are lost. A the end of the meeting IVOA.MarkusDemleitner concluded that the approach followed so far was definitely too ambitious and we had to start with revised ambitions and more stable technology. This started at the Heidelberg meeting in May 2013.

---++ Conclusion

Why was all this so slow and eventually "failed"? In IVOA.FrancoisBonnarel opinion there was a sociological and a technical reason. The sociological one was that the Provenance task was nobody's priority task (including DM "permanents" like Mireille, Anita, Igor or François) just because there was much more critical VO businness for them at the time. Things have changed now. The technical point on the other side was that there were a lot of classes to develop and many ways were possible to describe the metatada. Probably moving to the W3C core architecture helped to structure the efforts around a sound and stable core.

Attachments

Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
xsd	Provenance.xsd	r1	manage	3.0 K	2016-09-16 - 14:06	FrancoisBonnarel	Provenance xml schema
xml	provenance.xml	r1	manage	3.7 K	2016-09-16 - 15:06	FrancoisBonnarel	Provenance xml example document

Topic revision: r6 - 2016-10-04 - KristinRiebe

IVOA

Log in or Register

IVOA.net
Wiki Home
WebChanges
WebTopicList
WebStatistics

Twiki Meta & Help
IVOA
Know
Main
Sandbox
TWiki

TWiki intro
TWiki tutorial
User registration
Notify me

Working Groups

Interest Groups

Committees

Stds&Procs

www.ivoa.net
Documents
Events
Members
XML Schema