Provenance meeting in Paris, France, 2018 August 28th to August 30thsupported by OV-France, GAVO, Paris Data Center and Asterics ProjectFollow_up_meetinghosted at Observatoire de Paris, FranceAll details provided at the indico page at https://indico.obspm.fr/event/59/Finalising the proposed recommendation for Fall interoperability meeting | |||||||||
Changed: | |||||||||
< < | Minutes / ML to be completed | ||||||||
> > | Minutes / ML to be completed | ||||||||
side discussion before start: Poster for Adass conference ??? | |||||||||
Changed: | |||||||||
< < |
| ||||||||
> > |
| ||||||||
| |||||||||
Added: | |||||||||
> > | To be decided before Thursday noon. | ||||||||
Deleted: | |||||||||
< < | To be decided before Thursday noon. | ||||||||
The ProvDM draft update we plan to validate the next days:
| |||||||||
Changed: | |||||||||
< < | What does DEPTH= mean : | ||||||||
> > | What does DEPTH= mean : | ||||||||
Changed: | |||||||||
< < | 1- ? Generation levels for entities | ||||||||
> > | 1- ? Generation levels for entities | ||||||||
2- ? Relations between 2 classes of Prov Core : then we need to subclass the kind of relations entity/entity or entity/activity/entity? Presentations around the table —Tour de table - participants | |||||||||
Changed: | |||||||||
< < | *Mathieu Servillat , Luth | ||||||||
> > | *Mathieu Servillat , Luth | ||||||||
| |||||||||
Deleted: | |||||||||
< < | |||||||||
Discussed other data models presented at Provenance week:
| |||||||||
Deleted: | |||||||||
< < | |||||||||
Parameter discussion : are there context where they are different from entities . Simulation ? A result is computed from an activity : is it an entity ? A parameter? | |||||||||
Changed: | |||||||||
< < | Proposal : WasDerivedFrom is the relation to express a parameter stems from an Entity ? | ||||||||
> > | Proposal : WasDerivedFrom is the relation to express a parameter stems from an Entity ? | ||||||||
still to discuss | |||||||||
Changed: | |||||||||
< < | Scope for the draft ? | ||||||||
> > | Scope for the draft ? | ||||||||
| |||||||||
Changed: | |||||||||
< < | Wednesday Aug 29 | ||||||||
> > | Wednesday Aug 29 | ||||||||
Changed: | |||||||||
< < | Presentations
| ||||||||
> > | Presentations
| ||||||||
Deleted: | |||||||||
< < | What format and content should come out of Prov-SAP? Proposed by Postdam: This the tool to be compatible with W3C tools : Agreed | ||||||||
Changed: | |||||||||
< < | Will display more concepts than Prov W3C, | ||||||||
> > | Translation for relations: IVOA —> W3C | ||||||||
Deleted: | |||||||||
< < | Extends attributes provided in the core W3C to specific entity classes. But each class derived from entity can be shown as entity type : parameter, parameter description, Activity description, etc… | ||||||||
Changed: | |||||||||
< < | Translation for relations: | ||||||||
> > | had context —> was influenced by | ||||||||
Deleted: | |||||||||
< < | IVOA —> W3C | ||||||||
Deleted: | |||||||||
< < | had context —> was influenced by | ||||||||
had config —> was influenced by | |||||||||
Changed: | |||||||||
< < | But then it is not easy to read for the user . | ||||||||
> > | But then it is not easy to read for the user . Looses the fine classification for applications sorting the provenance info . Should we derive new visualisation methods for the ivoa? Enrich the Provenance suite for instance … | ||||||||
Deleted: | |||||||||
< < | Looses the fine classification for applications sorting the provenance info . Should we derive new visualisation methods for the ivoa? Enrich the Provenance suite for instance … | ||||||||
Changed: | |||||||||
< < |
| ||||||||
> > |
| ||||||||
Changed: | |||||||||
< < | Input output context config —> trace what happens | ||||||||
> > | Input output context config —> trace what happens This is to cover the CTA pipeline execution ML: I see it as a workflow tracking system | ||||||||
Deleted: | |||||||||
< < | This is to cover the CTA pipeline execution ML: I see it as a workflow tracking system | ||||||||
Changed: | |||||||||
< < |
| ||||||||
> > |
| ||||||||
Deleted: | |||||||||
< < | The description of the step is taken from the ActivityDescription definitions A form can be filled by the job subscriptor user: | ||||||||
Changed: | |||||||||
< < | Describe the ActivityDescription record with Param names , UCD , datatypes | ||||||||
> > | Describe the ActivityDescription record with Param names , UCD , datatypes Used , generated section Includes defaults parameter values When the job is run, values are transmitted Provenance graph is generated at the end of job execution and displayed . | ||||||||
Deleted: | |||||||||
< < | Used , generated section Includes defaults parameter values When the job is run, values are transmitted Provenance graph is generated at the end of job execution and displayed . | ||||||||
Changed: | |||||||||
< < | DEPTH param in prov SAP: | ||||||||
> > | DEPTH param in prov SAP: Option All agents - means track them all or show them all ?? To be discussed | ||||||||
Deleted: | |||||||||
< < | Option All agents - means track them all or show them all ?? To be discussed | ||||||||
Changed: | |||||||||
< < | Identifying files locally and across various partners or a common store : | ||||||||
> > | Identifying files locally and across various partners or a common store : Used name space to generate the unique ids. | ||||||||
Deleted: | |||||||||
< < | Used name space to generate the unique ids. | ||||||||
Changed: | |||||||||
< < |
| ||||||||
> > |
| ||||||||
Added: | |||||||||
> > | Log pages to be hooked to the scanning process : you could have a scanning activity . Not bound to a simple Prov object. Not clear . Provenance for light curves. The source for each point comes from a different observation. It makes the graph too crowded. | ||||||||
Changed: | |||||||||
< < | Log pages to be hooked to the scanning process : you could have a scanning activity . | ||||||||
> > | How to distinguish data points provided, from observations provenance ? A source/ rather a detection/ as an entity . How can you adjust the granularity ? | ||||||||
Deleted: | |||||||||
< < | Not bound to a simple Prov object. Not clear . Provenance for light curves. The source for each point comes from a different observation. It makes the graph too crowded. | ||||||||
Changed: | |||||||||
< < | How to distinguish data points provided, from observations provenance ? | ||||||||
> > | ? Michael : Did you try to use summary functions from the ProvToolBox Summaries ? It helps to gather info in a coarse grain view. | ||||||||
Deleted: | |||||||||
< < | A source/ rather a detection/ as an entity . How can you adjust the granularity ? | ||||||||
Changed: | |||||||||
< < | ? Michael : Did you try to use summary functions from the ProvToolBox Summaries ? It helps to gather info in a coarse grain view. | ||||||||
> > |
| ||||||||
Added: | |||||||||
> > |
| ||||||||
Changed: | |||||||||
< < |
| ||||||||
> > | What you pay for provenance : +45% info to store what you get : trust improve 99% for use case 1 , and % ( a bit less for use case 2 | ||||||||
Deleted: | |||||||||
< < | 2 use cases :
| ||||||||
Changed: | |||||||||
< < | b. Was the image badly calibrated due to a bad choice in the standard star? Trace how the standard star is measured and chosen. | ||||||||
> > | UML2PROV Carlos … to generate Provenance helps to upgrade data quality and trust Highly use case dependent : you must track the info that may cause errors or improve trust | ||||||||
Deleted: | |||||||||
< < | What you pay for provenance : +45% info to store what you get : trust improve 99% for use case 1 , and % ( a bit less for use case 2 | ||||||||
Deleted: | |||||||||
< < | UML2PROV Carlos … to generate Provenance helps to upgrade data quality and trust Highly use case dependent : you must track the info that may cause errors or improve trust | ||||||||
—> this feeds the Activity desc and parameters desc | |||||||||
Changed: | |||||||||
< < | How to construct Prov templates ? | ||||||||
> > | How to construct Prov templates ? What is the status ? | ||||||||
Deleted: | |||||||||
< < | What is the status ? | ||||||||
Changed: | |||||||||
< < |
| ||||||||
> > |
| ||||||||
Deleted: | |||||||||
< < | Mainly to track the history of a data set Prov tap is more general Parameters responseFormat | ||||||||
Changed: | |||||||||
< < | ML ? should RESPONSE Format be for graphics format ? | ||||||||
> > | ML ? should RESPONSE Format be for graphics format ? How could we distinguish both outputs? Response : this can be chained to another application : keep only one output | ||||||||
Deleted: | |||||||||
< < | How could we distinguish both outputs? Response : this can be chained to another application : keep only one output | ||||||||
Probably we will have advanced viewers that develops groups of metadata from the viewer hierarchical browsing. | |||||||||
Changed: | |||||||||
< < |
| ||||||||
> > |
| ||||||||
Added: | |||||||||
> > | Shows a ProvSap client interface Sankey representation for graphics Is a possible output format They are exclusive choices . Need to run it again if you want it again as metadata . Then it is not scriptable ? What to do with all the SVG files ? | ||||||||
Changed: | |||||||||
< < | Shows a ProvSap client interface | ||||||||
> > | What means DEPTH ? Relations counting or entity to entity generation layer? | ||||||||
Deleted: | |||||||||
< < | Sankey representation for graphics Is a possible output format They are exclusive choices . Need to run it again if you want it again as metadata . Then it is not scriptable ? What to do with all the SVG files ? | ||||||||
Changed: | |||||||||
< < | What means DEPTH ? Relations counting or entity to entity generation layer? | ||||||||
> > |
| ||||||||
Added: | |||||||||
> > | Draft ready since April, please provide comments Prototype for TAP service : | ||||||||
Changed: | |||||||||
< < |
| ||||||||
> > | This is an implementation on a simple database Tap service working Interface with aladin , topcat, etc . | ||||||||
Deleted: | |||||||||
< < | Draft ready since April, please provide comments Prototype for TAP service : | ||||||||
Changed: | |||||||||
< < | This is an implementation on a simple database | ||||||||
> > | To do to enrich the CDS prototype? insert the 300 Hips files | ||||||||
Deleted: | |||||||||
< < | Tap service working Interface with aladin , topcat, etc . | ||||||||
Changed: | |||||||||
< < | To do to enrich the CDS prototype? insert the 300 Hips files | ||||||||
> > | question Anastasia: How many Gbytes costs the Provenance metadata? | ||||||||
Changed: | |||||||||
< < | question Anastasia: How many Gbytes costs the Provenance metadata? | ||||||||
> > | Store data provenance for any source ? Is it possible? Discuss with the Gaia people ? | ||||||||
Changed: | |||||||||
< < | Store data provenance for any source ? | ||||||||
> > | François: If you can query the data base in SQL, and store all your sources, then TAP can do If you have a unique id for an entity and a dataset publisher id , | ||||||||
Deleted: | |||||||||
< < | Is it possible? Discuss with the Gaia people ? | ||||||||
Changed: | |||||||||
< < | François: If you can query the data base in SQL, and store all your sources, then TAP can do | ||||||||
> > | You can query both sides : the provenance data base , and the Obscore data base , by a data link , by a broadcast mechanism from one data base to the other ? | ||||||||
Deleted: | |||||||||
< < | If you have a unique id for an entity and a dataset publisher id , | ||||||||
Changed: | |||||||||
< < | You can query both sides : the provenance data base , and the Obscore data base , by a data link , by a broadcast mechanism from one data base to the other ? | ||||||||
> > | Give me the provenance of obs-publisherid =12345. —> query provtap or provsap with e-id =12345 And vice versa : find an e-id and build up the query on an Obstap service ….. Pbs with PosgresQL and server settings … May be try docker … a new package can include extra relations into a W3C provenance document and visualize them | ||||||||
Changed: | |||||||||
< < | Give me the provenance of obs-publisherid =12345. —> query provtap or provsap with e-id =12345 | ||||||||
> > | Extensions of the visualisation | ||||||||
Deleted: | |||||||||
< < | And vice versa : find an e-id and build up the query on an Obstap service ….. Pbs with PosgresQL and server settings … May be try docker … a new package can include extra relations into a W3C provenance document and visualize them | ||||||||
Changed: | |||||||||
< < | Extensions of the visualisation | ||||||||
> > | + Working Draft discussion : | ||||||||
Changed: | |||||||||
< < | + Working Draft discussion : | ||||||||
> > | Check the DM Requirements: if an entity exist, there is an Activity to create it. Update the expression of these constraints. Identifiers : must be unique . Replica of an entity ? is another entity. Must be specified as a copy Agreed on : Entities, Activities, Agent must be uniquely identifiable. | ||||||||
Changed: | |||||||||
< < | Check the DM Requirements: | ||||||||
> > | 12 . Contact info should be recorded for all Activities and entities … | ||||||||
Deleted: | |||||||||
< < | if an entity exist, there is an Activity to create it. Update the expression of these constraints. Identifiers : must be unique . Replica of an entity ? is another entity. Must be specified as a copy Agreed on : Entities, Activities, Agent must be uniquely identifiable. | ||||||||
Deleted: | |||||||||
< < | 12 . Contact info should be recorded for all Activities and entities … | ||||||||
i.e. there is always a known Agent linked with them ( even a general one) . | |||||||||
Changed: | |||||||||
< < | What is entityDescription? Do we need it ? Who did implement it? | ||||||||
> > | What is entityDescription? Do we need it ? Who did implement it? | ||||||||
Changed: | |||||||||
< < | How do we describe the role and the constraints to be used by an activity entities when they are to be used by an activity. | ||||||||
> > | How do we describe the role and the constraints to be used by an activity entities when they are to be used by an activity. Used and used description / What is needed for Input params ? An id A title For us we can have a role | ||||||||
Deleted: | |||||||||
< < | Used and used description / What is needed for Input params ? An id A title For us we can have a role | ||||||||
What is needed for output params ? MS: The description side is the degree of freedom for each project to feed its specific metadata. | |||||||||
Changed: | |||||||||
< < | EntityDescription could be one of this placeholder ??? | ||||||||
> > | EntityDescription could be one of this placeholder ??? We need more info about this… more practical usage examples. | ||||||||
Deleted: | |||||||||
< < | We need more info about this… more practical usage examples. | ||||||||
Changed: | |||||||||
< < | Parameter : | ||||||||
> > | Parameter : What is a parameter in our model? A parameter cannot be used or generated by ? Is an entity but restricted… | ||||||||
Deleted: | |||||||||
< < | What is a parameter in our model? A parameter cannot be used or generated by ? Is an entity but restricted… | ||||||||
Changed: | |||||||||
< < | It can be derived from an Entity when it is Data. | ||||||||
> > | It can be derived from an Entity when it is Data. ?? Do we need to say it ?? We coud also setup this rule: If your parameter must be traced , define it as an entity . | ||||||||
Deleted: | |||||||||
< < | ?? Do we need to say it ?? We coud also setup this rule: If your parameter must be traced , define it as an entity . | ||||||||
Changed: | |||||||||
< < | EntityDescription | ||||||||
> > | EntityDescription | ||||||||
Changed: | |||||||||
< < | What would you put here ? | ||||||||
> > | What would you put here ? I would put the required format for my entities to be used with Activitydesc Axx The role | ||||||||
Deleted: | |||||||||
< < | I would put the required format for my entities to be used with Activitydesc Axx The role | ||||||||
Changed: | |||||||||
< < | example : ActivityDescription : regridding | ||||||||
> > | example : ActivityDescription : regridding Constraints input-format=Fits. ??? Mimetype ? output formats= fits errorMapPreviewformats = jpeg | ||||||||
Deleted: | |||||||||
< < | Constraints input-format=Fits. ??? Mimetype ? output formats= fits errorMapPreviewformats = jpeg | ||||||||
Changed: | |||||||||
< < | Sextractor : with particular output styles : .dat --> check for an example to describe | ||||||||
> > | Sextractor : with particular output styles : .dat --> check for an example to describe | ||||||||
Changed: | |||||||||
< < | Do we specialize the table for each particular specialized entities. | ||||||||
> > | Do we specialize the table for each particular specialized entities. Yes for a full archive …. | ||||||||
Deleted: | |||||||||
< < | Yes for a full archive …. | ||||||||
Changed: | |||||||||
< < | What goes into the draft: | ||||||||
> > | What goes into the draft: | ||||||||
| |||||||||
Changed: | |||||||||
< < | this part is important to provide metadata to the Core Model. | ||||||||
> > | this part is important to provide metadata to the Core Model. We explore the PROV-ONE proposition and have also checked the possibility of the W3C Plan . | ||||||||
Deleted: | |||||||||
< < | We explore the PROV-ONE proposition and have also checked the possibility of the W3C Plan . | ||||||||
Changed: | |||||||||
< < | Mention that we consider we could reuse the input and output ports from PROV-ONE | ||||||||
> > | Mention that we consider we could reuse the input and output ports from PROV-ONE Process from PROV-ONE is similar to our Activity Description, but may be not with the same granularity. To be checked . | ||||||||
Deleted: | |||||||||
< < | Process from PROV-ONE is similar to our Activity Description, but may be not with the same granularity. To be checked . | ||||||||
Changed: | |||||||||
< < | — End of day 2 | ||||||||
> > | — End of day 2 | ||||||||
Useful links:Add useful links here<--
|