*IVOA Interop April 2022 *Data Model WG session 1: Data Models Date: Wednesday April 27 2022, 15:00 UTC Participants: * Paul Harrison - VO-DML Tooling Gradle Tooling: text-represented model-first development with code generation Tool already standard to cover ProposalDM Generated Java and VO-DML of model packaged into jar Added XMI ask for UML->VO/DML Integrated with PhotDM Demo on gradle code generation on PhotDM: complete generation of documentation, graph and java code VO-DML feedback intentified during implementation and proposals done Questions OL: Some initial precautions with VO/DSL that now have been improved. Probably mixing with VO/DML as language with the grammar integrated PH: Maintain both decouple makes VO/DSL concise * Mathieu Servillat - Last-Step Provenance Provenance DM and system allows the change from FAI to FAIR as Reusability depends on the quality/reliability/trustworthiness of the products Provenance information should be detailed enough. This is why IVOA provenance DM, ProvSAP and ProvTAP have been created ProvenanceDM is based on w3c provenance including activities, entities and agents Described pipeline from raw data to data products and activities involved Full provenance includes graph/tree/chain end-user proveance includes jey information for users last-step provenance includes general workflow, software, versions, etc Provenance graphs are complex so this should be simplified to allow a simple (flat) representation Keyword list entity/agent explained Difficult to serialize these parameters in FITS Questions JS: How far we are from Provenance to reproduce workflows? MS: Provenance is not defined for that but maybe could be reproduced step by step LM: Some exercises done in this line. Still some elements missing Comment Mireille: If the workflow is expressed as some published document, for instance as a cwl or yaml description, the workflow part in last step could be a reference to one step into the general workflow document. worth to try. The connexion to standardized software description, like Software Heritage records for instance , is also interesting when such software are published . * Paul Harrison - Observation Proposal DM Model to uniform a common proposal DM that allows wasy interaction with other observatories Proposal DM documentation generated automatically with gradle Eclipse plugin allows navigation between graphical representation and elements Connections with other IVOA elements: STC Coords Catalogue source DataSetMetadata Observatory/Instrument Metadata ObsLocTAP Ultimate model source code is vodsl which is translated to normative VO/DML Hack-a-thon tomorrow * Francois Bonnarel - Instrument Field of View DM and mapping in VOT Aladin has a functionality to overimpose instrumental FOVs on the sky The idea would be to standardise the representation Restart the project by deriving a new FoV data model from Coords Using MIVOT mapping synta to describe model elements Explore use outside Aladin Exetnding coords model with coordinates classes and coordinate system classes Serialization/Mapping using MIVOT: three flavours, fixed, multicentric and interactive TomDonaldson: One of features important is the availabiliy to change the center and rotation. is this preserved in this model? FB: Yes, it is preserved and it can be rotated around the refposition MCD: how dali regions could be integrated into this model. DALI has some DALO regions flats FB: we have not considered shapes as strings JS: Use case behind this proposal? FB: First standardise and replace in-house Aladin representation. Then, open to multiple uses PatrickD: Probably mapping of shapes could imply a internal debate on this. Also Pickle looks to be a particular shape that could be generalised using more geometrical definitions PD proposal: the definition of geometric shapes can become a type in the ivoa vodml package , and then be reused by different projects ? PierreF: Multiple requests from observatories to add this FoVs into Aladin so standardization would be good for the IVOA *Notes *Data Model WG session 2: Data Models Date: Thursday April 29 2022, 05:00 UTC Participants: 36 * Laurent Michel - Annotation Syntax Model View annotation added into the metadata section of VOTables Model mapping is one XML block located into a resource type=meta Inside a tab with one REPORT, MODEL, GLOBALS and TEMPLATES One TEMPLATE per mapped table through tableref to TABLE NAME, ATTRIBUTEs linked to TABLE FIELDS by ref to IDs Objects are defined with an INSTANCE with a dmrole and ATTRIBUTES have DMTYPEs and VALUEs REFERENCE iinside INSTANCE is used to make reference to other INSTANCES using dmref JOINS between different tables can be also defined with WHERE elements Check ivoa wiki and github for more info Questions: MD: What happens to tableref when a template is filled from multiple tables? LM: This is done through different TEMLATE elements and you can use JOIN to link both * Mireille Louys - Annotating TAP Responses on the Fly Test case deals with catalogs of astronomical sources Includes Coords, Measure, PhotDM binding them through ATRIBUTES Inputs are VODML XML elements that are used to create XML componets for the different attributes. Also there is a JSON metadata profile prepared byt the data provider that describes how their metadata map to the different VO models A mapping engine generated the annotation Short demo Questions: MarkTaylor: JSON profile is written by hand? ML: yes MT: Any expectation to do it in a different way? LM: not yet GillesLandais: Mapping in the TAP schema? It could be simpler * Laurent Michel - Implementation Status, issues and Prospects We currently have: - VO working draft - Annotation generation (including Jovial DSL library) - Anotation readout in python Pros of his is that data is umabigouus Cons is more work for the curators Explanation of the processes to create objects using the serialised metatada b composing the objects from the metadata included in the model view and into the VOTable tables In this exercise, a python instance would be the goal Hack-a-Thon at 13:30 Different notebooks with examples of the process Meas/Coord + unit conversion + distance computation Astropy SkyCoord PhotDM Questions: TomDonaldson: Notebooks can be run locally but also in mybinder? LM: yes MT: What is the level of maturity? LM: Mapping syntax is already a working draft. Not too many changes expected FB: As we had two proposals, this combination looks now stable MT: Taking the difficulty of the JSON profiles, how this is expected to be spread by data providers? LM: JSON profile is not part of the standard. If we have tools to generate them, maybe we could have repositories TomDonaldson: Already many metadata is matched manually *Notes Jesus Salgado - SKAO to Everyone (07:03) https://yopad.eu/p/IVOA_Apr22_DM-365days Markus Demleitner to Everyone (07:09) What happens to tableref when a template is filled from multiple tables? Markus Demleitner to Me (Direct Message) (07:13) For the uploaded slides, please change "ICRS" to FK5 in your "basic object instance" -- ICRS doesn't have equinoxes... Me to Markus Demleitner (Direct Message) (07:18) your qre right Tom Donaldson to Everyone (07:18) Are there any current DAL standards with multi-table results? Me to Everyone (07:21) not yet, but this feature is supported by VOTable and it may be used to serialize complex data sets Tom Donaldson to Everyone (07:25) 👍 Me to Everyone (07:28) The next release of the XMM catalogue TAP service hosted at STrasbourg will deliver annotated VOTables for those who want to play with annotations (based on the presneted work) Jesus Salgado - SKAO to Me (Direct Message) (07:36) gerard tool to generate the JSON profile? Tom Donaldson to Everyone (07:38) This is a very nice proof of concept! James Tocknell to Everyone (07:54) Is mybinder still working, I thought they had some funding issues? It's a nice service though