*DM sessions 49 Participants https://wiki.ivoa.net/twiki/bin/view/IVOA/InterOpOct2022DM *DM1 Oct 18 15h UTC *Matthieu Servillat: One-Step Provenance One Data model (ProvDM) and two protocols related to Provenance - ProvSAP - ProvTAP Reuse of Activity/Agent/Entity W3C approach for provenance IVOA Provenance graphs are complex. Approach followed by Use Cases, requirements gathering and definition of the Open Step Provenance: List of attributes to describe one step of data generation, links to identifiers Metadata proposed for One-Step Provenance record and proposed a YAML format for the documentation of the provenance step Example of implementation: Cherenkov data (opus job manager, job definition is expected input/output) There is a coarse provenance and a possible link to a detailed provenance using voprov python package Questions Markus Demleitner: Agent inside entity is not really clear (clearer in activity as executor of the activity) MS: Agent is used for a detailed contact for the entity itself *Mark Cresitello Dittmar: Onwards to a N-Dimensional Cube Model Cube Model goal: Faciilitate the representation of multi-dimensional cube data products As requirements: Provide common picture (VODML) Identify phsysical properties (Meas and Coords DMs) Support to functional axes/properties (transform model) Access to metadata (dataset data model) Flexible Container (cube model itself) Review of the status of the different standards connected to Cube Description of test cases including support to time series and spectral data from cubes Questions: Francois Bonnarel: How coupled is Cube and Transform models? Could we have a Cube DM before the Transform DM? MCD: For event list it could be done, for images is difficult. Maybe reducing the scope LM: We should reduce the scope to allow the Cube DM sooner MCD: There are complexities (to be reviewed offline) *François Bonnarel: Modeling Instrument Field of View: DM proposal Project already presented during last interop Serialisation is the first implementation (AladinDesktop and AladinLite and also an editor) Old system used for HST, CFHT, etc. Approach restarted with a more interoperable one FB presents the data model structure Approach followed by extending the coords data model Serialisation is making use of MIVOT that is not Proposed Recommendation (that already maps instances to Coords, Meas, PhotDM, Mango on top of VOTables) Showing the implementation on Aladin and the java editor Questions: Alberto Micol: Single Instrument into the data model. Is it possible to provide a footprint without an instrument? FB: We are working with a FoV class that it could be used for this case instead of using the full model AM: Could it be used as a replacement for the ObsCore serialisation? FB: It is not foreseen for that. There it is more for the FoV for one particular observation but it is not the detailed one Jesus Salgado: Is it in the plans to integrate this in the JWST proposal system (APT) (it is using the old serialisation and AladinDesktop connection). It would be a really powerful reference implementation FB: Whenever we have the rendering (colors and labels) it could be done *Paul Harrison - VO-DML Tooling Update Gradle is used as the standard for VO-DML tooling now Added functionality for the ProposalDM design Serialization is natural vs mapping to VOTable one It could be translate between MIVOT and this serialisation Review of XML serialisation Updated from previous one, including references first. New serialisation in Json RDB serialisation, main design decisions taken in the object relational mapping New: vodmlPythonGenerate gradle task (generates data classes form the vo-dml documentation) Questions: FB: What is the benefit of your serialisation in comparison with MIVOT? PH: It is more compact and easy to read. I think both serialisation could coexist. It looks MIVOT is more to retain information from services LM: Annotation data on the fly needs model components rather than complete model serialization in my view. PH: Multimodel support from MIVOT could complicate the serialisation *DM2 Oct 20 15h UTC Particiants: 37 *Laurent Michel - MIVOT Annotation Syntax, What is for Model Instance In Votable MIVOT defines how to annotate data models within IVOA. A bit change in the way data is consumed but an interesting feature. Today's presentation is to describe the benefits of this starndard Simple case described with a MIVOT representation of a mapping of PhotDM elements into a VOTable response Two elements in the VOTable, annotation and regular VOTable Same client should be able to read any kind of annotation Going beyond, e.g. annotation of positional elliptical errors. If client is able to read the MIVOT annotation, the object could be regenerated Same pattern could be reused to include error, meassure, MANGO, etc instances Example shown on how to transfer annotations also using SAMP between applications On the server side, mapping process should identify the MIVOT components to map the VOTable On the client, e.g. using XPath getters and getters based on @dmrole, the attributes could be read *Tom Donaldson - The road to implement MIVOT in AstroPy/PyVO Desrcription of ecosystem of affiliated packages For parsing VOTables, astropy has a VOTable parser Revision of astropy pyVO components related to MIVOT Main code into pyvo directory/utils protofeature py Continue with development process to include forks (only 1 needed) and pull requests for pyvo Clone your github repository into your local machine using your own copy https://github.com//pyvo.git Ensure synch with git fetch and create a branch Description of creation of local environment using conda and installing usin pip the packages Also, use of tox for tests and codestyle checks Recommendation of draft PRs to estimulate early discussion Also, request reviewers *Gilles Landais - Data Model in TAP schema Rich metadata through VO, using mivot and first motivation is to focus into VizieR Photometry and the Data origin Using JSON templates for the VizieR workflow Mapping between VizieR schema and data models First step was the mapping photometry Filters added by CDS documentalist Filters mapped from a local list at CDS (created from info from the SpanishVO FPS) A new table in TAP_Schema called phot added with the relevant information to mapping photometry points New table could be discovered by using a TAP client (e.g. Topcat) ADQL function cds_mag_to_jansky allows automatically conversion Data Origin mapping also used to map DatasetDM, by using a TAP_SCHEMA.dataset Proposal to enhance TAP_SCHEMA with rich metadata to allow to include inputs to create annotations MCD: Just to comment on the last slide that since the data set metadata model is one of the ones that we're thinking of getting back or working to get back into production, those topics will certainly be one So we can discuss FB: Is it allow to have new tables in TAP_SCHEMA? GL: As soon as you do not modify original tables, I think it could be done FB: Why in data_schema and not associated with the data_schema.tables GL: In this case it is 1-1 but in other cases you need a join table that could produce that the solution is not very clean XiuqinWu: We are in the process of annotation photometry so we will see into this particular solution GL: It could be really good to reuse the same solution GregoryDubois: i like the way it is propossed here as we also have the same photometry in different columns *Mireille Louys - Using MIVOT annotation in PhotDMv1.1 serialisation PhotDM 1.1. ahs become IVOA recomendation this week (18Oct) Revision of PhotDM 1.0 preserving classes and adding simple value attriutes for all the classes Documentation includes vodml.xml and html pages Description of the class diagram model Main class PhotCal with the zero point and it is possible to navigate to PhotometryFilter In MIVOT, metadata for Photometric Calibration, some examples are shown in the serialisation (Serialisation shown, including MIVOT annotations). Annotations include in one resource called meta Also, review the filter profile service implementation from SVO. Wrapper create to annotate the output using a wrapper into a PhotDM 1.1 version Also, photometry 1.1. could be used to annotate on the fly using MANGO data model FB: I was wondering if you have found concrete the FPS with the new format (maybe in astropy?) ML: The current VOTable output the annotations are in a object consistent format JS: Maybe Gilles proposal could include a call to the FPS to maintain the metadata up to date Discussion on workflow on pyvo (i.e. preparation to pull request merging by running locally the tests)