# Follow-up of the discussions on the design of a AstroPyVO API processing annotated data.

At last Interop (April 2022), we have had a PyVO hands-on session where we have open a discussion on the requirements for a PyVO API processing annotated data.

The minutes of that session are on GitHUB

https://github.com/ivoa/modelinstanceinvot-code/wiki/hack-a-thon-contributions

We all agreed that this new API must have no impact on the current one and that its design must be based on real use-cases.

The expectation for using annotated data is not necessary to allow doing new things. It is rather to facilitate common operations.To move on, we have decided to start from some typical PyVOT codes and to figure out how to do the same things with a code based on annotated data.

The rule is to write realistic pseudo-code.We are proposing now a brief discussion focused on a first sketch based on the proposal presented by Gilles in May 2021 and taking advantage of the experience we got while exercising MIVOT parsing.

https://github.com/ivoa/modelinstanceinvot-code/wiki/vizier-proposal

Meeting Wiki page:

https://github.com/ivoa/modelinstanceinvot-code/wiki/21_07_22/_edit

Note of the meeting held on 2022 July 21

Go on the GitHub Wiki to get more details about the work progress

Attendees - 6

Laurent Michel LM, Renaud Savalle RS, Tom Donaldson TD, Omar Laurino OL, Mark CD MCD, Jesus Salgado JS

Meeting notes

LM: Presentation of a brief introduction on the subject, PyVO-DM

The proposed model facilitate the way to map DMs into VOTables and how to read these VOTables in a efficient way in python using astropy (and PyVO)

In the client side, there are two things to do with this annotation: extract the mapping block adn and process this mapping block

Also, write the python instances that map the data model instances (e.g position, photometry, etc)

The focus of today is design and astropy API compliant with the native API and supports working with models

As a result of the hackathon, pages under https://github.com/ivoa/modelinstanceinvot-code/wiki were created

_Examples created by Vizier (Gilles Landais) described at https://github.com/ivoa/modelinstanceinvot-code/wiki/vizier-proposal__

_as they have experience on the difficulties of mapping dm elements into vizier tables

Gilles propose to iterate on measures on the resources and for each one obtain the values (there is another examople for spectral data that has not been explored further)

There is an interpretation of this pseudocode into pyvo code including a query to the service, requesting dm_mapping. That could allow to created mangoinstances from the response and obtain a modelview using modelviewer

That allows the navigation of the result in terms of mangoObjects.

It can be also iterated on the object parameters to get the values of the labels

Example of the parsing of a PhotometryFilter using the model description

MCD: the sentence containsMangoInstances could be false for, e.g. cubes or time series (the object is fully returned)

OL: how is this compared with the library RAMA?

TD: we need to have client code to navigate the vo/dml

LM: the example showed is based on mango but the idea is to create astropy classes to do the parsing. The process has been divided in three phases 1 parse VO/Table 2 work with the parameters inside classes 3 create classes

OL: Why you did not start with the library RAMA as most of the requirements are already implemented there?

LM: it was started the exercise as RAMA was not compatible with the mapping syntax

OL: yes, but this is a detail that could be implemented as other things are already there

The main target for today is to discuss about the model components vs astropy components at

https://github.com/ivoa/modelinstanceinvot-code/wiki/21_07_22

MCD: if the idea is to create an skycoord instance from an annotation, the first thing is to study the constructor

TD: it is an overloaded constructor and "humans" knowing the location of the columns create the instances using a simple constructor version

There is a affiliated package called SpecUtils that could be explored

We just draft some ideas on the aspect of the API during the last interop. We could check RAMA and I have some ideas on how to handle this in astropy

LM: Table with the correspondance between the model instances and astropy types (in the case of polarization, photometry, etc it would be astropy quantity)

For cases like PhotCal and Flag, there is not a clear correspondance with astropy

Also there is not clear link with magnitudes and error, it does not support data grouping and there is not a way to iterate over the list of available measures

MCD: it could be used an array for iteration

LM: but associated values are difficult to link

The path is to reuse API astropy classes, prevent the use of affiliated classes, extend astropy for quantities, extend to support, missing metadata, errors and data grouping

TD: we should not be just limited to astropy classes as many metadata is not included into the astropy classes and it should be available for clients

I think astropy core team will resist to include all these new classes but what we can do is upgrade the VOTable parser ensuring that we can load all the required metadata

PyVO is the natural place to put these extended classes

LM: so the idea would be to extend astropy adding these classes to pyVO

Community Notes

Previous Work

I note in passing that (for a simpler annotation) I've implemented an API and showed how it could be used in an astropy branch: https://github.com/msdemlei/astropy

And I would agree that most of this needs to go into astropy rather than pyVO, in particular basically everything dealing dealing with the annotation of coordinate metadata and error annotation.

This topic: IVOA > WebHome > IvoaDataModel > AstroPyVOMivot
Topic revision: r3 - 2022-07-25 - MarkusDemleitner