AstroPyVOMivot < IVOA

---+ # Follow-up of the discussion on design of a AstroPyVO API processing annotated data.

This page
---++ Note of the meeting held on 2022 July 21
<div id="magicdomid2"><strong><br /></strong></div> <div id="magicdomid5"> *Advertisement* </div> <div id="magicdomid7">At last Interop, we have had a PyVO hands-on session where we have open a discussion on the requirements for a PyVO API processing annotated data.</div> <div id="magicdomid8">The minutes of that session are on GitHUB </div> <div id="magicdomid9">
   *   https://github.com/ivoa/modelinstanceinvot-code/wiki/hack-a-thon-contributions
</div> <div id="magicdomid10">We all agreed that this new API must have no impact on the current one and that its design must be based on real use-cases.</div> <div id="magicdomid11">The expectation for using annotated data is not necessary to allow doing new things. It is rather to facilitate common operations.To move on, we have decided to start from some typical PyVOT codes and to figure out how to do the same things with a code based on annotated data. </div> <div id="magicdomid12">The rule is to write realistic pseudo-code.We are proposing now a brief discussion focused on a first sketch based on the proposal presented by Gilles in May 2021 and taking advantage of the experience we got while exercising MIVOT parsing.</div> <div id="magicdomid13">
   * https://github.com/ivoa/modelinstanceinvot-code/wiki/vizier-proposal
</div> <div id="magicdomid15">Meeting Wiki page: </div> <div id="magicdomid16">
   * https://github.com/ivoa/modelinstanceinvot-code/wiki/21_07_22/_edit
</div>

<div id="magicdomid26"> *Attendees - 6* </div> <div id="magicdomid27">Laurent Michel LM, Renaud Savalle RS, Tom Donaldson TD, Omar Laurino OL, Mark CD MCD, Jesus Salgado JS</div>

<div id="magicdomid29"> *Meeting notes* </div> <div id="magicdomid31">LM: Presentation of a brief introduction on the subject, PyVO-DM</div> <div id="magicdomid32">The proposed model facilitate the way to map DMs into VOTables and how to read these VOTables in a efficient way in python using astropy (and PyVO)</div> <div id="magicdomid33">In the client side, there are two things to do with this annotation: extract the mapping block adn and process this mapping block</div> <div id="magicdomid35">Also, write the python instances that map the data model instances (e.g position, photometry, etc)</div> <div id="magicdomid37">The focus of today is design and astropy API compliant with the native API and supports working with models</div> <div id="magicdomid39">As a result of the hackathon, pages under https://github.com/ivoa/modelinstanceinvot-code/wiki were created</div> <div id="magicdomid41">Examples created by Vizier (Gilles Landais) described at https://github.com/ivoa/modelinstanceinvot-code/wiki/vizier-proposal </div> <div id="magicdomid42">as they have experience on the difficulties of mapping dm elements into vizier tables</div> <div id="magicdomid44">Gilles propose to iterate on measures on the resources and for each one obtain the values (there is another examople for spectral data that has not been explored further)</div> <div id="magicdomid46">There is an interpretation of this pseudocode into pyvo code including a query to the service, requesting dm_mapping. That could allow to created mangoinstances from the response and obtain a modelview using modelviewer</div> <div id="magicdomid48">That allows the navigation of the result in terms of mangoObjects.</div> <div id="magicdomid49">It can be also iterated on the object parameters to get the values of the labels</div> <div id="magicdomid51">Example of the parsing of a PhotometryFilter using the model description</div> <div id="magicdomid53">MCD: the sentence containsMangoInstances could be false for, e.g. cubes or time series (the object is fully returned)</div> <div id="magicdomid55">OL: how is this compared with the library RAMA?</div> <div id="magicdomid57">TD: we need to have client code to navigate the vo/dml</div> <div id="magicdomid59">LM: the example showed is based on mango but the idea is to create astropy classes to do the parsing. The process has been divided in three phases 1 parse VO/Table 2 work with the parameters inside classes 3 create classes</div> <div id="magicdomid61">OL: Why you did not start with the library RAMA as most of the requirements are already implemented there?</div> <div id="magicdomid63">LM: it was started the exercise as RAMA was not compatible with the mapping syntax</div> <div id="magicdomid65">OL: yes, but this is a detail that could be implemented as other things are already there</div> <div id="magicdomid67">The main target for today is to discuss about the model components vs astropy components at</div> <div id="magicdomid68"> https://github.com/ivoa/modelinstanceinvot-code/wiki/21_07_22 </div> <div id="magicdomid70">MCD: if the idea is to create an skycoord instance from an annotation, the first thing is to study the constructor</div> <div id="magicdomid72">TD: it is an overloaded constructor and "humans" knowing the location of the columns create the instances using a simple constructor version</div> <div id="magicdomid73">There is a affiliated package called SpecUtils that could be explored</div> <div id="magicdomid74">We just draft some ideas on the aspect of the API during the last interop. We could check RAMA and I have some ideas on how to handle this in astropy</div> <div id="magicdomid76">LM: Table with the correspondance between the model instances and astropy types (in the case of polarization, photometry, etc it would be astropy quantity)</div> <div id="magicdomid78">For cases like PhotCal and Flag, there is not a clear correspondance with astropy</div> <div id="magicdomid79">Also there is not clear link with magnitudes and error, it does not support data grouping and there is not a way to iterate over the list of available measures</div> <div id="magicdomid81">MCD: it could be used an array for iteration</div> <div id="magicdomid82">LM: but associated values are difficult to link</div> <div id="magicdomid84">The path is to reuse API astropy classes, prevent the use of affiliated classes, extend astropy for quantities, extend to support, missing metadata, errors and data grouping</div> <div id="magicdomid86">TD: we should not be just limited to astropy classes as many metadata is not included into the astropy classes and it should be available for clients</div> <div id="magicdomid87">I think astropy core team will resist to include all these new classes but what we can do is upgrade the VOTable parser ensuring that we can load all the required metadata</div> <div id="magicdomid88">PyVO is the natural place to put these extended classes</div> <div id="magicdomid90">LM: so the idea would be to extend astropy adding these classes to pyVO</div>

<!--
      * Set ALLOWTOPICRENAME = IVOA.TWikiAdminGroup
-->
This topic: IVOA > WebHome > IvoaDataModel > AstroPyVOMivot
Topic revision: r1 - 2022-07-22 - LaurentMichel