(r4) Mango10RFC < IVOA

IVOA Web>Mango10RFC (revision 4) (raw view)~~EditAttach~~
---+ MANGO 1.0: Proposed Recommendation: Request for Comments

%TOC{depth="2"}%

There is a paradox in the VO world: source data which represent the basic building blocks of astronomers’ work, is not modelled. This paradox can be explained by the fact that the observation of source objects is multifaceted.<br />In a general way, the way features for source data are described and organised depends on the targeted science case.Principal investigators and archive designers set up the data profile and structure it according to this goal which varies from one project to another. Therefore this diversity cannot be served by a single static data model describing a source item for all possible cases. <br />Having a global source model would lead to a very complex solution not usable in practice.<br />This standard proposes to overcome this paradox and presents a template model gathering independent components from VO existing models embedded on demand in a container. MANGO is not designed to describe what a source is but to help clients to discover and to understand the quantities available for a particular source instance.

VOtable data mapped on MANGO with Mivot annotations can be consumed quantity par quantity instead of column per column. The way complex quantities are built is described by MANGO but no longer by the clients.

The main purpose of MANGO is to add an upper description level to the tabular data of query responses. MANGO is not designed to replace the meta-data already present in query responses,<br />Uses-cases have been collected since 2019 from representatives of various astronomical missions, archive designers and tools developers. These open contributions gave a good picture of the needs but we do not pretend that everything will be supported by this first version.

Latest version of IVOASTANDARD can be found at:
   * URL link to the document

---++ Reference Interoperable Implementations

The project contains 4 static files, shown as working examples located in _PROJECT_HOME/implementations._ <br />They cover most of the model features (see coverage <a href="https://github.com/ivoa-std/MANGO/blob/pr-1.0/implementations/README.md" target="_blank" title="table">table</a>). In addition to this, we have 2 services (Vizier, XMM) providing annotated data compliant to MANGO DM.

The term "reference implementation" has never been clearly defined in the context of a model. In the case of MANGO, whose purpose is to restore complex quantities from tabular data, we provide static or dynamic VOtables, whose data are mapped onto MANGO using the Mivot schema. Additionally, we consider the Pyvo Mivot API to be another type of implementation.
---+++ Static Files

4 references files implementing most of the model features are provided within the _implementations_ [[https://github.com/ivoa-std/MANGO/blob/pr-1.0/implementations][folder]] in the Git repository.
The detail of the covered properties is givel by this <a href="https://github.com/ivoa-std/MANGO/blob/pr-1.0/implementations/README.md" target="_blank" title="table">table</a>.
<ul dir="auto"> <li> <p dir="auto"> *xtapdb*:TAP query response on the 4XMMdr14 slim catalog. The VOTable has been annotated on the flight by the server. <br />A link binding the postion with a quality flag has been added by hand.</p> <ul dir="auto"> <li> *file*: =xtapdb.xml= </li> <li> *service*: =https://xcatdb.unistra.fr/xtapdb= </li> <li> *query*: =SELECT TOP1 * FROM "public".mergedentry= </li> <li> *format*: =application/x-votable+xml;content=mivot= </li> </ul></li> <li> <p dir="auto"> *Gaia*:Vizier query response on the GAIA DR3 catalog. The VOTable has been annotated on by hand with the Pyvo MIVOT API.</p> <ul dir="auto"> <li> *file*: =gaia_with_mivot.xml= </li> <li> *service*: =https://vizier.cds.unistra.fr/viz-bin/votable= </li> <li> *query*: see the =mango:origin.QueryOrigin= instance</li> </ul></li> <li> <p dir="auto"> *datalink*:Query on an Obscore table made to exercise the mapping on both =Footprint= and =Datalink= classes. The VOTable has been annotated on by hand on the project purpose.</p> <ul dir="auto"> <li> *file*: =datalink.xml= </li> <li> *service*: =https://ws.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/argus= </li> <li> *query*: =SELECT TOP 1 ivoa.ObsCore.obs_id,ivoa.ObsCore.s_region,ivoa.ObsCore.access_url FROM ivoa.ObsCore= </li> </ul></li> <li> <p dir="auto"> *vizier_cs_I_239*:Cone search response on the I/239 Vizier table. The VOTable has been annotated on the flight by the server.</p> <ul dir="auto"> <li> *file*: =vizier_cs_I_239.xml= </li> <li> *service*: =https://cds/viz-bin/conesearch/V1.5/I/239/hip_main= </li> <li> *query*: =RA=0&DEC=0&SR=0.5= </li> </ul></li> </ul>

The detail of the covered properties is given in the README file.
---+++ Services

There are two services in operation and serving data mapped on MANGO.
---++++ XTapDB (HE Data and Photometry)

A TAP service serving XMM catalogue (<a href="https://xcatdb.unistra.fr/xtapdb/" target="_blank">https://xcatdb.unistra.fr/xtapdb/</a>) data (table _mergedentry_) has been deployed.

This service demonstrates the ability of MANGO to properly describe the following quantities
   * Coordinate systems (space, time imported from Coords DM)
   * Position at a given epoch
   * Photometric filters and calibration (imported from Phot DM)
   * Fluxes and hardness ratio
   * Quality flags
Only quiered quantities are annoated.

The service can be connected with <a href="https://saada.unistra.fr/taphandle?url=https%3A//xcatdb.unistra.fr/xtapdb/" target="_blank">Taphandle</a> or by using curl (or any other client enable to set a specific format)):<b><br /><br /></b>
   * *url:* _https://xcatdb.unistra.fr/xtapdb/sync?LANG=ADQL&REQUEST=doQuery&QUERY=[QUERY]&FORMAT=application%2Fx-votable%2Bxml%3Bcontent%3Dmivot_
   * *query*: SELECT TOP1 * FROM "public".mergedentry
   * *format*: application/x-votable+xml;content=mivot
   * <b>example: </b>copy this [[https://xcatdb.unistra.fr/xtapdb/sync?LANG=ADQL&REQUEST=doQuery&QUERY=SELECT%20TOP%201%20%2A%20FROM%20%22public%22.mergedentry&FORMAT=application%2Fx-votable%2Bxml%3Bcontent%3Dmivot][URL]] in your clipboard.
---++++ Vizier Endpoints

There are 2 Vizier services returning VOtable partially mapped on the EpochPosition MANGO property.

   * ConseSearch: Vizier deployed a cone search service that returns VOtable partially mapped on MANGO
      * URL: <a href="https://vizier.cds.unistra.fr/viz-bin/conesearch/V1.5/TABLE_ID" target="_blank">https://vizier.cds.unistra.fr/viz-bin/conesearch/V1.5/TABLE_ID</a>
      * Example _curl "https://vizier.cds.unistra.fr/viz-bin/conesearch/V1.5/I/329/urat1?RA=52.26708&DEC=59.94027&SR=0.05"_
   * ASU: proprietary URL based protocol to query catalogs.
      * URL: [[https://vizier.cds.unistra.fr/viz-bin/votable/V1.5?-source=I/239/hip_main&-out.max=10][https://vizier.cds.unistra.fr/viz-bin/votable/V1.5?-source=TABLE_ID&-out.max=10]]
      * Example: _curl https://vizier.cds.unistra.fr/viz-bin/votable/V1.5?-so urce=I/239/hip_main&-out.max=10_
These services have been officially [[https://cds.unistra.fr/news][released]] in May 2025.
---+++ Python API

An important new module is being added to Pyvo. This module's purpose is to write and read Mivot annotations, with a particular focus on MANGO (see the [[https://pyvo.readthedocs.io/en/latest/mivot/index.html][doc]]).
   * *Reader* (since Pyvo 1.6):
      * Mivot annotations are rendered as dynamic Python objects that obey the model structure. This allows data to be explored through the model view.
      * If the data id mapped onto the EpochPosition property, this API can automatically extract SkyCoord instance.
   * <b>Generic writer (</b>since Pyvo 1.6):
      * Allows to build Mivot annotation element per element (attribute, instance, collection...). This API has been made for developpers to build mode user-friendly functionalities.
   * *Mango writer* (to be released with PyVO1.8)
      * Write coordinate frames (space, time ang photmétric calibration) that are part of Mago instances
      * Map data on _EpochPosition_, _Brightness_ and _Color_ properties. This is faciltated by a meta-data analizer included in the code.
---+++ Implementations Validator

VOTable whose data are mapped onto MANGO can be validated with _mivot-validator:_
   * Package on [[https://pypi.org/project/mivot-validator/][Pypi]]
   * Doc on [[https://app.readthedocs.org/projects/mivot-validator/][readthedocs]]

The validator operates at 3 levels

   * Validation against the VOTable schema
   * Validation against the Mivot schema
   * Validation of the annoatation agans the models they refer to

---++ Notre on the Repository

The process of building the document is very specific to the project. he workflow has been set up to make sure the diagrams keep in sync with the specified model.

   * The project is hosted by <a href="https://github.com/ivoa-std/MANGO" target="_blank">GitHub</a> as a regular _ivoa-std_ project
   * The working draft <a href="https://github.com/ivoa-std/MANGO/releases" target="_blank">PDF</a> is regularly udpdated
   * The descriptions of the model elements are stored in individual files located in _PROJECT_HOME/vo-dml/desc_. <br />This allows for an easy update. It also delegates the latex escape character handling to the builder tool.
   * The table of content of the normative sections is given by _PROJECT_HOME/utils/toc.json_
   * The document is built by the _PROJECT_HOME/utils/buildDoc.bash_ script:
      * Insert the description of all model elements in<i> mango.vo-dml.xml</i> and store the result in __desc.mango.vo-dml.xml__ which is the *model reference file.*
      * Escape some characters to please to latex
      * Build the _PROJECT_HOME/doc/model.tex_ file which contains the Latex of the model description (section 4). <br />This step is achieved by the XSLT developed by M. Cresitello Dittmar.
      * Update the publication date
      * Build the pdf with the _ivoatex_ tools
   * The <i>PROJECT_HOME/mivot </i>folder contains MIVOT snippets for the main classes. <br />This allows contributors to figure out what the MIVOT annotation for MANGO could look like.<br />These snippets have been generated with the _mivot-validator_ package (available on pip)
<b>Contributor Watchout: </b>If you want to open a PR, please *do not edit _PROJECT_HOME/doc/model.tex_* but the snippet files in __PROJECT_HOME/vo-dml/desc__ and then run __PROJECT_HOME/utils/buildDoc.bash__.

A brief <a href="https://wiki.ivoa.net/twiki/bin/edit/Href=Https://github/com/ivoa-std/MANGO/blob/wd-v1/0/utils/README/Md?topicparent=IVOA.MANGO-1_0;nowysiwyg=0" rel="nofollow" title="note (this topic does not yet exist; you can create it)">note</a> details the process.
---++ Comments from the IVOA Community during RFC/TCG review period: RFC_start_date - RFC_end_date

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

   * Sample comment by TWiki.WikiName
      * Response (by TWiki.WikiName)

---
---

---++ Comments from TCG member during the RFC/TCG Review Period: TCG_start_date - TCG_end_date

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

---+++ TCG Chair & Vice Chair

---+++ [[IvoaApplications][Applications Working Group]]

---+++ [[IvoaDAL][Data Access Layer Working Group]]

---+++ [[IvoaDataModel][Data Model Working Group]]

---+++ [[DistributedServicesAndProtocols][Distributed Services & Protocols Working Group]]

---+++ [[IvoaResReg][Registry Working Group]]

---+++ [[IvoaSemantics][Semantics Working Group]]

---+++ [[IvoaDCP][Data Curation & Preservation Interest Group]]

---+++ [[IvoaEducation][Education Interest Group]]

---+++ [[HEGroup][High Energy Interest Group]]

---+++ [[IvoaKDD][Knowledge Discovery Interest Group]]

---+++ [[IvoaOps][Operations Interest Group]]

---+++ [[IvoaRadio][Radio Astronomy Interest Group]]

---+++ [[IvoaSS][Solar System Interest Group]]

---+++ [[IvoaVOEvent][Time Domain Interest Group]]

---+++ [[IvoaStdsDocsProc][Standards and Processes Committee]]

---

---++ TCG Vote : Vote_start_date - Vote_end_date

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

| Group | Yes | No | Abstain | Comments |
| TCG | | | | |
| Apps | | | | |
| DAL | | | | |
| DM | | | | |
| DSP | | | | |
| Registry | | | | |
| Semantics | | | | |
| DCP | | | | |
| Edu | | | | |
| KDIG | | | | |
| Ops | | | | |
| Radio | | | | |
| SSIG | | | | |
| TD | | | | |
| &lt;nop&gt;StdProc | | | | |

---

<br /> &lt;!--<br />   * Set ALLOWTOPICRENAME = IVOA.TWikiAdminGroup <br />--&gt;
Topic revision: r4 - 2025-05-27 - LaurentMichel
IVOA
Log in or Register
IVOA.net
Wiki Home
WebChanges
WebTopicList
WebStatistics
Twiki Meta & Help
IVOA
Know
Main
Sandbox
TWiki
TWiki intro
TWiki tutorial
User registration
Notify me
Working Groups
Interest Groups
Time Domain
Committees
Stds&Procs
www.ivoa.net
Documents
Events
Members
XML Schema