TWiki> IVOA Web>Mango10RFC (2025-12-11, BrigittaSipocz) EditAttach

MANGO 1.0: Proposed Recommendation: Request for Comments

In the VO world, there is a paradox: source data, which represent the basic building blocks of astronomers' work, is not modeled.
Astronomical sources and their measured data, the key to astronomers' work, have no dedicated IVOA data models yet.
This paradox can be explained by the multifaceted nature of observed source objects.

Generally, the way features of source data are described and organized depends on the targeted scientific objective.
Principal investigators and archive designers set up the data profile, structuring it according to their goals, which vary from one project to another.
Therefore, a single static data model cannot serve this diversity by describing a source item for all possible cases.

A global source model would lead to an overly complex and impractical solution.
This standard proposes overcoming this paradox by presenting a template model that gathers independent components embedded in a container on demand.
MANGO is not designed to describe what a source is, but rather to help clients discover and understand the quantities available for each source instance in a catalog.

In a list of sources — typically a catalog — each column has a meaning that can be associated with a property covered by MANGO.
Thus, MANGO helps build an upper description level for tabular query responses and homogenizes descriptions of relevant quantities.
The metadata already present in the table columns can be annotated using MIVOT to facilitate catalog comparison, discovery, and analysis.

Latest version of IVOASTANDARD can be found at:

Reference Interoperable Implementations

The project contains 4 static files, shown as working examples located in PROJECT_HOME/implementations.
They cover most of the model features (see coverage table). In addition to this, we have 2 services (Vizier, XMM) providing annotated data compliant to MANGO DM.

The term "reference implementation" has never been clearly defined in the context of a model. In the case of MANGO, whose purpose is to restore complex quantities from tabular data, we provide static or dynamic VOtables, whose data are mapped onto MANGO using the Mivot schema. Additionally, we consider the Pyvo Mivot API to be another type of implementation.

Static Files

4 references files implementing most of the model features are provided within the implementations folder in the Git repository. The detail of the covered properties is givel by this table.

  • xtapdb:TAP query response on the 4XMMdr14 slim catalog. The VOTable has been annotated on the flight by the server.
    A link binding the postion with a quality flag has been added by hand.

    • file: xtapdb.xml
    • service: https://xcatdb.unistra.fr/xtapdb
    • query: SELECT TOP1 * FROM "public".mergedentry
    • format: application/x-votable+xml;content=mivot

  • Gaia:Vizier query response on the GAIA DR3 catalog. The VOTable has been annotated on by hand with the Pyvo MIVOT API.

    • file: gaia_with_mivot.xml
    • service: https://vizier.cds.unistra.fr/viz-bin/votable
    • query: see the mango:origin.QueryOrigin instance

  • datalink:Query on an Obscore table made to exercise the mapping on both Footprint and Datalink classes. The VOTable has been annotated on by hand on the project purpose.

    • file: datalink.xml
    • service: https://ws.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/argus
    • query: SELECT TOP 1 ivoa.ObsCore.obs_id,ivoa.ObsCore.s_region,ivoa.ObsCore.access_url FROM ivoa.ObsCore

  • vizier_cs_I_239:Cone search response on the I/239 Vizier table. The VOTable has been annotated on the flight by the server.

    • file: vizier_cs_I_239.xml
    • service: https://cds/viz-bin/conesearch/V1.5/I/239/hip_main
    • query: RA=0&DEC=0&SR=0.5
The detail of the covered properties is given in the README file.

Services

There are two services in operation and serving data mapped on MANGO.

XTapDB (HE Data and Photometry)

A TAP service serving XMM catalogue (https://xcatdb.unistra.fr/xtapdb/) data (table mergedentry) has been deployed.

This service demonstrates the ability of MANGO to properly describe the following quantities

  • Coordinate systems (space, time imported from Coords DM)
  • Position at a given epoch
  • Photometric filters and calibration (imported from Phot DM)
  • Fluxes and hardness ratio
  • Quality flags
Only quiered quantities are annoated.

The service can be connected with Taphandle or by using curl (or any other client enable to set a specific format)):

  • url: https://xcatdb.unistra.fr/xtapdb/sync?LANG=ADQL&REQUEST=doQuery&QUERY=[QUERY]&FORMAT=application%2Fx-votable%2Bxml%3Bcontent%3Dmivot
  • query: SELECT TOP1 * FROM "public".mergedentry
  • format: application/x-votable+xml;content=mivot
  • example: copy this URL in your clipboard.

Vizier Endpoints

There are 2 Vizier services returning VOtable partially mapped on the EpochPosition MANGO property.

These services have been officially released in May 2025.

Simbad Endpoints

The Simbad cone search can return data mapped to MANGO on demand. To access this data, append RESPONSEFORMAT=mivot to the query URL.

Each table row is mapped to one MangoObject instance, the available properties of which depend on the verbosity level (VERB parameter).

  • VERB=1:
    • The MANGO object is identified by the main Simbad ID
    • It contains 2 properties:
      • an EpochPosition instance mapping the position only
      • A Label mapping the object main type
  • VERB=2 (and 3):
    • The MANGO object is identified by the main Simbad ID
    • It contains 4 properties:
      • an EpochPosition instance mapping the position, the proper motion, the parallax and the velocity all with their errors.
      • Label instances mapping the object type, the spectral type and the mophology type
The Pyvo code able to process this output will be included in the next release.

The extension has been released on October 2025.

Python API

An important new module is being added to Pyvo. This module's purpose is to write and read Mivot annotations, with a particular focus on MANGO (see the doc).

  • Reader (since PyVO 1.6):
    • Mivot annotations are rendered as dynamic Python objects that obey the model structure. This allows data to be explored through the model view.
    • If the data id mapped onto the EpochPosition property, this API can automatically extract SkyCoord instance.
  • Generic writer (since PyVO 1.6):
    • Allows to build Mivot annotation element per element (attribute, instance, collection...). This API has been made for developpers to build mode user-friendly functionalities.
  • Mango writer (since PyVO 1.7)
    • Write coordinate frames (space, time ang photmétric calibration) that are part of Mago instances
    • Map data on EpochPosition, Brightness and Color properties. This is faciltated by a meta-data analizer included in the code.

Implementations Validator

VOTable whose data are mapped onto MANGO can be validated with mivot-validator:

The validator operates at 3 levels

  • Validation against the VOTable schema
  • Validation against the Mivot schema
  • Validation of the annoatation agans the models they refer to

Notre on the Repository

The process of building the document is very specific to the project. he workflow has been set up to make sure the diagrams keep in sync with the specified model.

  • The project is hosted by GitHub as a regular ivoa-std project
  • The working draft PDF is regularly udpdated
  • The descriptions of the model elements are stored in individual files located in PROJECT_HOME/vo-dml/desc.
    This allows for an easy update. It also delegates the latex escape character handling to the builder tool.
  • The table of content of the normative sections is given by PROJECT_HOME/utils/toc.json
  • The document is built by the PROJECT_HOME/utils/buildDoc.bash script:
    • Insert the description of all model elements in mango.vo-dml.xml and store the result in desc.mango.vo-dml.xml which is the model reference file.
    • Escape some characters to please to latex
    • Build the PROJECT_HOME/doc/model.tex file which contains the Latex of the model description (section 4).
      This step is achieved by the XSLT developed by M. Cresitello Dittmar.
    • Update the publication date
    • Build the pdf with the ivoatex tools
  • The PROJECT_HOME/mivot folder contains MIVOT snippets for the main classes.
    This allows contributors to figure out what the MIVOT annotation for MANGO could look like.
    These snippets have been generated with the mivot-validator package (available on pip)
Contributor Watchout: If you want to open a PR, please do not edit _PROJECT_HOME/doc/model.tex_ but the snippet files in PROJECT_HOME/vo-dml/desc and then run PROJECT_HOME/utils/buildDoc.bash.

A brief note details the process.

Comments from the IVOA Community during RFC/TCG review period: 2025-06-30 - 2025-08-2

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Comments from TCG member during the RFC/TCG Review Period: 2025-06-30 - 2025-08-02

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair

Applications Working Group

In a few places, the document states that MANGO is used to reconstruct tabular data from columns. I think this is wrong. MANGO is used to represent multi attribute properties and the correlations that exists between them but the actual linking of columns in tabular data is done by MIVOT. I’ve highlighted a few instances, but if agreed, this needs to be corrected through the document.

2,1 “add an upper description level” -> “add a higher level description layer”

2.1.1 "GAIA mission is producing the largest and more precise” -> "GAIA mission is producing the largest and most precise”

2.1.7 "This document does not recommend one approach over another. This is a matter for the data providers to decide.” - not sure what this refers to or if it’s needed especially in this section

2.2 “ - MANGO must be able to provide an accurate description of properties which attributes are spread out on multiple columns (e.g. positions, errors).” I would say that MANGO needs to provide accurate description of multi-attribute properties. Whether these are stored in separate columns and need to be linked back is an implementation detail that it’s being taken care by MIVOT.

“ - MANGO must be able to setup links between different properties”. Is that different from “-MANGO must be able to set up correlation links between properties” above?

2.3.1 "MANGO is primarily used to organise tabular data provided by TAP services”. Again, I think this is wrong. MANGO is used to represent catalogue data. MIVOT provides the linking of tabular data with a model, in this case MANGO.

3.1.3 - I’m not sure I understand, especially the second option: "Scattered Independent Quantities: Each table row is considered as collection of independent quantities.”

4.3: "constraint

detail: Property.One association at the time” - formatting error. Not sure what the meaning is either.

4.3.1 "This means that for concepts from the core vocabulary, the value in the semantics fieldz always starts with a hash. (datalink1.1).”. fieldz->fields. Also, hash missing in datalink example?

5 "The emergence of missions such as Gaia (Gaia Collaboration and Brown et al., 2018), which are capable of observing objects over long periods of time with very high astrometric accuracy, combined with great spectroscopic capabilities, making it possible to link star 3D positions with 3D velocities, thus defining a 6-dimensional astrometric solution, tagged with the epoch when the parameters were set.” Doesn’t make sense. I suggest breaking it in shorter phrases.

5.1.4-5.1.6 - For clarity, I would start with the description first and then mention that they are coords clones.

5.1.8: Why “_applied” instead of “Applied”?

6.4: Formatting problems

8.5: Missing description

11.2 This is not clear to me how it could work so I would remove it.

All points have been addressed by the PR https://github.com/ivoa-std/MANGO/pull/101

  • There is one commit per comment, each one referring the concerned section
  • A few other changes have been committed.

The requested changes that concerned the model itself have been merged with the DataTime representation suggested by DCP. This is due to the fact the the modeling tool (Modelio) does not support versionning.

  • The updated PDF can be accessed here
  • The updated VO-DML can be accessed here.

Data Access Layer Working Group

Data Model Working Group

Distributed Services & Protocols Working Group

Registry Working Group

Semantics Working Group

Data Curation & Preservation Interest Group

Ok for DCP with come minor suggestions/remarks: (gilles)

Q1:

  • I like the EpochPosition serialization in a flat output – Mango allows it, except for obsDate!
    see the alternative proposal which exploits dmtype ( github issue 85)
ANSWER
  • PR #100 - See complete answer in github issue 85
  • =EpochPosition:obsDate= is now a simple attribute typed as mango:jd/mjd/iso/year which are new primitive types derived from ivoa:datetime (see the PDF preview)

Q2:

ANSWER:

  • I have no objection to removing the “mango:DataLink” class, provided that no one requests that it be retained (see more in github issue 99)

Education Interest Group

High Energy Interest Group

Knowledge Discovery Interest Group

Operations Interest Group

Radio Astronomy Interest Group

Solar System Interest Group

Time Domain Interest Group

Standards and Processes Committee

TCG Vote : Vote_start_date - Vote_end_date

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

Group Yes No Abstain Comments
TCG        
Apps        
DAL        
DM        
DSP        
Registry        
Semantics        
DCP        
Edu        
KDIG        
Ops        
Radio        
SSIG        
TD        
<nop>StdProc        


<!--
* Set ALLOWTOPICRENAME = TWikiAdminGroup
-->

Edit | Attach | Watch | Print version | History: r15 < r14 < r13 < r12 < r11 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r15 - 2025-12-11 - BrigittaSipocz
 
This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright © 2008-2025 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback