(r8) SourceCatalogs < IVOA

IVOA Web>IvoaDataModel>SourceCatalogs (revision 8) (raw view)~~EditAttach~~
---+ Source and Catalog Contribution Page
---++ Rationale
<ul id="docs-internal-guid-e3845370-7fff-947e-99d7-7fe30eaf7fe0"> </ul>
This page gathers contributions and discussions of the Source and Catalogs focused session which took place in Paris [[https://wiki.ivoa.net/twiki/bin/view/IVOA/InterOpMay2019DM][interop 2019]]

The goal of this session was to ask people invovled in data curation and client development (and any other people) what are their requirements to make source related data interoperable. This requires both modeling effort and serialization mechanisms and is a complex work:
<ul id="docs-internal-guid-e3845370-7fff-947e-99d7-7fe30eaf7fe0"> <li dir="ltr"> <p dir="ltr">There is no clear common definition of what a source is</p> <ul> <li dir="ltr">Can be a detection by a telescope </li> <li dir="ltr">Can be one row in a catalogue of compiled data</li> <li dir="ltr">Can be anything related to a sky location emitting a signal detectable by astronomers</li> </ul></li> <li dir="ltr"> <p dir="ltr">Source data are multi-faceted</p> <ul> <li dir="ltr">Something that modelers do hate </li> </ul></li> <li dir="ltr"> <p dir="ltr">Is a model relevant or possible or even desirable?</p> <ul> <li dir="ltr">The scope of a global source model is very huge </li> <li dir="ltr">It covers a large part of our science </li> <li dir="ltr">It couldn't likely not be described by a model </li> </ul></li> </ul>
This work should be shared among multiple working groups. The models need both domain and modelling expertise.

   * The effects on VOTable makes it clearly of interest to Apps,
   * The data models themselves, with appropriate use cases, are the primary driver led by DM.
---++ Challenges (on behalf of TD)

   * Data model complexities, and lack of actual data models, has often been conflated with mapping complexities and lack of successful demonstrations. The mapping is seen as the problem, when it's only a part, sometimes a small part. 
      * Pierre Fernique's very fair question in Victoria 2018 was a great example of this. "But what about my proper motions?"
   * This a complex problem, and we haven't been able to bring a commensurate amount of focused resources to it.
   * In my opinion, we lack specific, documented, user stories, use cases, etc. Pierre's case would be a good example if written up, and it's a case that must be demonstrated as clearly working in order to move on.
Straw Man Steps Forward

So to address some of those issues, we suggest something like the following steps to get moving forward. It may be that we don't have the resources to do all that, in which case maybe we settle on per-topic work-arounds (like with proper motion), *or* find a way to get more involvement from the greater community.
   1 <p>Get the TCG/Exec to agree on, then coordinate (mandate?) some steps like these.</p>
   1 <p>Get a group of people to commit to work on this. a. This can worked in parallel with the other steps, but it is important to know who is out there with the time, interest, commitment and expertise to help. b. Keep sharing, presenting, and inviting participation throughout.</p>
   1 <p>Define what problem(s) we are solving with a short-ish list of concrete user stories. For now, only include very important stories, and make them very specific, including what data sources will be used and what we will do with the data.</p>
   1 <p>From those stories, decide what data we need to model.</p>
   1 <p>Model the necessary data and map these models on real data</p>
      1 Need first to define the mapping from the model(s) to Astropy objects.
      1 Astropy mapping is not sufficient. Explore different mapping syntaxes by implement the user stories. a. This can be done in parallel with the modelling effort, and can evolve with it.
      1 Knowing the pencil and paper mapping is a prerequisite for each experiment. Before trying the mapping syntax, we must know unambiguously how the model maps to an implemented data structure (hopefully including Astropy).

---++ Mango Project

The MANGO model proposes a flexible way to expose data related to astronomical source objects in an interoperable way.
   * The MANGO model attaches an identifier on an astronomical source and associates to it all data related : observed physical quantities called parameters in this context, and other information like spectra, time series, preview image, for instance,for that source.
   * Parameters usually appear in the columns of a source catalogue. Additionnal dataproducts are bound to the source to contribute to the science analysis and enhance data understanding.
   * Parameters are modeled by the IVOA MCT DM reusing both native and extended classes. Parameters' roles are given by UCDs and semantic tags.
   * Associated data can be simple URLs, VO service endpoints or VO data model instances. Their roles are also qualified by semantic tags.
MANGO stands for Metadata ANnotation for Generic Objects (in astronomy).

The project can be followed on [[https://github.com/ivoa-std/MANGO/][GitHub]]

Mango comes with a VOTable annotation proposal that can be followed on [[https://github.com/ivoa-std/ModelInstanceInVot/][GitHub]]
---++ Resources
   * Model [[https://github.com/lmichel/vodml-lite-mapping/tree/master/python_workflow][working draft]]
   * Mapping [[https://github.com/lmichel/vodml-lite-mapping/tree/master/doc][working draft]]
   * Python [[https://github.com/lmichel/vodml-lite-mapping/tree/master/python_workflow][demo]] based on XMM data
---+ Former contributions

There were last years some proposals
| 2006 | Catalog Data Model Work Package | P. Osuna et al. | [[https://wiki.ivoa.net/twiki/bin/view/IVOA/IVAODMCatalogsWP][Wiki page]] |
| 2011 | Providing Photometric Data MeasurementsDescription in VOTables | S. Derriere et al. | [[https://wiki.ivoa.net/internal/IVOA/PhotometryDataModel/NOTE-PPDMDesc-0.2-20110512.pdf][note]] |
| 2016 | SourceDM proposal  | M. Demleitner G.Lemson | [[https://volute.g-vo.org/svn/trunk/projects/dm/SourceDM][Volute page]]  |

Wiki pages focused on this issue
| 2016 | Source DM | J. Salgado et al. | [[https://wiki.ivoa.net/twiki/bin/view/IVOA/SourceDataModel][Wiki page]] |
| 2018 | VO Survey | F. Bonnarel L. Michel | [[https://wiki.ivoa.net/twiki/bin/view/IVOA/SourceDM][Wiki page]] |
---+ Interop Contributions

| Title | Authors | Link |
| Focused Session | Interop Paris 2019 | [[https://wiki.ivoa.net/twiki/bin/view/IVOA/InterOpMay2019DM][Session page]] |
| Presentation | Interop Groningen 2019 | [[https://wiki.ivoa.net/internal/IVOA/InterOpOct2019DM/A_Component_and_Association__URLFixed.pdf][pdf]] |
| Presentation | LockTown May 2020 | [[%ATTACHURL%/CABMSD_MAI_2020.pdf][pdf]] |

<!--
      * Set ALLOWTOPICRENAME = IVOA.TWikiAdminGroup
-->
   * [[%ATTACHURL%/Source_Model_Notes.pdf][Source_Model_Notes.pdf]]: Chandra Source Catalog Source Model notes
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
pdf	CABMSD_MAI_2020.pdf	r1	manage	1916.2 K	2020-05-11 - 12:35	LaurentMichel	Presentation given at the Virtual Interop
pdf	Source_Model_Notes.pdf	r1	manage	74.1 K	2019-05-16 - 07:42	FrancescaCivano	Chandra Source Catalog Source Model notes
Topic revision: r8 - 2020-09-09 - LaurentMichel
IVOA
Log in or Register
IVOA.net
Wiki Home
WebChanges
WebTopicList
WebStatistics
Twiki Meta & Help
IVOA
Know
Main
Sandbox
TWiki
TWiki intro
TWiki tutorial
User registration
Notify me
Working Groups
Interest Groups
Committees
Stds&Procs
www.ivoa.net
Documents
Events
Members
XML Schema