IVOA Roadmap for 2014A

This outlines the roadmap for development activities by the various IVOA working and interest groups in 2014 between the Madrid and Banff Interops.

---

---

Applications

• MOC 1.0: Following discussions between authors a PR has been submitted for publication in January 2014, and the RFC is running from 7 February to 7 March. If all goes smoothly, we hope to achieve REC status near the May 2014 interop.
• Code Repositories: Following a request from the Exec, the Apps WG will start a discussion on use within the IVOA of code repositories beyond those at the project level. The Exec made this request in the context of the discussion in the Time Domain focus session at the Heidelberg (May 2013) interop in which Mario Juric (LSST) expressed his disappointment at not finding a one-stop shop for IVOA-blessed implementations of VO standards, specifically in Python. We will consider the current status and future possibilities for the IVOA and its member projects in interacting with existing repositories (notably Astropy) or setting up new ones, taking into account the desirability, costs and benefits of the various options.

Data Access Layer

• pre-Hawaii Interop status:
  • DALI-1.0 TCG review almost complete, new PR doc with all changes to date has been submitted
  • WD-DataLink-1.0 is late (to be submitted asap, it is missing 2 sections), final issues to be discussed at Interop
  • WD-STC-S.1.0 was posted in Sept, session at Interop
  • no work on TSA-1.0
  • cube access prototypes to be discussed at Interop

• DataLink-1.0
  • WD in October 2013
  • WG review January 204
  • PR document: February 2014
  • RFC in March 2014
  • TCG review in April 204
  • Recommendation: May 2013
• STC-S-1.0
  • this has been deemed not needed for SIA-2.0 and AccessData-1.0; these will adopt simple geometry value(s) without metadata
• SIA-2.0
  • agreed that ObsCore will define the bulk of the metadata returned by SIA query, additional metadata when ImageDM is available
  • WD in November 2013, will include query capability, maybe metadata (get-gory-details) capability
  • implementations in next few months
  • WG review in March 2014
  • RFC?
• AccessData-1.0
  • WD after WD-SIA stabilises
  • implementations need to be done
  • more discussions in May 2014
• TSA-1.0 -- small standard that re-uses SSA-1.1 with the time series DM produced by DM-WG (presumably SimpleTimeSeriesDM)
  • WD: TBD
  • WG review:
  • RFC period:
  • TCG review:
  • Recommendation: April 2014

In order to deliver useful functionality for data cube access in a short timescale, with incremental improvements over the next year, DAL will develop a series of much smaller components that will be the capablities delivered by web services (RESTful web services followng the DALI recommendations). These capabilities are, in order of priority (with relevant standard listed where they are known already):

• discovery of data cubes:
  • SIA query interface (SIA query capability)
  • TAP + ObsCore
• download of complete data cubes:
  • direct link in query response for single-file case
  • DataLink
• get additional metadata (SIA metadata capability)
  • return instance of ImageDM, format TBD
  • needed to perform some of the more advanced data access operations
• data access (AccessData capability)
  • extract subsection of cube (doable without detailed metadata)
  • flattening along one or more axes (sum or avg) (req: metadata)
  • rebinning along one or more axes (req: metadata)
  • precision pixel-level access (req: metadata)

The DAL roadmap is simplified to include the capabilities: DataLink, SIA query, SIA metadata (get-gory-details), and accessData (from DAL) as well as dependencies on the DM-WG for ObsCore and imageDM. We expect several versions of SIA-2.x and AccessData-1.x to be produced before all use cases are satisfied.

Data Model

PhotDMv1-1 1.0

TCG was recommendation on October 2013

PhotDMv1-1 will be adapted at certain point (October 2014 interop?) following VO-DML approach

Spectral DM 2.0

"Trying to be final" version released to the working group in March 2013. No major comments received. To be presented to WG during Spring 2013 interop. Implementation pending to formally start RFC process, RFC process started May 2014

Characterization DM 2.0

Stable version released to the working group in September 2012. No major comments received. Implementation to be presented during Spring 2013 interop.

Changes done to be presented during Spring 2014 (technical support)

Image DM

Presented during Spring 2013 interop. Connections with DAL. RFC to be started close to 2014 October interop

VO-DML

Draft presented during Spring 2013 interop. Prototypes and comments received between Spring nd Autumn 2013 interops

Stable/final version distributed to WG before Spring 2014 interop

Grid and Web Sevices

* PDL:

• May 2012: first WD: PDL-0.1-WD-20120516
• May 2013: revised WD in 1.0 and status discussion in Heidelberg GWS session 1
• August 2013 : RFC
• March 2014 : end of the TCG Review
• Spring 2014: REC?

* VOSpace:

• Comments and suggestions from the community to prepare the VOSpace WD 2.1 for Madrid
• REC ? : probably in Fall 2014 or Spring 2015

* UWS:

• Comments and suggestions from the community to prepare the WD UWS 1.1 for Madrid
• REC ? : probably in Fall 2014 or Spring 2015

Registry

RegTAP

We will integrate the feedback from RFC into the RegTAP PR and circulate it for TCG review. Although the list of changes after RFC was fairly substantial, most of the changes were actually clarifications for fringe cases. We hope to have RegTAP ready for REC by Banff.

There are now about four implementations of RegTAP. We plan to have them all pass the validation suite by Banff.

Registry Interface and RegTAP

With RegTAP becoming REC, the Registry Interfaces document will need revision. There is a working draft on volute that retains the standard OAI protocol for harvesting from publishing registries, but it will remove the search capabilities and instead refer to standards like RegTAP and a planned simple REST registry interface.

VOEventRegExt

While the actual authoring a registry extension for VOEvents, VOEventRegExt, is in the hands of GWS, RWG will continue to work closely with the authors.

An underlying Registry relational data model, RegTAP, has been developed which describes the core registry metadata to be exchanged using the RI 2.0 standard interface. This document is currently beginning the RFC phase.

Registry Metadata for Footprints

The RWG will continue to have discussions and evolve the current standards infrastructure in support of metadata for footprint descriptions, i.e. spatial regions of coverage. The advancement in the MOC standards definition will be evaluated in the context of footprint representation for support of applications in development of data discovery. Future coordination between IVOA groups will involve potential development for spatial indexing in support of application integration with registry collections.

Identifiers

We plan to issue an essentially minor update of the Identifiers recommendation; it will remove the obsolte XML form of identifiers and add text on the recommended forms for standard and dataset identifiers.

Registry Operations and Curation

The Registry WG continues to drive regular discussions during Interoperability meetings about IVOA Registries operations, including curation of VO Resources and validation of DAL services. The Registry WG is pending effort to develop a coordinated plan accepted by TCG and proposed to the Exec for developing the procedures and processes of curating nonresponsive or non-valid registered DAL services. This also includes consistency in Registry managed authorities and harvested resource metadata across the IVOA registry network.

The RWG also keeps monitoring the health of the registry system as a whole, working with the operators of the publishing registries to resolve problems as they crop up.

-- MarkusDemleitner - 2014-06-26

Semantics

The WG will continue to contribute to other WGs and IGs as appropriate.

* Extension of the UCD List for planetary data:
There are requirements from the Planetary community to use and extend the IVOA UCD list for using these semantic tags while mapping their metadata.
Some concepts , current in Planetary science are missing .
An effort to propose new possible UCD strings and validate them in the IVOA UCD maintenance group is taking place .
Proposed list by May Interop 2014

Data Curation & Preservation

The IG will continue discussions on general topics of interest to the community. Some of the topics we expect to focus on during the next 12 months include:

• The Unified Astronomy Thesaurus
  • Review of current effort described online at http://astrothesaurus.org
  • Participation of IVOA members in merging / mapping similar vocabularies (e.g. the Paris Observatory's Etymological Dictionary: http://dictionary.obspm.fr/)
  • Drafting editors to support the development and review of the thesaurus
• The Research Data Alliance
  • Follow developments of the RDA: http://rd-alliance.org/
  • Participate in working groups on persistent identifiers and data citation

-- AlbertoAccomazzi - 2013-03-19

Knowledge Discovery in Databases

Theory

The main goal of the Theory I.G. is to propose an access protocol to discover and retrieve simulated data.

From last InterOp, several implementations have been tested. We also had a few teleconferences with some members of the Theory I.G.

Search for simulations

Some technical difficulties have been identified. The most important of them, is that, contrary to an observationnal data, a simulated data may require several thousands quantities to be described in a way allowing us to answer our usecases. Because RDBMS are not efficient with tables with more than thousands columns, the access protocol for theoretical data should not rely on specific technologies (as TAP/RDBMS). During the second Theory session in Heidelberg InterOp we will discuss this important topic. A possible solution, would be to define a few APIs with a main one, the SEARCH/CUTOUT function, to discover interesting simulations. The query language would be a subset of ADQL. On the other hand, the exchange format will have to be defined. We will discuss the oportunity to use the serialization of SimDM in VO-Tables following the proposal of the utype tiger team.

Extraction of data

Concerning the extraction of data, the situation is more simple. Services will provide a Datalink document to describe the possible operations on data. This will be presented and discussed in the common DAL / Theory session about datalink.

Goals of this InterOp for the Theory I.G. are : 1) conclude on a solution to discover simulations that is independent on the technology used to store metadata, 2) define the large picture about the exchange format document in the communication client / server during the search phase of simulations, 3) advance on the syntax of datalink document.

Status after mai InterOp

SimDAL has been presented by David Languignon at the may InterOp at ESAC and has been discussed during two sessions. A general agreement has been found on what has been presented :

http://wiki.ivoa.net/internal/IVOA/InterOpMay2014Theory/InterOpESAC14_Theory_SimDAL.pdf

Now, planning is :

1) discuss with Registry W.G. to identify possible overlaps / complementarities between SimDAL repository and registries

2) try to prepare a first version of a WD for next InterOp on some parts of the SimDAL standard (depends on the available time of David Languignon)

3) continue implementations in France, Spain, Italy on various services.

Education

Edu IG prompted the development of VESPA (VO Educational Service Publisher and Archive), a web procedure offering managers of educational telescopes the possibility of an easy publication of their data in the VO. At the moment VESPA is an advanced prototype which we plan to develop into an official first version before the next interop meeting. In order to do so we need to test the application with a real interested user. The user has already been identified, but the user requires VESPA to implement SSAP beside the already implemented SIAP.

A related task for Edu IG is to propose solutions and desirables for the curation of educational resources inside or along standard registries. Most of the relevant aspects of this task are the subject of a Note available in the Edu IG page Ideas.

Education is a field where the localization of documents in different languages is essential. There is no immediate solution to this problem but we plan to start developing ideas and strategies in order to tackle it. The already mentioned Note on Educational Resources already considers the existence of multilingual documents and their hierarchy.

Edu IG continues to collect desiderata and ideas from educators and the general public and to propose them to VO, possibly leading to an even wider diffusion of VO resources toward communities other than professional astronomers.

-- MassimoRamellaINAF - 2014-05-18

Time Domain

A note endorsing the use of SimpleTimeSeries for representing time series data within the context of the VO was issued in May 2014. When the new version of the Document Standards defines the appropriate procedure, this note will be put forward for approval by the TCG.

An advanced working draft of VOEventRegExt was issued in May 2014. At the May 2014 InterOp, the TDIG agreed to advance this to PR status (as soon as a few minor formatting issues are resolved). We have now received additional feedback from Registry WG members, and are considering a further revision of this document. A PR is expected to follow shortly.

An advanced draft of the VOEvent Transport Protocol was issued in May 2014. This draft will undergo a further round of discussion within the TDIG before advancing to PR. We expect it to be ready for PR mid-2014.

-- JohnSwinbank - 2014-06-16

Standard and Processes

A new version of the Document Standards is foreseen in 2014 to:

• implement a new kind of Notes which will be endorsed by the TCG following a procedure that will be defined in the document
• implement the possibility of publishing Errata, and the procedure which should be followed

-- FrancoiseGenova - 2014-05-18

Science Priorities

Science priority areas are: Multi-dimensional data, and Time Domain Astronomy. Focus sessions at the May 2013 interop meeting will engage projects and surveys that produce and use multi-dimensional data (including radio astronomy, IFU, high energy and simulation data) and Time Domain data (time series, light curves, transient event reports) in a discussion on the role of VO technologies in these areas. Multi-d and Time Series use cases are being collected as part of this effort.