DAL Roadmap as of May 2007 (Beijing)

Roadmap Summary

Spectral Data

• SSAP V1.0
  • The SSAP V1.0 working draft (currently at V1.01) has been updated to reflect the discussions in Beijing, and in early June advanced to IVOA Proposed Recommendation (PR) status. The Spectrum data model has advanced to a PR in Beijing and is in the RFC process now. Multiple implementations of both SSAP and the Spectrum data model already exist.
    
  • To avoid further delay in completing SSAP V1.0, the agreement reached in Beijing was to omit the new VOSI/Grid operations (getCapabilities, getAvailability) for now, and include the older FORMAT=METADATA construct instead, as the mechanism provided to introspect the service interface. A description of this has already been added to the SSAP specification. We apologize to SSAP early implementors who have already implemented a getCapabilities operation and now have to implement FORMAT=METADATA as well - but leave getCapabilities in, as it does not interfere with V1.0 compatibility, and we will be moving to this for V1.1 shortly in any case (see below).

• SSAP V1.1
  • SSAP V1.1 is expected to be essentially the same as V1.0, with the addition of the getCapabilities and getAvailability operations, required for registry and GWS integration. Other minor changes are also possible, but are not currently planned.
  • It was agreed in Beijing to limit getCapabilities to returning service metadata, that is, the service-defined "Capabilities" element of a VOResource (general resource metadata will be managed elsewhere). The GWS VOSI (VO standard interfaces) specification has already been modified to reflect this agreement. It remains to specify the content of getCapabilities for SSAP and to update the SSAP specification. A getAvailability operation (used to monitor service status and availability) will also be added. These will be compliant with the broader VOSI specification, but the SSAP versions will be documented within the SSAP specification to make the SSAP specification self-contained and remove any ambiguity for service implementors.
  • Work on SSAP V1.1 can proceed immediately after SSAP 1.0 moves to PR, and SSAP V1.1 should replace V1.0 within several months after V1.0 is finalized. V1.1 is likely to be upwards compatible with V1.0, although other minor changes are always possible. Once getCapabilities becomes a part of the SSAP standard the older FORMAT=METADATA mechanism will eventually be deprecated and removed from future versions of the SSAP interface, however this will be done gradually to allow client applications time to be modified.
  • Our expectation is that SSAP V1.1 will follow V1.0 within several months (by fall 2007), assuming that there are no unexpected delays due to the need to coordinate with other working groups.

• SLAP V1.0
  • Early SLAP testing has revealed a few minor problems which still need to be addressed before SLAP V1.0 can be produced and advance to PR. Some line data model problems need to be resolved. The metadata extension mechanism is being investigated as a way to pass additional quantum number information which does not fit well into the main table. Support is needed both for original data units and for homogenized units. Additional information needs to be passed back via the error mechanism to allow clients more fine control over error handling.
  • A V0.9 release of SLAP is planned to follow shortly after the Beijing meeting. V1.0 versions of the SLAP interface and line data model are planned for August, and the updated target for the PR is moved to the fall 2007 Cambridge interop.

• Multi-Segment Spectra, SEDs, and Spectral Associations
  • Support for these (and also time series data, which is closely related) has long been planned to follow after SSAP, however the approach and details require further discussion before anything more definite can be scheduled.

Catalog Data

• Cone Search V1.0
  • The PR RFC for the simple cone search (SCS) legacy interface has concluded. An updated version of the document will be passed to the IVOA Exec shortly to consider for advancement as a Recommendation. Again, the decision made earlier was to promote the frozen V1.0 cone search interface to a standard since it is already widely implemented and used, and to make it possible to use the standard IVOA mechanisms and processes to manage SCS and eventually replace it with a new standard which has gone through the full IVOA process.

• Cone Search V1.1
  • Interest (led by Ray Plante) has been expressed in a V1.1 version of cone search, to allow for minor changes which were not possible for the frozen V1.0 version of the interface, but without affecting the limited scope of the interface (TAP will already provide an opportunity to provide for more sophisticated table access functionality). Improvements which have been suggested including updating the UCDs to the most recent UCD specification, and possibly adding additional query parameters such as COLLECTION or TABLE, to allow for services that provide access to multiple collections or data tables.

• TAP (Table Access Protocol)
  • TAP is a joint effort of the VOQL WG and DAL, and is being discussed primarily within a broad-based VOQL-TEG "tiger team" which has been active since late 2006. In addition to TAP, the VOQL WG also separately considers ADQL and a higher level cross-match portal, which are related to TAP and help define the scope of TAP.
  • From the point of view of data access service providers and client applications, it is desirable to have a common approach and as much shared interface and software as possible for TAP and the other data access interfaces, consistent with providing an effective table access interface.
  • TAP was discussed in two full sessions in Beijing, but significant interface issues remain to be resolved before a draft interface specification can be produced. The updated target for a first (possibly incomplete) draft TAP interface specification is for the 2007 Cambridge interop.
  • It was agreed that TAP should support only ADQL-based queries. "Cone search" like queries will not be supported directly by TAP, but similar functionality could be provided using ADQL with a UTYPE-based data model and a REGION clause. It would also be possible for a TAP service implementation to support a cone search interface in addition to a TAP interface.
  • Key issues remaining to be resolved include the basic form of the TAP interface and how compatible this is with the other IVOA data service interfaces, and the form and scope of the interface used for table metadata queries. Support for large (non-distributed) queries, and simultaneous queries of multiple regions, are requirements for TAP. Support for asynchronous queries is required but is lower priority than support for simple synchronous queries.

Image Data

• SIA V1.0
  • As with cone search, there is a need to promote the legacy SIA V1.0 interface to the status of an IVOA standard, to allow use of the IVOA standards process to manage this existing widely used interface, and to improve the SIA specification to clarify details which are not currently well enough defined. This has been delayed to avoid interfering with completion of the SSAP standard, but should go forward immediately after SSAP and before beginning work on the SIA V2.0 specification.

• SIA V2.0
  • With the completion of SSAP it is time to go forward with SIA V2.0, which will build upon the work done for SSAP as well as provide other functional enhancements. The SSAP-related enhancements include an enhanced query interface, and much more comprehensive generic dataset metadata, including standard metadata for characterization, dataset identification, curation, and so forth.
  • SIA V2.0 was discussed in a special session in Beijing as well as in earlier working group sessions, including during 2007 when issues such as cube data access and representation of complex data associations were first discussed. The priorities identified in Beijing for SIA V2.0 were the following:
    • Retaining simple usage for simple cases is essential
    • Highest priority is for upgraded interface, grid capabilities
    • Data cube support is required
    • Multiple region support is required
    • Much interest in issues involving complex data associations
  • In addition the following were suggested:
    • Want to be able to return image + auxiliary data products
    • Support for multi-resolution images should be considered
  • The topic of "grid capabilities" includes asynchronous operations, data staging, authentication, getCapability and getAvailability, and support for queries of multiple regions, required to scale SIA up to be able to support large aggregate operations.
  • Both the proposed DAL "stageData" operation and the more general Universal Worker Service (UWS) mechanism were discussed in Beijing. An agreement between DAL and GWS was reached to work together on a unified specification for these, including prototypes, with the intention of being able to report on this in time for the fall 2007 interop in Cambridge.
  • We plan on having a first, probably only partially completed, version of the SIA V2 interface specification to discuss in Cambridge as well (Sep 2007). Full development of some features such as data cube and complex data support, with a working draft specification addressing all the key areas, will likely not be possible until May 2008.

Theory Data

• SNAP
  • A simple numerical access protocol (SNAP) is being investigated by the Theory interest group. Exactly what is required is still under discussion within the Theory IG, and this needs to be clarifed before more definite plans for a possible data access interface can be made.

Solar and Planetary Data

• Image Access
• Table Access
• Spectrophotometric Data (time series, spectra)
  • All of the second generation DAL interfaces (SSA, SIA V2, etc.) include support for generalized queries where all individual query parameters are optional, details such as the coordinate systems supported by a service are configurable, query by target name or other parameters is provided, and so forth. For example, queries are no longer required to be positional and are not limited to the celestial coordinate systems. Much of the dataset metadata returned is generic and not specific to any one type of astronomical data. Optional parameters can metadata extension can be used to extend the standard interfaces for a new type of data.
  • These generalizations should make the standard data interfaces potentially useful for solar and planetary data, however until we have actually tried this with real data and real use-cases we don't know what the issues will be.

- DougTody, KeithNoddle, MarkusDolensky, June 4 2007