From pjq@eso.org Thu Jan 29 07:27:08 2004 Date: Mon, 26 Jan 2004 16:45:05 +0100 From: Peter Quinn To: 'IVOA' Subject: FW: Meeting next week FYI PQ ------ Forwarded Message > From: Doug Tody > Date: Sat, 24 Jan 2004 17:31:00 -0700 (MST) > To: Peter Quinn > Subject: Re: Meeting next week > > Hi Peter - > > On Wed, 21 Jan 2004, Peter Quinn wrote: >> Dear WG Chairs >> I remind you that I would like to get short written reports (1-2 pages) >> from each WG regarding the status of working group discussions, documents, >> progress since Strasbourg, plans for upcoming documents/standards and >> involvement in January demos. All those WG chairs who are actually in >> Garching for the AVO demos or the IVOA meeting are more than welcome to >> present their reports verbally at the meeting. > > The following is the IVOA DAL WG report. I'm sorry that I will not be > able to attend the meeting next week. > > - Doug > > > ----- > IVOA Data Access Layer Working Group > D.Tody, January 24, 2004 > > > Data Access Layer Working Group > > The DAL effort has been organized into the following areas: > > o Data access services - service interfaces and protocols > o Data representation - format of data returned by services > o Framework - component framework for VO data access > o Data provider / consumer implementations > > > Data Access Services > > Catalog access. W. O'Mullane and others at JHU, as part of the > IVOA VOQL WG (Masatoshi OHISHI, NAO), are developing an astronomical > data query language (ADQL), which is SQL-like with astronomy-specific > extensions for region-based queries and the like. A family of catalog > query and cross-matching services are currently in the prototyping > stage. The plan is to develop a new catalog access service for DAL > to replace cone search, based on the SkyNode prototype being developed > at JHU. This will be a basic catalog access service - the more advanced > query services, with query portals, cross-matching, etc., are part > of VOQL and the VO services. ADQL will later be integrated into all > the DAL services to provide an optional advanced query capability. > > Image access. Version 1.0 of the Simple Image Access interface (SIA) > has been in use for some time now. Improvements to the general VO > infrastructure are required before a new version of SIA is warranted. > Priorities for enhancement to SIA include the following: > > - Image provenance and identification > - Characterization of the physical attributes of an image > - Improved support for image formatting options, e.g., templating > - Registry integration > - Service verification > - Error handling > > R. Plante (NCSA) and others within the registry working group have > produced a draft standard for VO resource identifiers, which will > serve as the basis for image identification metadata in SIA and other > DAL interfaces. D. Tody (NRAO, DAL WG Chair) is working with the data > models working group on the problem of dataset characterization, a key > issue for DAL dataset queries and image characterization within SIA. > F. Bonnarel (CDS) has identified requirements for improved image > formatting options for SIA, and proposed an access reference URL > templating mechanism as a possible means for addressing this problem. > > Spectrum access. Development of the Simple Spectral Access interface > (SSA) was the most active area of development for DAL in the past > quarter. SSA will provide uniform access to both 1D spectra and SEDs. > M. Dolensky (ESO) has written an internal draft specification for > the SSA query interface. J. McDowell (SAO) has written an internal > draft specification for the SSA data model. T. Budavari (JHU) has > developed a spectral data service for JHU, including an experimental > SOAP-based interface for spectral access. D.Tody has been working > with the FITS WCS authors to ensure that the proposed FITS spectral > WCS standard will be consistent with SSA. > > Time series. Our thinking at this point is that time series can > be handled as a specialization the general SED data model. A SED > consists of a series of photometry points with both spectral and time > attributes. A 1D spectrum is a projection of the SED model along > the spectral axis at constant time. A time series is a projection > of the SED model along the time axis at a constant spectral value. > Further work is required to evaluate the characteristics of actual > time series data before we can be sure this approach will work, > but if so basing time series data access on SSA would allow time > series data to be handled at little additional cost. > > Event and visibility data. Little has been done yet about VO > access for event data. Within the radio community, efforts are > currently focused on data models and data formats for visibility data. > ALMA (led by A. Wicenek of ESO) is attempting to use a combination of > VOTable and FITS BINTABLE to represent the ALMA science data model. > This is one of the earliest attempts we are aware of to attempt to > use VOTable to handle large quantities of data. > > The priorities for near-term development of DAL services include: > > o Concluding the current work on spectral and time series data > access and releasing the first version of SSA. > > o Initial implementation of SSA services and end-to-end testing and > tuning of SSA. > > o Continuing work on the DAL infrastructure followed by an update > to SIA once this new infrastructure is available. > > o Specification of a new catalog access service to supplant cone search. > > o Work on multi-protocol (Web Service) versions of the DAL services. > > Issues and Concerns > > At some point we will need to update all the DAL services to uniformly use > new VO infrastructure, e.g., for data characterization, advanced queries > (ADQL), data model-based data representation in XML/VOTable, multi-protocol > access, and so forth. It is difficult to predict when this will be possible > without knowing when the necessary infrastructure will be available. > > > Data Representation > > The problem of how to represent astronomical data within the VO is a major > issue which is just now starting to be addressed. We have made a good > start with catalog data, which the current VOTable handles fairly well. > Further work is needed on catalog metadata but this is more of a data > modeling problem than a data representation problem. > > For astronomical datasets such as images, spectra, time series, etc., > little has yet been done. SIA avoids the problem by using VOTable to > convey standard metadata for images. The image data itself is returned > using FITS, with no attempt to standardize the contents of the FITS > file other than what is already provided by FITS. A hybrid scheme such > as this is probably what is needed in the long run as well, using XML > for complex structured metadata, and FITS for bulk binary data. > > For SSA the data representation issue is something we can no longer avoid. > There is no standard FITS representation for spectra, so to provide uniform > access to spectra via SSA we need to define a standard data format for > spectra. More precisely, we need to define a data model for 1D spectra > and SEDs, and a standard mapping of this model into several different > storage formats, e.g., VOTable (or some other XML-based format), FITS, > and simple text. J. McDowell (SAO) and D. Tody have defined a preliminary > data model for 1D spectra and SEDs, and are currently working on mappings > of this data model into XML (VOTable) and FITS. > > Issues and concerns > > o An issue is how to model complex astronomical datasets such as > a spectrum. Do we produce one model for the entire spectrum, > or model the components of astronomical datasets and merely > associate these components to model actual datasets? > > o How to represent a data model in XML or in VOTable is an issue. > For example, if we map the attributes of a data model to the fields > of a VOTable how do we associate this storage representation back > to the abstract data model? How do we do schema validation in such > a case? Two alternate mechanisms have thus far been proposed. > The UTYPE tag allows the fields of a table to be associated to > the attributes of an externally defined data model. The VOTable > "group" mechanism provides a mechanism for describing structure > and relationships among subsets of the fields of a flat table. > > o Whether to use generic containers like table to represent > astronomical data objects is an issue. To merely pass an object > across a wire, e.g., via a Web service, the simplest approach > is to serialize the object as an XML entity, with a custom > schema defined for the object. Mapping such an object into a > table complicates a simple "object to wire and back to object" > mapping, but allows generic table tools and libraries to be used > to manipulate the data component of a dataset. > > o How to handle bulk binary data is an issue. FITS can do this, but > FITS alone does not go far enough, and FITS is not a good solution > for handling complex stuctured information such as metadata. > XML is much better for this purpose. The challenge will be to > determine how to combine the two to provide both the flexibility > and standard-based strengths of XML, plus the efficiency of binary > format standards such as FITS. > > All of these issues are actively being researched by the DAL, data model, > VOTable, and other working groups within VO. The SSA interface will > provide the first real-world test case for working out how to represent > astronomical data natively within the VO. > > > Data Access Framework > > The initial DAL protocols are simple enough to be implemented "from the > ground up" by service providers and client applications. As data access in > the VO becomes more sophisticated this will no longer be practical; it will > become necessary to provide reference-grade software implementing the data > access protocols, both on the server and the client side. Integration and > reuse of legacy data analysis software with the VO is also a goal, for > example to enable application of server-side data analysis components > during data access. Further work on the DAL framework is not scheduled > until Q2 2004. > > > Data provider/consumer implementations and end-to-end testing > > Numerous implementations of the cone search and SIA services currently > exist, along with registry integration and client applications to make use > of these services. This software was most recently demonstrated by the > NVO at the AAS in January 2004. An early test version of the SSA query > interface has been implemented by ESO and ISO and will be demonstrated > at the AVO demo in January 2004. > > The current plans for SSA are to provide at least two initial test > implementations of the service (at JHU and ESO), with at least one client > implementation (probably the SpecVIEW Java client from STScI). ------ End of Forwarded Message