Publishing Data into the VO

0. Introduction

The purpose of this page is to provide practical information about how to publish you data holdings into the VO. The content of this page is user contributed by members of the IVOA community and its content will evolve as IVOA standards and implementations evolve.

Depending of your needs and your technical experience, there are several ways to publish you data into the VO:

• if you have very little technical expertise, you might want to contact your national VO projects who might already have the infrastructure to host your data and make it available to the VO. See the list of national VO projects and the selection of open data centers offering publication as a service.

• if you would like to find your VO services in client applications, please publish your service in a VO Registry.

• if you have some technical expertise but don't want to spend time to development, you might want to use directly some existing VO Publishing toolkits.

• if you prefer VO interfaces to your data yourself, you might find some useful VO software tools and libraries. A good start would be to determine what type of data you want to publish (images, catalogues, spectra, ...). Reading through the IVOA Architecture document might also be useful before going into the more details of the specific IVOA standards that you might need to use.

• please use the Validation Tools to ensure your services actually behave according to the standards, and
• remove your service from the registry if you stop running it

1. Questions & Answers about what type of data to be published into the VO

The following section should help you figuring out what tools you need in order to publish your data in the Virtual Observatory. Since there are several tools that can be useful in different situations, we arranged this section to be requirements driven.

For completeness, you can also find links to the technical specifications of the different protocols involved. However, many of the tools presented in this page aim at abstracting the data publisher from the technicalities of the protocols.

Q: How do I publish images?

A: Images are handled in the VO using a protocol called Simple Image Access Protocol (SIAP). To publish such data, look for packages speaking SIAP in the list of toolkits below.

A more involved alternative -- suitable for complex data sets like data cubes, but right now with much less support in user tools -- is a publication using ObsTAP, a fairly new standard that lets clients query metadata tables using SQL. To run an ObsTAP service, you need a server speaking the Table Access Protocol TAP, and you need to understand the VO's Observation Data Model.

Technical documentation: SIAP Specification, Observation Data Model Core Components

Q: How do I publish spectra?

A: Spectra are handled in the VO using a protocol called Simple Spectral Access Protocol (SSAP). To publish such data, look for packages supporting SSAP in the list of toolkits below.

If you have data cubes rather than individual spectra, SSAP may not quite fit your bill. See the remarks on ObsTAP in the section on publishing images above.

Technical documentation: SSAP Specification

Q: How do I publish catalogues or generic data tables?

A: Simple tables containing data for celestial objects can be published to the VO using a protocol called Simple Cone Search (SCS). See the list of toolkits below for ready-made servers already supporting SCS. Also, it is fairly straightforward to write a simple SCS server, in particular when VOTable generation is already taken care of. See Developer's corner for leads.

A more complex interface capable of performing arbitrary queries using the Astronomical Data Query Language (ADQL) is called TAP.

Being flexible and generic, TAP is much more powerful but also more complex, while SCS has a more narrow scope and is thus much more simple.

If you do not want to install any code and just want to publish tables in the VO, contact the VizieR team as explained in the relevant question, at at cats (at) cdsarc.u-strasbg.fr or at the general CDS email cds-question (at) unistra.fr.

Technical documentation:

• Simple Cone Search Specification
• TAP Specification

A: Use VOEvent. Look at the notes on VOEvent for details.

Technical documentation:

• VOEvent Specification

A: Quite a few of the publication toolkits discussed below support multiple protocols in one package. Since the protocols have been designed so they can run alongside each other, in all likelihood you will only have to run a single server.

Q: I have data files (e.g. FITS files representing catalogues, spectra and/or images) and I want to publish them using VO protocols. What tools can I use to publish this data to the VO?

A: Most of the tools build their services around metadata tables kept in relational databases. Getting this metadata (e.g., date of observation, wavelength range, area observed, etc) into such a database is called ingestion, and several publication toolkits (e.g., Saada or DaCHS) have "ingestion components", i.e., programs that understand, say, FITS and let you map your headers to VO-compliant descriptions.

Q: I have a working database and I want to publish the data therein in the VO. What tools can I use?

A: Several publication toolkits are designed so as to work without modifications on existing tables or at least requiring no schema changes. In particular, look for VO-Dance, DALToolkit, DSA, and DaCHS..

Q: I want to develop my own code to create VO services, but I would like to leverage existing libraries. Where can I find them?

A: See Section 3 below.

Q: I don't want to write any code. Are there applications I can install and configure in order to serve my data?

A: Saada comes fairly close to a point-and-click publication infrastructure. If you have a Debian/Ubuntu machine, DaCHS lets you quickly install the infrastructure, although metadata definition requires a bit documentation-reading.

Q: I don't want to write any code and I don't want to install anything. I just want to publish my data!

A: Some services are designed to reduce the effort of publishing data to the bare minimum:

• VO-Dance is able to connect to remote DBs and archives (once given proper authorizations) and publish VO services from those resources (have a look at VO-Dance web page at IA2 for more information).

• VizieR is a reference service providing access to catalogues and tables, with a simple standardized description of the table(s). More complex data items such as images, spectra, time series, can be added. You can contact the CDS team at cats (at) cdsarc.u-strasbg.fr or at the general CDS email cds-question (at) unistra.fr.
• The GAVO Data Center publishes data of all kinds of data (images, spectra, tables, cubes) for third parties; it also cares about the registration of your data. For more information, contact gavo (at) ari.uni-heidelberg.de
• Also, the IA2 data center can host and publish data from third parties (through VO-Dance and a Distributed Archiving System application). If you are interested contact ia2 (at) oats.inaf.it
• The CAB-SVO Data Center also hosts and published data provided by the Spanish community. Should you are interested, contact svo-support@cab.inta-csic.es

2. Registry for VO Data Service Discovery

Q: How does my service appear in Client Applications?

A: Once you have a valid VO data compliant service (see section 1), you have to publish your service in a IVOA registry, a "yellow page" or directory of service web locations.

There are 2 methods for publishing your service into an IVOA registry.

• The most common form is to enter your service information in a Fully Searchable Registry. Two systems are available:
  • EURO VO Registry resource insert page
  • VAO Registry publishing interface
• A secondary method is available via creation of your own Publishing Registry. This option is typically used for archive data centers which have many services. DaCHS (see Toolkit listing) has a publishing registry built in.

3. Toolkits to publish data into the VO

(VO).

4. Developer's corner : other useful software tools and libraries for VO development

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->