Publishing Data into the VO

---

---

0. Introduction

The purpose of this page is to provide practical information about how to publish your data holdings into the VO. The content of this page is user contributed by members of the IVOA community and its content will evolve as IVOA standards and implementations evolve.

Depending on your needs and your technical experience, there are several ways to publish you data into the VO:

• if you have very little technical expertise, you might want to contact your national VO projects who might already have the infrastructure to host your data and make it available to the VO. See the list of national VO projects and the selection of open data centers offering publication as a service.

• if you would like to find your VO services in client applications, please publish your service in a VO Registry.

• if you have some technical expertise but don't want to spend time to development, you might want to use directly some existing VO Publishing toolkits.

• if you prefer to add VO interfaces to your data yourself, you might find some useful VO software tools and libraries. A good start would be to determine what type of data you want to publish (images, catalogues, spectra, ...). Reading through the IVOA Architecture document might also be useful before going into the details of the specific IVOA standards that you might need to use.

When publishing, please keep in mind that once your service is in the registry, it is part of a large distributed information system -- the VO. When your service fails or misbehaves, the user experience for the whole VO is degraded. So:

• please use the tools listed in IvoaValidatorsSummary to ensure your services actually behave according to the standards, and
• remove your service from the registry if you stop running it.

If, having consulted the resources described above or the FAQ below, you are still unsure how to proceed, please send a message to datapub (at) ivoa.net and we will endeavour to answer as soon as possible.

---

1. Questions & Answers about what type of data to be published into the VO

The following section should help you figuring out what tools you need in order to publish your data in the Virtual Observatory. Since there are several tools that can be useful in different situations, we arranged this section to be requirements driven.

For completeness, you can also find links to the technical specifications of the different protocols involved. However, many of the tools presented in this page aim at abstracting the data publisher from the technicalities of the protocols.

Q: How do I publish images?

A: Images are handled in the VO using a protocol called Simple Image Access Protocol (SIAP). To publish such data, look for packages speaking SIAP in the list of toolkits below.

A more involved alternative -- suitable for complex data sets like data cubes, but right now with much less support in user tools -- is a publication using ObsTAP, a fairly new standard that lets clients query metadata tables using SQL. To run an ObsTAP service, you need a server speaking the Table Access Protocol TAP, and you need to understand the VO's Observation Data Model.

Technical documentation: SIAP Specification, Observation Data Model Core Components

Q: How do I publish spectra?

A: Spectra are handled in the VO using a protocol called Simple Spectral Access Protocol (SSAP). To publish such data, look for packages supporting SSAP in the list of toolkits below.

If you have data cubes rather than individual spectra, SSAP may not quite fit your bill. See the remarks on ObsTAP in the section on publishing images above.

Technical documentation: SSAP Specification

Q: How do I publish catalogues or generic data tables?

A: Simple tables containing data for celestial objects can be published to the VO using a protocol called Simple Cone Search (SCS). See the list of toolkits below for ready-made servers already supporting SCS. Also, it is fairly straightforward to write a simple SCS server, in particular when VOTable generation is already taken care of. See Developer's corner for leads.

A more complex interface capable of performing arbitrary queries using the Astronomical Data Query Language (ADQL) is called TAP.

Being flexible and generic, TAP is much more powerful but also more complex, while SCS has a more narrow scope and is thus much more simple.

If you do not want to install any code and just want to publish tables in the VO, contact the VizieR team as explained in the relevant question, at at cats (at) cdsarc.u-strasbg.fr or at the general CDS email cds-question (at) unistra.fr.

Technical documentation:

• Simple Cone Search Specification
• TAP Specification

Q: I want to publish alerts of celestial transient events. How can I do that?

A: Use VOEvent. Look at the notes on VOEvent for details.

Technical documentation:

• VOEvent Specification

Q: I want to publish a combination of generic data tables, spectra and images, possibly with links to each other. Are there toolkits/services for doing so?

A: Quite a few of the publication toolkits discussed below support multiple protocols in one package. Since the protocols have been designed so they can run alongside each other, in all likelihood you will only have to run a single server.

Q: I have data files (e.g. FITS files representing catalogues, spectra and/or images) and I want to publish them using VO protocols. What tools can I use to publish this data to the VO?

A: Most of the tools build their services around metadata tables kept in relational databases. Getting this metadata (e.g., date of observation, wavelength range, area observed, etc) into such a database is called ingestion, and several publication toolkits (e.g., Saada or DaCHS) have "ingestion components", i.e., programs that understand, say, FITS and let you map your headers to VO-compliant descriptions.

Q: I have a working database and I want to publish the data therein in the VO. What tools can I use?

A: Several publication toolkits are designed so as to work without modifications on existing tables or at least requiring no schema changes. In particular, look for VO-Dance, DALToolkit, DSA, and DaCHS..

Q: I want to develop my own code to create VO services, but I would like to leverage existing libraries. Where can I find them?

A: See Section 3 below.

Q: I don't want to write any code. Are there applications I can install and configure in order to serve my data?

A: Saada comes fairly close to a point-and-click publication infrastructure. If you have a Debian/Ubuntu machine, DaCHS lets you quickly install the infrastructure, although metadata definition requires a bit documentation-reading.

Q: I don't want to write any code and I don't want to install anything. I just want to publish my data!

A: Some services are designed to reduce the effort of publishing data to the bare minimum:

• VO-Dance is able to connect to remote DBs and archives (once given proper authorizations) and publish VO services from those resources (have a look at VO-Dance web page at IA2 for more information).

Also, some data centers ingest data from published papers or data provided by authors and create standard services that serve it:

• VizieR is a reference service providing access to catalogues and tables, with a simple standardized description of the table(s). More complex data items such as images, spectra, time series, can be added. You can contact the CDS team at cats (at) cdsarc.u-strasbg.fr or at the general CDS email cds-question (at) unistra.fr.
• The GAVO Data Center publishes data of all kinds of data (images, spectra, tables, cubes) for third parties; it also cares about the registration of your data. For more information, contact gavo (at) ari.uni-heidelberg.de
• Also, the IA2 data center can host and publish data from third parties (through VO-Dance and a Distributed Archiving System application). If you are interested contact ia2 (at) oats.inaf.it
• The CAB-SVO Data Center also hosts and published data provided by the Spanish community. Should you are interested, contact svo-support@cab.inta-csic.es

You can also refer to your national VO initiative for support.

---

2. Registry for VO Data Service Discovery

Q: How does my service appear in Client Applications?

A: Once you have a valid VO data compliant service (see section 1), you have to publish your service in a IVOA registry, the "yellow pages" of the VO, queried by user applications (and other things) when looking for services.

To make this work, you need to provide a standardised service description (that's called “resource record” in VO language) and then push that into the big pool of existing registry records. Depending on how many data collections you publish and how often they change, there are various ways to do that, and GettingIntoTheRegistry discusses them.

3. Toolkits to publish data into the VO

Toolkit Name	Contact Point	DAL Services Supported	Date of last Release	Short Description
SVOCat	Enrique Solano SVO	ConeSearch (CS)	v1.0 Apr 2014	SVOCat is a tool developed by the Spanish VO to publish tables and catalogues. Through a series of configuration files, a web interface as well as a ConeSearch access is created. No code to write and little technical expertise is required.
MySpec-MyImg	Enrique Solano csic.es	Simple Spectral Access Protocol (SSAP) and Simple Image Access Protocol (SIAP)	v1.0 Apr 2014	MySpec -MyImg is a tool developed by the Spanish VO to publish spectra and images. Through a series of configuration files, a web interface as well as a SIAP (images) or SSAP (spectra) access is created. No code to write and little technical expertise is required.
SAADA	Laurent Michel	SIAP/SSAP/CSP/TAP	1.8.build15b November 2015	Saada builds databases from data files: no code to write - heterogeneous dataset storage - can host multiple data collections - support persistent links between data - metadata tagging - access by Web interface, VO protocols or API.
DALServer	Doug Tody, USVAO	SIA/SSA/SCS/SLAP; SIAV2 prototype; TAP implemented but not yet integrated	1.0 dev Apr 2014	The DALServer framework is intended to provide support for all VO data services within a common, easily user configurable or extensible framework implemented in Java. A production version of DALServer is currently under development within VAO; an older version was released by NVO. The current development version of DALServer (May2014) includes a prototype SIAV2 (and SIAV1) implementation supporting the draft ImageDM and advanced image access via AccessData.
VO-Dance	VObs.it, IA2	SCS, SIAP [v1.0], SSAP; TAP (using a different application)	May 2016 (update)	VODance is a tool for rapid Virtual Observatory compliant services deployment. Data Access Layer services are created on the fly and published to VO without the need to move your data or DB metadata. A TAP dedicated application has been developed and supported by a TAP_SCHEMA GUI manager. Please note the this publishing infrastructure is under revision, thus it is suggested to contact Marco Molinaro for further and more fresh information.
DaCHS	MarkusDemleitner, GAVO	SCS, SIAP, SSAP, TAP, SLAP, OAI-PMH, SIAv2, Datalink	December 2022	The Data Center Helper Suite is intended as an all-round data center software, including data ingestion (from, e.g., FITS, text files, binary data), unified metadata handling (including an OAI-PMH registry interface), and multi-protocol publication (including TAP, SCS, SIAP, SSAP, and web interfaces). For recent developments, see DaCHS on the GAVO blog
OpenCADC	Patrick Dowler CADC			Open source software (Java, Python) repositories for implementing IVOA standards: SSO, CDP, UWS, SIA, TAP, TAPRegExt, VOSpace, VOSI. The repository also includes test code to test conformance of a service with the specification; these are generally designed to be used as unit or integration tests.
DALToolkit	ESA-VO	SIAP, SSAP	June 2009	The ESAVO DALToolkit publishing software is a Java based server allowing you to publish astronomical images and spectra into the VO through the SIAP and SSAP Data Access Layer (DAL) protocols. Specific tutorials can be found here. Unfortunately, the DALToolkit has not been maintained since then.
VAPE	VObs.it	SIAP [edu]	0.97 June 2014	VAPE is a web application for the publication of educational data in the Virtual Observatory. Use the link on first column to reach its main page if you want to register.
CASDA VO Tools Binary	JamesDempsey, CSIRO	TAP, SCS, SIAP [v2.0]	1.2 May 2016	A public use implementation of the TAP, cone search, datalink and SIAP virtual observatory protocols as used in the CSIRO ASKAP Science Data Archive. This Java application can be deployed against an existing database and configured interactively or via text files to allow access to catalogues and image metadata. See CASDA Data Access for a SODA implementation provided access to image files.
Daiquiri	Jochen Klar or E-Science AIP Potsdam	TAP, Cone Search	Oct 2019	Daiquiri - Python based framework for the publication of scientific databases - Daiquiri on Github. It is deployed on Gaia@AIP or APPLAUSE - Archives of Photographic PLates for Astronomical USE

---

4. Developer's corner : other useful software tools and libraries for VO development

Tool / Library Name	Contact Point	Date of last Release	Short Description
STIL	MarkTaylor	2022	Table I/O and processing library in Java, including full VOTable I/O. Fast and scalable.
STILTS	MarkTaylor	2022	Table command-line tools suite. Includes votlint (VOTable validator) and taplint (TAP service validator).
SAVOT	AndreSchaaff	5/2014	Simple Access To VOTable, parsing / editing / writing VOTable / N.B.
votable.js	AndreSchaaff	6/2015	VOTable parsing (including base64) in Javascript
CDS UWS Library	GregoryMantelet	2/2018	A UWS (Universal Worker Service) is defined by IVOA ( http://www.ivoa.net/Documents/UWS) as a web-service which lets execute one or several jobs in an asynchronous manner. To sum up, a UWS is a set of jobs lists. By sending requests, a user can create, start, stop or delete a job. It can also ask a summary of a job, get its result(s) or get the content of a jobs lists. This library is a convenient implementation of a UWS which already implements all behaviors and functionalities described by the IVOA recommendation v1.0, and has a complete set of documentation. It is designed to be as quick and easy to use as possible so that the developer of a UWS has not to worry with the UWS management. Basically, a developer has firstly to define jobs to use, and then, to create a HTTP servlet in which he creates its UWS and forwards it all requests. The library is implemented in the CDS TAP Library.
CDS ADQL Library	GregoryMantelet	2/2018	ADQL (Astronomical Data Query Language) is a SQL-like language specifically designed to query VO services and particularly TAP. It is defined by the IVOA in the following document: http://www.ivoa.net/Documents/latest/ADQL.html. To sum up ADQL adds to the basic SQL syntax some functions to select sky regions. For instance: CONTAINS(POINT('ICRS', ra, dec), CIRCLE('ICRS', 10, 5, 2) = 1 corresponds to a cone search around (10°,5°) with a radius of 2°. This library lets parsing, editing and translating queries expressed in ADQL 2.0.
CDS TAP Library	GregoryMantelet	2/2018	Java framework to quickly and easily set up a TAP service. A such service lets query an astronomical database with ADQL queries. This library uses the CDS UWS Library and the CDS ADQL Library.
UCiDy	GregoryMantelet	2/2017	Lightweight Java library to validate UCDs (Unified Content Descriptors). A documentation is provided on the Wiki of this GitHub repository. Stable releases are available in the Releases page.
UCD assignation tool	sebastien.derriere (at) astro.unistra.fr	10/2008	The interface allows to find the UCD corresponding to a description in natural langage (previous version available here)
samp.js	MarkTaylor	2013	Small javascript library to allow web pages/web applications (for instance ones associated with data providers) to interact with client-side tools using the SAMP Web Profile. Old, but still working in 2022.
MocInfo	Pierre Fernique	2017	MOC libraries. MOC (MultiOrder Coverage map) allows one to specify arbitrary sky regions. The goal is to provide a very fast comparison mechanism between coverage maps
JSAMP	Mark Taylor	2015	Java library for incorporating SAMP into java applications. Also contains SAMP utilities and diagnostic tools
Hipsgen	Pierre Fernique	2018	HiPS generator tool allowing to create a Hierarchical Progressive Survey for image or cube collection. See "How to make your HiPS in 10 steps" and HiPS CDS page.
Hipsgen-cat	François Xavier	2018	HiPS generator tool allowing to create a Hierarchical Progressive Survey for catalogs (see HiPS CDS page)
CDS FITS validator	Gilles Landais	2019	A web service based on the Saada API to check the FITS header ObsCore compatibility.

---