The VOEvent format has been a recommendation of the IVOA since 2006, and is gaining wide acceptance. The purpose is to announce celestial transient events that may need rapid follow-up observation, and it is designed so that automated systems, as well as humans, can make decisions about VOEvent packets. This document defines a way in which the Registry infrastructure of the Virtual Observatory can be used to allow publication, discovery, and utilization of VOEvent resources. We define a new registry extension schema - VOEventStream - that specifies two new types resource: VOEventStream and a VOEventServer. The stream resource describes a scientifically coherent collection of events; that come from the same motivation, team, project, or experiment; each event of a stream uses the same vocabulary (parameters) to describe events; while the primary thrust of the VOEvent work is responding to future events, a stream can also describe events that have already happened. The server resource describes which computers and interfaces can be used to send out future events (subscription capability) and/or can run queries on past events (query capability); it is described by the streams that it knows, and then by the capabilites it offers for each of those streams.
This document has been developed with support from the National Science Foundation's Information Technology Research Program under Cooperative Agreement AST0122449 with The Johns Hopkins University, from the UK Particle Physics and Astronomy Research Council (PPARC), and from the Eurpean Commission's Sixth Framework Program via the Optical Infrared Coordination Network (OPTICON).
The Virtual Observatory (VO) is general term for a collection of federated resources that can be used to conduct astronomical research, education, and outreach. The International Virtual Observatory Alliance (IVOA) is a global collaboration of separately funded projects to develop standards and infrastructure that enable VO applications.
XML document validation is a software process that checks that an XML document is not only well-formed XML but also conforms to the syntax rules defined by the applicable schema. Typically, when the schema is defined by one or more XML Schema [Schema] documents (see next section), validation refers to checking for conformance to the syntax described in those Schema documents. This document describes additional syntax constraints that cannot be enforced solely by the rules of XML Schema; thus, in this document, use of the term validation includes the extra checks that goes beyond common Schema-aware parsers which ensure conformance with this document.
UTC refers to Universal Coordinated Time as defined by....
The eXtensible Markup Language, or XML, is document syntax for marking textual information with named tags and is defined by the World Wide Web Consortium (W3C) Recommendation, XML 1.0 [XML]. The set of XML tag names and the syntax rules for their use is referred to as the document schema. One way to formally define a schema for XML documents is using the W3C standard known as XML Schema [Schema].
This document defines the VOEventStream schema using XML Schema. The full Schema document is listed in Appendix A. Parts of the schema appear within the main sections of this document; however, documentation nodes have been left out for the sake of brevity.
Reference to specific elements and types defined in the VOEventStream
schema include the namespaces prefix, voe
, as in
voe:VOEventStream
(a type defined in the VOEventStream schema).
Use of the voe
prefix in compliant instance documents is
not required (see section 3); its use in this
document is simply to indicate that it is an entity defined in the
VOEventStream schema.
"Real-time astronomy" is an increasingly important component of modern astronomy, the study of rapid phenomena in the sky such as supernovae, cataclysmic variables, blazars, gamma-ray bursts, microlensing, etc. Traditional event reporting systems include the Central Bureau for Astronomical Telegrams and the Astronomers Telegram, written in natural language, and inform astronomers so that they can personally run follow-up observations. However, real-time astronomy can discover more science by automation: for rapid decisions in seconds without the need to wait for humans to wake up. Automation is also necessary to deal with large numbers of events, such as from LSST in years to come, and automation also allows machines to fetch and collate other relevant data that may elucidate the reason for an event, and also improve the ability to decide and prioritize follow-up observation.
The primary conceptual idea of this document is the VOEventStream. It is analogous to an astronomical catalog, in the sense that all the entries of the stream are uniformly observed, with each entry having the same parameters. A stream is also like a catalog in that it has authors who take scientific responsibility for the data. The most obvious difference, however, is that the catalog entries come from observations in the past, whereas an event stream may have both past and future events. An example of a stream is the set of events published by the Catalina Real-Time Sky Survey: by repeatedly scanning the sky, this project finds and announces each night a handful of sky positions where the source is much brighter than previous observations. Another example could be the stream of events from the NASA Swift orbiting observatory.
Now that the structure of the VOEvent packet is converging, interoperation is enabled. Now we should turn attention to the infrastructure that carries the events: network protocols, standard services, and how a user can discover, find, and utilize VOEvent resources. We will achieve this with the Registry infrastructure of the virtual observatory, a distributed, international metadata repository. In this document we will define how VOEventStreams are recorded in the registry, and also the VOEventServers, that allow actual data to be received from streams.
In forming these collections of events as streams, the question of granularity arises: should we, for example, make all the high-energy burst events into a single stream (Swift + Integral + Fermi etc)? Or should each satellite have its own event stream? Or should each instrument of each satellite have its own stream? This is of course a matter of judgement; but there are some guidelines. First, a stream should have single point of leadership and scientific responsibility, so the idea of combining NASA and ESA resources into a single stream seems inappropriate. Second, a stream should have a single set of parameters for the event vocabulary, and the greater granularity implies a much larger number of Params, most of which would not be used in any given event. However, splitting in many small event streams is also inappropriate, since there would be much more metadata to define for the Author (once of each stream), and much more work for the Subscriber (once for each stream).
The registry enhancements proposed here will make it easier to work with VOEvents. Some use cases are:
ivo://my.datacenter.here/resourceName#localNameBy splitting this into pieces, we can find other IVORNs buried inside. The IVORN rules say there can only be one pound (#) sign (or none) in the IVORN. If so, the localName (after the #) is something not interpreted to the registry system. The part before the #, or the whole IVORN if there is no #, is like a call-number in the library, it is a reference to something known to the IVOA registry system: a dataset, a cone-search service, a workflow component, an organization. In order to create your own IVORNs, you will need an Authority ID, (for example ivo://my.datacenter.here), which most astronomy data centers already have.
We propose in this document to add two new kinds of resource (also known as extension schema) to those that the registry handles already. We do not register VOEvents themselves, but rather a coherent collection of events that we call VOEventStream. Each VOEvent is thus a representative of a class of events (the VOEventStream). The other extension scheme is VOEventServer, which lists one or more streams, and how to either pull event data from the past, or have event data pushed as soon as events occur.
The proposed syntax for the IVORN for a VOEvent is:
ivo://my.datacenter.here/streamName#eventNameAn event is resolved at the registry as follows: By splitting on the # sign, we can look up the stream in the registry and find the meaning of the quoted parameters. We can query the registry for services which handle that stream, in order to probe the event repositories for the event primary data and annotations that may have arisen about that event. Different repositories can add new data as they wish.
As noted above, each VOEventStream will get an IVORN upon registration, that can be used as the root for the IVORNs for events that are members of that stream.
Since the VOEventStream is defined as an extension of the current registry schema, much of this information is inherited from there. Therefore we describe only briefly, and refer readers to the registry documents and the examples below.
A key feature of the concept of VOEventStream is that each stream has a defined set of named "parameters", and each event that is a member of the stream should use only parameters that are selected from the list in the stream definition. The parameter set of an event is analogous to a FITS header: keyword-value pairs that convey specific information about the event; however in the case of events (unlike most FITS files), the set of possible paramteters and their meanings is defined in the stream record.
In VOEvent, the parameter definition follows, to a great extent, the VOTable standard. Each Param element has:
<Param name="max" ucd="meta.number" datatype="int" value="50"> <Description>Maximal number of records to retrieve</Description> </Param>In the next section, we split the defintion of the parameter from its value. The defintion resides in the regsitry with the VOEventStream, and the value resides in the VOEvent itself.
Now we come to the new version, where the definition of a parameter's meaning is separate from its value. In the VOEventStream, we define a ParamTemplate to be like the Param shown above but without the value of the parameter.
<Param name="max" ucd="meta.number" datatype="int"> <Description>Maximal number of records to retrieve</Description> </Param>and therefore all that is needed in the event itself is the name-value association:
<Param name="max" value="50">All of the other information can be added if the author desires, so that the VOEvent can stand by itself. But when the stream from which the event comes has been properly registered, only the name and value are strictly necessary.
Parameters can be collected into groups in VOEvent, for example a measurement and its error estimate can be combined:
<Group name="Rmag"> <Param name="value" ucd="phot.mag;em.opt.R" datatype="float" value="17.3"> <Param name="error" ucd="phot.mag;em.opt.R;stat.err" datatype="float" value="0.1"> </Group> phot.mag;em.opt.RTo build a ParamTemplate for a VOEventStream, the same structure can be used, but without the "value" attibutes. Note also that VOEvents and VOEventStreams may not have Groups within Groups: a Group can only contain Param elements, not other Groups.
The same rules of uniqueness are in force for ParamTemplate and Groups in VOEventStream, as for Params and Groups in VOEvent:
Once the stream is defined in the registry, the question is how to find it, with the kinds of services that might be wanted. These are known as Capabilities in the VO registry model. Three kinds of capability that we can mention are subscription, formal query, and informal query. If a server supports a subscription capability, it means that a client can submit a criterion (for example "R magnitude brighter than 17"), and events will be sent by the server in the future which satisfy that criterion. A formal query capability implies the existence of a formal request-response protocol by which queries can be made and results returned; this will be the "Simple Event Access Protocol" when it becomes an IVOA recommendation. An informal query capability means a web page with forms by which a collection of events can be browsed in some way.
The capability of a server is split further into Interfaces, which express precisely how the capability can be accessed by a client. For example, subscription capability could be achieved by having email, instant messaging, or text message sent; these are three different interfaces to the subscription capability, and there is different metadata associated with each (email address vs phone number for example). Having mentioned these basic concepts of the VO registry, we cna now define the parts of the VOEventServer definition:
This is about who is running the service, description and links about the service, and other information. It is precisely the same kind of XML as all registry resopuirces have, as alluded to in the section above.
Here the server record lists the streams to which it can provide access, with a set of <VOEventStream> elements, each of which has the IVORN of a stream. Further into the VOEventServer document is where ways of accessing those streams are defined.
This section enumerates the types and elements defined in the VOEventService schema and describes their meaning.
vr:Resource
type for describing three specific types of
resources:voe:DataStream
, voe:VOEventStream
and VOEventServer
.
<facility>
,
<instrument>
, <rights>
and
<coverage>
. This is very similar to the
DataCollection resource type defined in the VODataService extension
schema but this uses local type definitions and so cannot be
restricted or extended.
voe:DataStream Extension Metadata Elements | |||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Element | Definition | ||||||||||||||||
facility |
| ||||||||||||||||
instrument |
| ||||||||||||||||
rights |
| ||||||||||||||||
coverage |
|
dictionary
.
voe:VOEventStream Extension Metadata Elements | |||||||
---|---|---|---|---|---|---|---|
Element | Definition | ||||||
dictionary |
|
<dictionary>
element is of the complex type
voe:DataDictionary
which provides the defining metadata for the
<Param>
and <Group>
elements
employed in the <What>
section of a VOEvent packet:
<Group>
and
<Param>
elements for consistency with the
corresponding elements used in the <What>
section
of a VOEvent packet.
voe:DataDictionary Metadata Elements | |||||||
---|---|---|---|---|---|---|---|
Element | Definition | ||||||
description |
| ||||||
Group |
| ||||||
Param |
|
Multiple dictionaries are possible within the same
<VOEventStream>
element with different
versions. This allows for new parameters to be added to a stream's
dictionary but validation of stored events is still possible with the
existing resource record containing both the old and new dictionaries
with different versions. The attributes help distinguish between
multiple dictionaries within the same element:
voe:DataDescription Attributes | |||||||
---|---|---|---|---|---|---|---|
Attribute | Definition | ||||||
name |
| ||||||
version |
|
<Group>
element in the
<What>
section of a VOEvent packet. This is defined
by a <WhatGroup>
complex type in the
<dictionary>
element:
voe:WhatGroup Metadata Elements | |||||||
---|---|---|---|---|---|---|---|
Element | Definition | ||||||
description |
| ||||||
Param |
|
voe:WhatGroup Attribute | |||||
---|---|---|---|---|---|
Attribute | Definition | ||||
name |
|
<Param>
elements, appearing both
individually and collectively (in <Group>
elements) in the
<What>
section of a VOEvent, correspond to <WhatParam>
complex types in the <dictionary>
element:
voe:WhatParam Metadata Elements | |||||||
---|---|---|---|---|---|---|---|
Element | Definition | ||||||
description |
|
The purpose of the primary
attribute is to capture the
VOEvent author's opinion regarding the importance of a particular
parameter, especially in the context of displaying an abbreviated list
of parameters in a UI.
voe:WhatParam Attributes | ||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Attribute | Definition | |||||||||||||||||||
name |
| |||||||||||||||||||
unit |
| |||||||||||||||||||
ucd |
| |||||||||||||||||||
dataType |
| |||||||||||||||||||
use |
| |||||||||||||||||||
std |
| |||||||||||||||||||
primary |
|
The VOEventServer resource type is used to describe a server that
provides VOEvent streams. It extends the
<vs:Service>
type by adding an additional element
to the core set of metadata, <voeventStream>
.
The <voeventStream>
elements list the VOEvent
streams that are served by the VOEventServer. Each should be
resolvable to a resource record of type <VOEventStream>
.
voe:VOEventServer Extension Metadata Elements | |||||||
---|---|---|---|---|---|---|---|
Element | Definition | ||||||
voeventStream |
|
A VOEventServer can have a number of <capability>
elements; in particular, capabilities of type
<voe:SimpleEventAccess>
and
<voe:Subscription>
. Abstract complex types are
defined for both so that the value of the standardID
attribute can be set to appropriate fixed URIs represeting the service
standards. For the <voe:SimpleEventAccess>
capability, this is the SEAPCapRestriction
complex type:
voe:SEAPCapRestriction Metadata Elements | |||||||||
---|---|---|---|---|---|---|---|---|---|
Element | Definition | ||||||||
validationLevel |
| ||||||||
description |
| ||||||||
interface |
|
The fixed URI representing the Simple Event Access Protocol (SEAP)
standard is defined by the standardID
attribute:
voe:SEAPCapRestriction Attribute | |||||||||
---|---|---|---|---|---|---|---|---|---|
Attribute | Definition | ||||||||
standardID |
|
The SimpleEventAccess
capability is then defined as an extension of
the abstract <SEAPCapRestriction>
:
The capability defines three elements relating to characteristics
of the SEAP:
<maxQueryRegionSize>
,
<maxRecords>
and <testQuery>
:
voe:SimpleEventAccess Extension Metadata Elements | |||||||
---|---|---|---|---|---|---|---|
Element | Definition | ||||||
maxQueryRegionSize |
| ||||||
maxRecords |
| ||||||
testQuery |
|
<maxQueryRegionSize>
is of the complex
type <voe:RegionSize>
:
The elements of this specify the maximum extent along the longitudinal (R.A.), latitudinal (Dec) and temporal axes that the service can handle:
voe:RegionSize Metadata Elements | |||||||
---|---|---|---|---|---|---|---|
Element | Definition | ||||||
long |
| ||||||
lat |
| ||||||
time |
|
The <testQuery>
child of the
<SimpleImageAccess>
capability is of the complex
type <voe:Query>
:
This awaits the definition of SEAP before it can be fully specified.
The abstract complex type for the <voe:Subscription>
capability is EventSubCapRestriction
:
voe:EventSubCapRestriction Metadata Elements | |||||||||
---|---|---|---|---|---|---|---|---|---|
Element | Definition | ||||||||
validationLevel |
| ||||||||
description |
| ||||||||
interface |
|
The fixed URI representing the Subscription standard is defined by
the standardID
attribute:
voe:EventSubCapRestriction Attribute | |||||||||
---|---|---|---|---|---|---|---|---|---|
Attribute | Definition | ||||||||
standardID |
|
The Subscription
capability is then defined as an
extension of the abstract <EventSubCapRestriction>
The capability defines one element indicating whether a user can filter the events they subscribe to with this service, i.e. can they specify subsets that they are interested in or do they just receive all events emanating from the server.
voe:Subscription Extension Metadata Elements | |||||||
---|---|---|---|---|---|---|---|
Element | Definition | ||||||
supportsFilters |
|
The VOEventService schema defines three derived
vr:Interface
types to be used with the
voe:Subscription capability:
voe:Jabber
,
voe:TCPV
and voe:RSS
.
The voe:Jabber
interface indicates that VOEvents are
available from this server through a Jabber/XMPP interface. The
<accessURL>
gives the endpoint for the Jabber server.
The <feedNode>
element gives the details of the
Jabber subscription nodes for each of the VOEvent streams that are
accessible through this interface.
voe:Jabber Extension Metadata Elements | |||||||
---|---|---|---|---|---|---|---|
Element | Definition | ||||||
referenceURL |
| ||||||
description |
| ||||||
feedNode |
|
The <FeedNode>
is of the complex type <voe:Feed>
:
voe:Feed Attribute | |||||
---|---|---|---|---|---|
Attribute | Definition | ||||
streamId |
|
The voe:TCPV
interface indicates that VOEvents are
available from this server though a TCP Vanilla interface. The
<accessURL>
gives the endpoint from the server.
voe:TCPV Extension Metadata Elements | |||||||
---|---|---|---|---|---|---|---|
Element | Definition | ||||||
referenceURL |
| ||||||
description |
|
The voe:RSS
interface indicates that VOEvents are
available from this server through a RSS feed. The
<accessURL>
gives the endpoint for the RSS server.
The <feed>
gives the details of the RSS
subscription endpoints for each of the VOEvent streams that are accessible
through this interface.
voe:RSS Extension Metadata Elements | |||||||
---|---|---|---|---|---|---|---|
Element | Definition | ||||||
referenceURL |
| ||||||
description |
| ||||||
feed |
|
<?xml version="1.0" encoding="UTF-8"?> <Resource xsi:type="voe:VOEventStream" created="2008-09-23T00:00:00Z" updated="2008-09-23T00:00:00Z" status="active" xsi:schemaLocation="http://www.ivoa.net/xml/VOEventService/v0.1 VOEventService.xsd" xmlns:vr="http://www.ivoa.net/xml/VOResource/v1.0" xmlns:voe="http://www.ivoa.net/xml/VOEventService/v0.1" xmlns:stc="http://www.ivoa.net/xml/STC/stc-v1.30.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xlink="http://www.w3.org/1999/xlink"> <validationLevel validatedBy="ivo://nvo.caltech/registry">1</validationLevel> <title>Catalina Real-time Transients Survey VOEvent Stream</title> <shortName>catot</shortName> <identifier>ivo://nvo.caltech.voeventnet/catot</identifier> <curation> <publisher ivo-id="ivo://nvo.caltech/voeventnet">VOEventNet</publisher> <creator> <name>Andrew Drake</name> </creator> <date role="creation">2007-11-01</date> <contact> <name>Andrew Drake</name> <email>ajd@cacr.caltech.edu</email> </contact> </curation> <content> <subject>optical</subject> <subject>transient</subject> <description>The Catalina Real-time Transient Survey (CRTS) uses data obtained by the Catalina Sky Survey (CSS) to search for optical transients. CSS observations are usually taken 21 nights per lunation.</description> <referenceURL>http://www.voeventnet.org</referenceURL> <type>Other</type> <contentLevel>Research</contentLevel> <contentLevel>Amateur</contentLevel> </content> <facility>CSS Schmidt Telescope, Mt Bigelow, AZ</facility> <coverage> <stc:STCResourceProfile> <stc:AstroCoordSystem xlink:type="simple" xlink:href="ivo://STClib/CoordSys#UTC-ICRS-TOPO" id="VOEvent_UTC_ICRS_TOPO"/> <stc:AstroCoordArea coord_system_id="VOEvent_UTC_ICRS_TOPO"> <stc:AllSky/> </stc:AstroCoordArea> </stc:STCResourceProfile> <waveband>Optical</waveband> </coverage> <dictionary version="1.0"> <Group name="Asteroid params"> <description>...</description> <Param unit="deg" ucd="pos.posAng" name="Opposition angle" dataType="float"/> <Param unit="arcsec/min" ucd="pos.pm" name="Inner motion" dataType="float"/> <Param unit="arcsec/min" ucd="pos.pm" name="Outer motion" dataType="float"/> <Param unit="deg" ucd="pos.ecliptic.lon" name="Ecliptic Longitude" dataType="float"/> <Param unit="deg" ucd="pos.ecliptic.lat" name="Ecliptic Latitude" dataType="float"/> <Param unit="arcsec" ucd="pos.pm" name="Apparent motion" dataType="float"/> <Param unit="min" ucd="time.interval" name="Timespan" dataType="float"/> <Param unit="deg" ucd="pos.posAng" name="Ecliptic Inclination" dataType="float"/> <Param unit="arcsec/min" ucd="stat.error.sys" name="Motion uncertainty RA" dataType="float"/> <Param unit="arcsec/min" ucd="stat.error.sys" name="Motion uncertainty Dec" dataType="float"/> </Group> <Group name="First Detection params"> <Param unit="mag" ucd="phot.mag;em.opt.R4" name="magnitude" dataType="float" primary="true"/> <Param unit="mag" ucd="phot.mag;stat.error" name="mag_error" dataType="float"/> <Param ucd="meta.id" name="ID" dataType="long" /> <Param unit="pixels" ucd="instr.det.psf" name="average seeing" dataType="float"/> <Param unit="cts" ucd="instr.background" name="average background" dataType="float"/> <Param unit="days" ucd="time.epoch" name="JD" dataType="float"/> <Param unit="deg" ucd="pos.eq.ra" name="RA" dataType="float" primary="true"/> <Param unit="deg" ucd="pos.eq.dec" name="Dec" dataType="float" primary="true"/> </Group> </dictionary> </Resource>
<?xml version="1.0" encoding="UTF-8"?> <Resource xsi:type="voe:VOEventServer" created="2008-09-23T00:00:00Z" updated="2008-09-23T00:00:00Z" status="active" xsi:schemaLocation="http://www.ivoa.net/xml/VOEventService/v0.1 VOEventService.xsd" xmlns:vr="http://www.ivoa.net/xml/VOResource/v1.0" xmlns:voe="http://www.ivoa.net/xml/VOEventService/v0.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <validationLevel validatedBy="ivo://nvo.caltech/registry">1</validationLevel> <title>VOEventNet VOEvent Server</title> <shortName>voeventnet</shortName> <identifier>ivo://nvo.caltech/voeventnet</identifier> <curation> <publisher ivo-id="ivo://nvo.caltech/CACR">Center for Advanced Computing Research</publisher> <creator> <name>Roy Williams</name> </creator> <date role="creation">2006-04-01</date> <contact> <name>Andrew Drake</name> <email>ajd@cacr.caltech.edu</email> </contact> </curation> <content> <subject>VOEvent</subject> <subject>transient astronomy</subject> <description>VOEventNet gathers streams of astronomical alerts and reports in a common format, and acts as a clearinghouse, so that both people and robotic systems can get the alerts quickly enough to respond with follow-up observations.</description> <referenceURL>http://www.voeventnet.org</referenceURL> <type>Other</type> <contentLevel>University</contentLevel> <contentLevel>Amateur</contentLevel> <contentLevel>Research</contentLevel> </content> <voeventStream>ivo://nvo.caltech.voeventnet/catot</voeventStream> <capability xsi:type="voe:SimpleEventAccess" standardID="ivo://ivoa.net/std/VOEventSEAP"/> <capability xsi:type="voe:Subscription" standardID="ivo://ivoa.net/std/VOEventSubscribe"> <validationLevel validatedBy="ivo://nvo.caltech/registry">1</validationLevel> <interface xsi:type="voe:Jabber"> <accessURL>http://moriori.cacr.caltech.edu</accessURL> <securityMethod standardID="ivo://ivoa.net/std/UserPass"/> <feedNode streamId="ivo://nvo.caltech.voeventnet/catot">home/moriori.cacr.caltech.edu/catalina</feedNode> </interface> <interface xsi:type="voe:RSS"> <accessURL>http://www.voeventnet.org</accessURL> <feed streamId="ivo://nvo.caltech.voeventnet/catot">http://www.voeventnet.org/feeds/Catfeed.xml</feed> </interface> <supportsFilters>true</supportsFilters> </capability> </Resource>