Sky Event Reporting Metadata (VOEvent)
Version 0.2

IVOA WG Internal Draft 2004-11-23

Editor:

Roy Williams

Other Authors

Joshua Bloom

Matthew Graham

Ray Plante

Chris Stoughton

Przemyslaw Wozniak

---

Abstract

The objective of the VOEvent effort is to define the content and meaning of a standard information packet for representing, transmitting, archiving, and publishing a discovery of an immediate event in the sky. We will call this packet VOEvent. The objective is to drive robotic telescopes, to drive archive searches, to alert the community, and to build interoperable archives. The effort will include not just "photon" events, but also gravitational waves, neutrinos, air showers, etc.

Structured data is the target here, rather than natural language, so that automated systems can be most effective in interpreting VOEvent packets. The packet would contain "Who, When, Where, What, How" of the event. The "How" section would be instrument metadata, which would be an included schema specific to each instrument, and the "What" section would define event types, that would also be an included schema, because every project would have differne types of events.

VOEvent packets would gain persistent identifiers and be persistently stored in databases and registries. VOEvent packets could therefore reference other packets in various ways. The intention is that subscribers could get immediate notification of events, based on previously defined criteria. The standard does not define a transport layer or the construction of registries, clients, etc. However, the intended usage of the VOEvent packet would, for example, require packets to be small, and to be processed quickly, and would allow humans or machines to get very fast notification. The standard does not cover policy issues such as who can publish, who can build a registry of events, who can subscribe to a particular registry, nor the intellectual property issues.

1. Introduction

Throughout human history, unexpected events in the sky have been interpreted as portents and omens. In the next years this study will be greatly extended with digital technology, with many new projects making wide-area systematic searches for events in the sky. Such events may be local phenomena (comets, asteroids, Kuiper-belt objects, etc), or more distant (novae, cataclysmic variables, supernovae, gamma-ray bursts, lensing events, etc). Most exciting of all would be new and unknown types of event, heralding a new field of astrophysics. Searches for astrophysical events are taking place at all the electromagnetic wavelengths from radio to gamma, but also underway are gravitational wave searches, neutrino detectors, and high-energy cosmic ray searches.

For many types of events, astrophysical knowledge is gained through fast, comprehensive followup observation -- a spectrum for example -- and by views with different types of instrument. To satisfy this need, several projects are setting up robotic telescopes, that can respond to a digital message to point the telescope and make observations -- but in near-real-time and without human intervention. We can also imagine automated systems for examining archives that respond to reports of events.

Therefore we propose the following sequence for building an information infrastructure for events, obtaining international agreements where necessary to ensure interoperability.

• First we agree upon the semantics of the structured messages that move in the infrastructure (this document). Representatives of projects are expected to assess the usefulness of this message against their intended use, and use this point of view to modify this document.
• The VOEvent information is then encoded in a suitable format such as XML. Different working groups will make specific schema that will be included in the VOEvent schema: these include defining instrument charactaristics, and defining the nature of the event that is being reported. See the figure below and sections 3.3 and 3.4.
• A transport and storage layer for the messages involves defining the services that databases implement. Issues include persistency, how to subscribe to a database, how identifiers are obtained, how databases communicate for harvesting, how the system responds to flooding or abuse, how messages should appear to subscribers (email, phone, etc), and how queries can be made to the database.
• Implementations of VOEvent databases can be made with different technologies, and there should be a forum for discussing these, especially failure to interoperate.
• Each database will have policies about who can submit discovery packets, and who can subscribe to receive them.

1.1 Plan

Part of the VOEvent semantics is common to all reports of transients, for example the publication information about who is responsible for the report, and the estimates of time and place in sky. However, there are two parts of the schema that are specific to a given publisher: the description of the nature of the event, and the description of the instrumental state when the observation occurred. In each case, we expect that sub-groups of the VOEvent working group will take on these definitions. For example, the gravitational-wave community would define the schema for the "state of the detector" corresponding to a reported event, and they would also decide the nature of the event metadata itself, perhaps as "inspiral event, M1=1.4, M2 = 2.1"

The responsibility for defining the VOEvent schema is sketched in the figure below. Some parts are common to all such events, being the curation, sky location, and timing of the event: these cvan be defined as an IVOA standard, allowing interoperability between different observational groups. Other parts of the schema are specific to the instrument that observed the event, and we expect sub-groups of the VOEvent Working Group to define these.

1.2 Use of VO Identifiers

In building the VOEvent packet, we make use of the IVOA identifier syntax that has been developed as part of the VO registry framework. These identifiers are recognizable because they begin with ivo://, and they are meant as a stand-in for a particular metadata packet that is obtainable from a VO registry. A registered VOEvent packet is one that has a valid identifier -- meaning that a registry exists that can resolve that identifier to the full packet. The identifiers thus provide a citation mechanism, a way to express that one VOEvent packet is a follow-up of a previous packet.

Another way to use VO identifiers is for efficiency. One section of the VOEvent schema is about authorship (who is responsible for this candidate discovery), and that section may be replaced by a VO identifier which points to the relevant "organization". If a group creates VOEvent packets regularly, all with the same authorship, it would then be efficient for them to use the same VO identifier in each packet rather than sending the whole list of people and contacts each time.

2. Expected Usage

We are expecting VOEvent packets to be used as the basis of an information infrastructure of databases, or registries, that hold VOEvent packets, keyed by their identifiers. These databases may harvest packets from each other, so that a packet may be held at more than one database. In addition to the harvesting protocol, there would be three ways for clients to interact with the database servers:

• Publishing: a client creates a VOEvent packet without an IVO identifier, communicates with some database server to add the packet to the holdings, and get the persistent identifier.
• Subscription: a client can create a query and lodge it with a VOEvent database; whenever a packet comes in that matches the query, the client is immediately informed about the new packet.
• Search: a client can send a query to a VOEvent database server, and obtain all event packets which match it.

A subscription to an event service is a filter on the stream of events that an event registry obtains: whenever certain criteria are met for an incoming event, then the subscriber is notified by a transport mechanisme that the subscriber has chosen. The filter may involve the curation part of the event (eg. all events published by the Swift spacecraft), the location (eg anything in M31), or they may involve the detailed metadata of the event itself (eg. whenever the cosmic ray enengy is greater than 3 TeV).

An announcement of a new phenomenon in the sky may eventually be Nobel-prize material, and we hope that a VOEvent packet would be the chosen medium for the very first announcement. However, we felt that the astronomical community would generally be happy with an open system, and therefore VOEvent packets do not contain any information about intellectual property restrictions on the data they contain. If there are projects that produce "secret" packets, they can certainly work within a closed system of clients and servers, without allowing any harvesting. We felt this latter solution to be simpler and more effective than demanding that all servers understand a schema for IP restriction.

3. VOEvent Packet Content

3.1 Message types

A VOEvent packet expresses the discovery of a sky event, located in a specified spacetime region, observed by some instrument, and published by some some group of people. This "Discovery" type is the primary purpose of VOEvent and the default packet type

packetType (may be discovery, followup, intendedFollowup, merge, veto, reject)
The "packetType" may have other values that modify the semantics: besides the "Discovery" type, a VOEvent may have type "Followup", and cite one or more other packets -- meaning that the observation was done as a response to cited packet. In this case, the positional information is assumed to to be a new, independent measurement, not just a repeat of what was in the cited packet.

The packetType may also be "IntendedFollowup", which expresses an intention of following up on the cited packet. In this case, the positional information is interpreted as a "region of study" in the followup observation ("this is where we will look").

The merge packet type indicates that discoveries or followups which were not initially identified recognized to be due to the same physical event should be merged with each other.

The veto packet type indicates that followup is not proposed for a specific reason

The reject packet type indicates that the initial discovery is in error and further work should not continue.

packetRole (may be Real, TestPleaseIgnore)
Another flag that may be present in the packet is the "packetRole" which can take the default value "Real", meaning that the packet describes an observation of the sky, or it may take the value "TestPleaseIgnore", meaning that the packet DOES NOT DESCRIBE ASTRONOMY, but rather is part of a testing procedure of some kind. It is the responsibility of all who receive VOEvent packets to pay attention to the packetRole, and be quite sure of the difference between a real discovery and a test of the system.

3.2 Generalized Position

The VOEvent packet may include information about where and when the event was detected, complete with error boxes of various kinds, as noted below. If the spatial or temporal locators are not present, it is to be assumed that the information is either unknown or irrelevant.

Coord1 (double, angle in degrees)
Coord2 (double, angle in degrees)
Frame (string, default J2000)
The sky position will be expressed by a pair of angular coordinates, in degrees, and a "frame" name that allows intepretation of those coordinates. The default frame is "J2000", in which case the coordinates are interpreted as right ascension and declination in the J2000 frame. The error estimate of a sky position may be expressed as a circular, elliptical, or polygonal region of error. Furthermore, the positional estimate could be disconnected -- for example the sky position could be an intersection of two annuli.

Time (double, modified Julian Day)
TimeUncertainty (double, fraction of a Julian Day)
Time is an estimate of the time at which the event occurred, as seen from the location of the observing instrument, in Universal Time. Time will be expressed as a modified Julian day number (number of days since November 17th, 1858). The MJD includes a fractional day to arbitrary precision. The TimeUncertainty is an estimate of the uncertainty of the Time value. It is assumed to subsume exposure times, interval since last observation, as well as statistical errors in determining time.

detectionChannel (string)
While much astronomy has been done by collecting photons, other types of detector are appearing that may have the ability to detect events. Thus the traditional "wavelength regime" of the photon astronomer is generalized to "detectionChannel", that may be "Electromagnetic" (together with the wavelength range), or "Gravitational Wave", "Neutrino", or "Cosmic Ray", or others.

3.3 Instrument Configuration

Another section of the VOEvent packet indicates the instrument with which the event observation was made, together with information about how the instrument was being used. For an optical telescope, for example, the description micht include aperture, bandpass, and magnitude limit; for an Xray telescope it might include high and low energy bandpass and flux limits.

instrumentName (string)
instrumentType (one of set of known schema)
instrumentConfiguration (XML document)
We model the instrument as a combination of an "instrumentName" (eg Palomar 60-inch), and an instrument type (eg "opticalTelescope"). The instrument type implicitly refers to a schema for that type of instrument, a schema that has been agreed among domain experts. The instrument description can then be written as an instance of the relevant schema.

This is another occasion where a VO identifier could be used as an abbreviation of providing full instrument metadata. If a survey team always uses its instrument in the same configuration, it may be worth registering the metadata packet that describes the instrument, so that an identifier can be used in place of repeating the description for each VOEvent packet. In this case, the available information would be: instrument name, instrument type, and the identifier for the specific configuration.

A good possibility for the description of instrument configuration would be the RTML language (Remote Telescope Markup Language).......

3.4 Event Description

eventType (one of set of known schema)
eventDescription (XML document)
This is the "What" part of the definition of the event, and obviously depends significantly on the instrument, detection channel, and investigating team that took the data. Subscribers to an event service will expect to build queries based on this part of the event metadata -- an example from optical astronomy might be: when the B magnitude difference is greater than 1.0, or for a gravitational wave detector it might be: when the SNR for an inspiral event is greater than 3.0 in two detectors within 1.0 seconds. Note that the last is an example of a joint query: two events must be reported close together before the subscriber is alerted.

The VOEvent working group will expect instrument teams to cooperate with others to build these definitions of the event, keeping always to the stricture that VOEvent reports must be kept small.

3.5 Curation

A section of the VOEvent packet is devoted to curation information: who is responsible for the information content of the packet. We follow the IVOA Resource Metadata document [1] in defining curation.

Publisher (string)
An entity responsible for making the resource available. Examples of a Publisher include a person or an organization. Users of the resource should include Publisher in subsequent credits and acknowledgments. The Publisher may be either a text string or a VO identifier pointing to an organization.

Contributor (string)
An entity responsible for making contributions to the content of the resource. Examples of a Contributor include a person or an organization. Users of the resource should include all Contributors in subsequent credits and acknowledgments.

Contact (string, e-mail address)
The e-mail address for contacting the persons responsible for the resource. Contact is split into two components for clarity, Contact.Name and Contact.Email.

Date (YYYY-MM-DD)
A date associated with an event in the life cycle of the resource. Typically, Date will be associated with the creation or availability (i.e., most recent release or version) of the resource. ISO8601 is the preferred format (YYYY-MM-DD).

3.6 Reliability

Reliability (double between 0 and 1)
Reliability of an observation is highly subjective, and therefore difficult to interpret by another, yet at the same time it should be possible for the observing team to indicate reliability by comparison with other observations made by the same team. Therefore we simply state that any creator of VOEvent packets should make the reliability number however they wish; they may publish an explanation or not. Zero means unreliable, and unity means very reliable. However, it should be stated that a reliability of zero is NOT the same as setting the PacketRole to TestPleaseIgnore.

3.7 Links

a (href=URL string)
It may be that there are images, spectra, time-series or other information that acts as supporting evidence for the event detection. These may be linked from the VOEvent packet by the usual HTTP mechanism <a href=http://...>.

4. Persistent Identifiers

ivoID (ivo identifier URI)
As noted in the Introduction, VOEvent packets may contain VO identifiers, as defined and discussed in [2]. They take the form ivo://authority/resourceKey, which are references to metadata packets that may be found at a VO registry or VOEvent database.

The lookup procedure is similar to looking up a URL on the world wide web: each registry controls a number of authority IDs. These are like domain names on the net: each is resolved to exactly one endpoint machine through a system of distributed knowledge. Once that machine is discovered, it should be able to resolve the secondary part of the identifier, the resourceKey. Indeed, the machine that holds an authority ID has made a promise to resolve all the resourceKeys that it has issued.

As work continues on the VOEvent structure, we will work with the IVOA to define a message protocol by which a packet may obtain a valid IVOA identifier from a database of such packets. Broadly speaking, the creator of the packet (client) would be submit to the server a packet with empty identifier, the server check syntax and respond with the same packet, but with the ivo:// identifier filled in.

5. References

• Resource Metadata for the Virtual Observatory
  http://www.ivoa.net/Documents/latest/RM.html
• IVOA Identifiers
  http://www.ivoa.net/Documents/latest/IDs.html

Changelog

• Chris Stoughton -- October 11, 2004
  1. change precision of coordinates and time from float to double
  2. the time data structure needs system and precision
  3. added some packetTypes, not sure if this is the right place. For one neutrino burst every 100 years, we don't need to worry about a veto or error, but when LSST generates a bazillion events each night, we need a triage system. Just as you can decide whether to pay attention to a group's discoveries, you can also decide whether to pay attention to some other group's veto, merge, and reject suggestions.
  4. Questions:
     • Events on solar system bodies: volcanoes on Io, impacts on the Moon, Solar flares, and so on.
     • Do you want to be more explicit about what you expect to happen next with this document? You mention in the first bullet point in the Introduction that "representative of projects are expected to asses the usefulness of this message against their intended use." It may be more effective to have a more explicit call to action.
• Roy Williams -- November 18, 2004
  1. change references to "transients" to "events"
  2. modified abstract for clarity
  3. added "Nature of event" section