Sky Event Reporting Metadata (VOEvent)
Version 0.3

IVOA WG Internal Draft 2004-12-27

Roy Williams


The objective of the VOEvent effort is to define the content and meaning of a standard information packet for representing, transmitting, archiving, and publishing a discovery of an immediate event in the sky. We will call this packet VOEvent. The objective is to drive robotic telescopes, to drive archive searches, to alert the community, and to build interoperable archives. The effort will include not just "photon" events, but also gravitational waves, neutrinos, air showers, etc.

Structured data is the target here, rather than natural language, so that automated systems can be most effective in interpreting VOEvent packets. The packet would contain "Who, When, Where, What, How" of the event. The "How" section would be instrument metadata, which would be an included schema specific to each instrument, and the "What" section would define event types, that would also be an included schema, because every project would have differne types of events.

VOEvent packets will gain persistent identifiers and be persistently stored in databases and registries. VOEvent packets could therefore reference other packets in various ways. The intention is that subscribers could get immediate notification of events, based on previously defined criteria. The standard does not define a transport layer or the construction of registries, clients, etc. However, the intended usage of the VOEvent packet would, for example, require packets to be small, and to be processed quickly, and would allow humans or machines to get very fast notification. The standard does not cover policy issues such as who can publish, who can build a registry of events, who can subscribe to a particular registry, nor the intellectual property issues.

1. Introduction

Throughout human history, unexpected events in the sky have been interpreted as portents and omens. In the next years this study will be greatly extended with digital technology, with many new projects making wide-area systematic searches for events in the sky. Such events may be local phenomena (comets, asteroids, Kuiper-belt objects, etc), or more distant (novae, cataclysmic variables, supernovae, gamma-ray bursts, lensing events, etc). Most exciting of all would be new and unknown types of event, heralding a new field of astrophysics. Searches for astrophysical events are taking place at all the electromagnetic wavelengths from radio to gamma, but also underway are gravitational wave searches, neutrino detectors, and high-energy cosmic ray searches.

For many types of events, astrophysical knowledge is gained through fast, comprehensive followup observation -- a spectrum for example -- and by views with different types of instrument. To satisfy this need, several projects are setting up robotic telescopes, that can respond to a digital message to point the telescope and make observations -- but in near-real-time and without human intervention. We can also imagine automated systems for examining archives that respond to reports of events.

There are several projects coming online that will discover immediate events in the sky, for example Swift, Ligo, Pannstars, Palomar-Quest, and LSST. There are also many new robotic telescopes that will follow-up such events very quickly. In the past, several "human-centric" event systems have been very successful, for example the IAU telegrams, the Gamma-ray burst Coordinate Network, the Astronomer's Telegram, etc. These systems use natural-language text to describe the nature of the event, and send all qualified events to all subscribers.

However, we expect a larger number of events as the new projects come online, and we expect them to be handled by machines. We expect subscribers to make a careful filter for the events they want, for example events from a specific survey, events that will be above the horizon. There could be very detailed subscription requirements, perhaps events where the R-band magnitude increase was at least 2. Machines can select events based on coincidence. A gravitational wave detector, for example, might produce a large number of candidate events, but the interesting ones coincide with candidates from another detector.

Therefore we propose the following sequence for building an information infrastructure for events, obtaining international agreements where necessary to ensure interoperability.

1.1 Plan

In this document, we define the semantics of an announcement that an event has been found; with attached information about who, how, and where; with what instrument; and what is the reliability of the announcement. The semantics will then be represented in a structured data schema, presumably as an XML document, and the resulting instances of that schema used to drive an information infrastructure as noted above.

Part of the VOEvent semantics is common to all reports of transients, for example the publication information about who is responsible for the report, and the estimates of time and place in sky. However, there are two parts of the schema that are specific to a given publisher: the description of the nature of the event, and the description of the instrumental state when the observation occurred. In each case, we expect that sub-groups of the VOEvent working group will take on these definitions. For example, the gravitational-wave community would define the schema for the "state of the detector" corresponding to a reported event, and they would also decide the nature of the event metadata itself, perhaps as "inspiral event, M1=1.4, M2 = 2.1"

The responsibility for defining the VOEvent schema is sketched in the figure below. Some parts are common to all such events, being the curation, sky location, and timing of the event: these can be defined as an IVOA standard, allowing interoperability between different observational groups. Other parts of the schema are specific to the instrument that observed the event, and we expect sub-groups of the VOEvent Working Group to define these.

1.2 Use of VO Identifiers

In building the VOEvent packet, we make use of the IVOA identifier syntax that has been developed as part of the VO registry framework. These identifiers are recognizable because they begin with ivo://, and they are meant as a stand-in for a particular metadata packet that is obtainable from a VO registry. A registered VOEvent packet is one that has a valid identifier -- meaning that a registry exists that can resolve that identifier to the full packet. The identifiers thus provide a citation mechanism, a way to express that one VOEvent packet is a follow-up of a previous packet.

Another way to use VO identifiers is for efficiency. One section of the VOEvent schema is about authorship (who is responsible for this candidate discovery), and that section may be replaced by a VO identifier which points to the relevant "organization". If a group creates VOEvent packets regularly, all with the same authorship, it would then be efficient for them to use the same VO identifier in each packet rather than sending the whole list of people and contacts each time.

2. Expected Usage

We are expecting VOEvent packets to be used as the basis of an information infrastructure of databases, or registries, that hold VOEvent packets, keyed by their identifiers. These databases may harvest packets from each other, so that a packet may be held at more than one database. In addition to the harvesting protocol, there would be three ways for clients to interact with the database servers:

The subscription mechanism is expected to be the chief way in which users will be informed of new events, and we expect it to happen quickly. Transport mechanisms could include email and cellphone, as well as one-way web services. In this area, we expect the IVOA Events Working Group to find best practices and ensure interoperability.

A subscription to an event service is a filter on the stream of events that an event registry obtains: whenever certain criteria are met for an incoming event, then the subscriber is notified by a transport mechanisme that the subscriber has chosen. The filter may involve the curation part of the event (eg. all events published by the Swift spacecraft), the location (eg anything in M31), or they may involve the detailed metadata of the event itself (eg. whenever the cosmic ray energy is greater than 3 TeV).

An announcement of a new phenomenon in the sky may eventually be Nobel-prize material, and we hope that a VOEvent packet would be the chosen medium for the very first announcement. However, we felt that the astronomical community would generally be happy with an open system, and therefore VOEvent packets do not contain any information about intellectual property (IP) restrictions on the data they contain. If there are projects that produce "secret" packets, they can certainly work within a closed system of clients and servers. We felt this latter solution to be simpler and more effective than demanding that all servers understand a schema for IP restriction.

3. VOEvent Packet Content

3.1 Message types

A VOEvent packet expresses the discovery of a sky event, located in a specified spacetime region, observed by some instrument, and published by some some group of people. This "Discovery" type is the primary purpose of VOEvent and the default packet type

packetType (may be Discovery, Followup, IntendedFollowup, Merge, Veto, Reject)
The "packetType" may have other values that modify the semantics: besides the "Discovery" type, a VOEvent may have type "Followup", and cite one or more other packets -- meaning that the observation was done as a response to cited packet. In this case, the positional information is assumed to to be a new, independent measurement, or an improvement of the original.

The packetType may also be "IntendedFollowup", which expresses an intention of following up on the cited packet. In this case, the positional information is interpreted as a "region of study" in the followup observation ("this is where we will look").

The Merge packet type indicates that discoveries or followups which were not initially identified recognized to be due to the same physical event should be merged with each other.

The Veto packet type indicates that followup is not proposed for a specific reason

The Reject packet type indicates that the initial discovery is in error and further work should not continue.

packetRole (may be Real, TestPleaseIgnore)
Another flag that may be present in the packet is the "packetRole" which can take the default value "Real", meaning that the packet describes an observation of the sky, or it may take the value "TestPleaseIgnore", meaning that the packet DOES NOT DESCRIBE ASTRONOMY, but rather is part of a testing procedure of some kind. It is the responsibility of all who receive VOEvent packets to pay attention to the packetRole, and be quite sure of the difference between a real discovery and a test of the system.

3.2 Generalized Position

The VOEvent packet may include information about where and when the event was detected, complete with error boxes of various kinds, as noted below. If the spatial or temporal locators are not present, it is to be assumed that the information is either unknown or irrelevant.

Coord1 (double, angle in degrees)
Coord2 (double, angle in degrees)
Frame (string, default J2000)
The sky position will be expressed by a pair of angular coordinates, in degrees, and a "frame" name that allows intepretation of those coordinates. The default frame is "J2000", in which case the coordinates are interpreted as right ascension and declination in the J2000 frame. The error estimate of a sky position may be expressed as a circular, elliptical, or polygonal region of error. Furthermore, the positional estimate could be disconnected -- for example the sky position could be an intersection of two annuli.

Time (double, modified Julian Day)
TimeUncertainty (double, fraction of a Julian Day)
Time is an estimate of the time at which the event occurred, as seen from the location of the observing instrument, in Universal Time. Time will be expressed as a modified Julian day number (number of days since November 17th, 1858). The MJD includes a fractional day to arbitrary precision. The TimeUncertainty is an estimate of the uncertainty of the Time value. It is assumed to subsume exposure times, interval since last observation, as well as statistical errors in determining time.

detectionChannel (string)
While much astronomy has been done by collecting photons, other types of detector are appearing that may have the ability to detect events. Thus the traditional "wavelength regime" of the photon astronomer is generalized to "detectionChannel", that may be "Electromagnetic" (together with the wavelength range), or "Gravitational Wave", "Neutrino", or "Cosmic Ray", or others.

3.3 Instrument Configuration

Another section of the VOEvent packet indicates the instrument with which the event observation was made, together with information about how the instrument was being used. For an optical telescope, for example, the description micht include aperture, bandpass, and magnitude limit; for an Xray telescope it might include high and low energy bandpass and flux limits.

instrumentName (string)
instrumentType (one of set of known schema)
instrumentConfiguration (XML document)
We model the instrument as a combination of an "instrumentName" (eg Palomar 60-inch), and an instrument type (eg "opticalTelescope"). The instrument type implicitly refers to a schema for that type of instrument, a schema that has been agreed among domain experts. The instrument description can then be written as an instance of the relevant schema.

This is another occasion where a VO identifier could be used as an abbreviation of providing full instrument metadata. If a survey team always uses its instrument in the same configuration, it may be worth registering the metadata packet that describes the instrument, so that an identifier can be used in place of repeating the description for each VOEvent packet. In this case, the available information would be: instrument name, instrument type, and the identifier for the specific configuration.

A good possibility for the description of instrument configuration would be the RTML language (Remote Telescope Markup Language) -- see section 5.4.

3.4 Event Description

eventType (one of set of known schema)
eventDescription (XML document)
This is the "What" part of the definition of the event, and obviously depends significantly on the instrument, detection channel, and investigating team that took the data. Subscribers to an event service will expect to build queries based on this part of the event metadata -- an example from optical astronomy might be: when the B magnitude difference is greater than 1.0, or for a gravitational wave detector it might be: when the SNR for an inspiral event is greater than 3.0 in two detectors within 1.0 seconds. Note that the last is an example of a joint query: two events must be reported close together before the subscriber is alerted.

The VOEvent working group will expect instrument teams to cooperate with others to build these definitions of the event, keeping always to the stricture that VOEvent reports must be kept small.

One possibility for the Event Description schema is simple text, or perhaps text with HTML tags. While this type of description does not allow the machine to respond to the event intelligently, it is appropriate for a purely human-driven system.

3.5 Curation

A section of the VOEvent packet is devoted to curation information: who is responsible for the information content of the packet. We follow the IVOA Resource Metadata document [1] in defining curation.

Publisher (string)
An entity responsible for making the resource available. Examples of a Publisher include a person or an organization. Users of the resource should include Publisher in subsequent credits and acknowledgments. The Publisher may be either a text string or a VO identifier pointing to an organization.

Contributor (string)
An entity responsible for making contributions to the content of the resource. Examples of a Contributor include a person or an organization. Users of the resource should include all Contributors in subsequent credits and acknowledgments.

Contact (string, e-mail address)
The e-mail address for contacting the persons responsible for the resource. Contact is split into two components for clarity, Contact.Name and Contact.Email.

A date associated with an event in the life cycle of the resource. Typically, Date will be associated with the creation or availability (i.e., most recent release or version) of the resource. ISO8601 is the preferred format (YYYY-MM-DD).

3.6 Importance

Importance (double between 0 and 1)
Importance of an observation is highly subjective, and therefore difficult to interpret by another, yet at the same time it should be possible for the observing team to indicate importance by comparison with other observations made by the same team. Therefore we simply state that any creator of VOEvent packets should make the importance number however they wish; they may publish an explanation or not. Zero means unreliable, and unity means very important. However, it should be stated that an importance of zero is NOT the same as setting the PacketRole to TestPleaseIgnore.

3.7 Links

a (href=URL string)
It may be that there are images, spectra, time-series or other information that acts as supporting evidence for the event detection. These may be linked from the VOEvent packet by the usual HTTP mechanism <a href=http://...>.

4. Persistent Identifiers

ivoID (ivo identifier URI)
As noted in the Introduction, VOEvent packets may contain VO identifiers, as defined and discussed in [2]. They take the form ivo://authority/resourceKey, which are references to metadata packets that may be found at a VO registry or VOEvent database.

The lookup procedure is similar to looking up a URL on the world wide web: each registry controls a number of authority IDs. These are like domain names on the net: each is resolved to exactly one endpoint machine through a system of distributed knowledge. Once that machine is discovered, it should be able to resolve the secondary part of the identifier, the resourceKey. Indeed, the machine that holds an authority ID has made a promise to resolve all the resourceKeys that it has issued.

As work continues on the VOEvent structure, we will work with the IVOA to define a message protocol by which a packet may obtain a valid IVOA identifier from a database of such packets. Broadly speaking, the creator of the packet (client) would be submit to the server a packet with empty identifier, the server check syntax and respond with the same packet, but with the ivo:// identifier filled in.

5. Existing event systems

Several systems already exist for notification of immediate astrophysical effects. The earliest is the Central Bureau for Astronomical Telegrams of the International Astronomical Union (IAU); another is the Gamma-ray Burst Coordinate Network that is associated with satellites that detect GRBs. The Astronomer's Telegram system is more general, and has been successful for a number of years; the RealTime Virtual Observatory (RTVO) is building the database and messaging structures that will be necessary for robotic event-driven astronomy. Finally, we mention the Remote Telescope Markup Language (RTML), which is a sophisticated schema for defining the sequencing and state of an optical telescope.

5.1 Central Bureau of Astronomical Telegrams

The Central Bureau for Astronomical Telegrams (CBAT), under the auspices of the International Astronomical Union (IAU), is a nonprofit organization, with principal funding coming from subscriptions to the various services offered by the Bureau. The CBAT is responsible for the dissemination of information on transient astronomical events, via the IAU Circulars (IAUCs), a series of postcard-sized announcements issued at irregular intervals as necessary in both printed and electronic form. Messages announce new discoveries of supernovae, novae, comets, satellites of major/minor planets, and other interesting transient astronomical objects (particularly those that are unusual, such as cataclysmic variables that have outbursts less frequently than once every year or two, or very unusual variable stars or non-stellar objects

The schema for an IAU Telegram is:

In addition to this general information, each of several classes of event has a specific schema. For details, see the reference below. These classes of event are: [Comets, Supernovae, Novae, Outbursts Of Unusual Variable Stars, Features On Planetary Surfaces].

5.2 Gamma-ray Burst Coordinate Network

The GCN system has the capability to receive_from and distribute_to the GRB research community timely information on GRBs (called GCN Circulars). These reports are in the prose-style (as opposed to the highly formated "TOKEN: value"-style of the original GCN Notices). The process is simple. People can e-mail their reports to a central location and then those reports will be automatically distributed to a list of people. This list of Circular recipients is completely separate from the list of Notice recipients.

These GCN Observation Report Circulars will allow the GRB follow-up community to make optimum use of its limited resources (labor and telescope time) by communicating what has already been done or will soon be done. This facility is also fast (~1min) and cheap (0$) for the submitter.

The schema for a GCN packet is:

5.3 Astronomer's Telegram

The Astronomer's Telegram (ATEL) is for the reporting and commenting upon new astronomical observations of transient sources. In addition to the Telegram Index on the web, readers may request a Daily Email Digest (see Email Options ). Readers select those subject areas of interest, and Telegrams marked with those subject headings will be mailed to them after each 24-hr period (no email will be sent if no such Telegrams are received). Readers may also request the Instant Email Notices, used to report the discovery, with coordinates, of one of several different types of objects, as well as new outbursts of previously known transients. The Instant Email Notices also include a "Request for Observations", which provides alerts of observational opportunities in the coming 72 hours. The Instant Email Notices are sent immediately (within a few seconds) upon receipt. Reports submitted to The Astronomer's Telegram are not filtered or edited: the final editing on all submissions is made by the author. Submissions are posted onto the web instantly, by software.

The schema for an ATEL packet is:

5.4 Remote Telescope Markup Language

Remote Telescope Markup Language (RTML) is an XML-based protocol for the definition and exchange of telescope observing requests. The further development of RTML is being pushed by many wishes, constraints, and needs, particularly the capability: to support more complicated requests, instruments, and observing constraints; to support the "charging" of telescope time against user and external network accounts; to let the requests evolve as more information becomes available; and to use RTML documents to fully document the complete evolution of a request from the planning stage, the definition of the request in all its detail, the description of the actual observation, and all the way to the final delivery of data to the original client/user. The primary elements of an RTML description are: choice of telescope, the instrument that is mounted (eg camera, spectrograph), and detector; choice of exposure time; scheduling information; and target. There may also be information on how the retrieved information was processed (eg calibration, pipeline, etc).

We are evaluating the RTML schema as a candidate for describing telescopic observations in a general way, as the "instrument configuration" schema mentioned above. Other instruments might build custom schemas, for example the LIGO gravitational wave detector probably would not use RTML to describe instrument configuration.

5.5 RTVO.net

This new system is an attempt to make a prototype event database, featuring XML message format, subscription services, an XML database to hold the event descriptions, and IVOA identifiers for the events themselves.

6. References