VOEventV2RFC < IVOA

IVOA Web>VOEventV2RFC (2011-06-27, RobSeaman)

VOEvent v2.0 Proposed Recommendation: Request for Comments

This document will act as RFC center for the VOEvent V2.0 Proposed Recommendation:

Interoperable Implementations

Skyalert and Dakota are active projects that read and write VOEvents and have transitioned (or ready to transition) from VOEvent 1.1 to 2.0. A VOEvent2 library in Python is available at VOEventLib, and another in dotNet called Dakota. Many projects produce VOEvent: IAU CBAT, NASA GCN, AAVSO. It is expected that Astronomers Telegram will make the transition, also the gravitational wave alert system LVAlert will move to VOEvent once it is REC.

RFC Review Period: 8 Apr 2011 - 6 May 2011 (Completed)

TCG Review Period: 9 May 2011 - until concluded

Comments from the IVOA Community during RFC period

WilliamOMullane

1. The first thing which caught my eye was WHO in section 3. > /Identification of scientifically responsibel Author/

apart from the spelling error for Gaia of course this may not make sense. Hence also 3.2.1. This would more likely be a helpdesk or some such for Gaia not an individual.

2. in 3.4.2 Observatory_Location You should consider adding "L2 = within 150000 KM of Earth Sun Lagrange Point 2" or something. There are enough missions going out there even if you suggest space missions can construct their own complex location later.

-- WilliamOMullane - 2011-04-08

Reply:

1) This appears to reference an old version of the PR. The current wording is "The Author IVORN element contains the identifier of the organization responsible for making the VOEvent available." Spelling must have been previously corrected

2) There has been some discussion of this point on the VOEvent WG mailing list. Not obvious if any change is required to the v2.0 PR as long as the schema supports this feature.

-- RobSeaman - 2011-04-24

NormanGray

I looked at the new section 3.9 on Reference. I can see the change to requiring @meaning to be a full URI, but the new text might still be a little thin on detail, which could easily lead to variant implementations

In particular:

3.9.2 meaning — The nature of the document referenced (anyURI). This attribute is optional.

I'd have hoped to see a little more of an explanation of what this URI should be, beyond "It is anticipated that a Note will be written" about them.

3.9.3 mimetype — An optional MIME type [36] for the referenced document.

Similarly, this MIME type could be a variety of things, and since the MIME type given here could potentially differ from the MIME type of the document received, it'd be good to note which has priority (the MIME type declared by the document beinr received). It'd also be good to note what this is for, naly a hit rather than something you'd necessarily expect to work with.

I included some text for this section in my message of 2011 March 24 00:37:03 GMT. That may have been too prolix (and I admit a tendency to run to the formal in these contexts), but I thought a fair proportion of that text was at least useful.

We wouldn't want the VOEvent document to be full of legalese, but if it's too vague and suggestive, people will implement things based on what they guess the meaning to be, which could cause problems later.

(I originally posted this as a VOEvent list message, but it was more appropriate as an RFC comment, since it concentrates on language rather than technical content)

-- NormanGray -- 2011-05-04

Reply:

Expanded description in section 3.9.

-- RobSeaman - 2011-05-09

Applications Working Group (Tom Mcglynn, Mark Taylor)

This document looks mostly good to me (Mark), but there are some issues I'd like the authors to address.

Two major points:

Please provide details of the required two interoperable implementations and available validation tools, as per item 4 of section 2.1 of DocStd.
The VOEvent Schema is not part of this document. It is usual practice in IVOA standards which present and describe an XML Schema to include that schema as an Appendix. I don't know whether this practice is mandatory for IVOA standards (can Christophe advise?), but it seems like a good idea as it makes explicit what the schema text is for the declared version of the standard. The authors should either include the schema text as an Appendix, or explain why they prefer to omit it.

Some more minor items:

Abstract (and elsewhere): "...one or more of the who, what, where, when and how..." --- it looks to me like a Citations retraction element could appear on its own as the content of a VOEvent packet. If I've got that right, then should it read "... zero or more ..."?
Status of the document: "Click on the image below..." --- the single required format for IVOA standards track documents is PDF, so references to clicking should be removed (or perhaps work out how to get a PDF to do that, which is probably possible, but not implemented in the submitted PR PDF version).
Section 2.4: A reference is made to registry enhancements, proposed elsewhere, with the names VOEventStream and VOEventServer. I think these should be named instead VOEventStreamRegExt and VOEventServerRegExt (or similar) to follow standard practice for registry extension schemas. Perhaps somebody from the Reg WG can confirm.
Section 3.1.2: 'A "test"...' should read 'The value "test"...' for consistency with the rest of that list.
Section 3.3: Representation of tables --- This document has reinvented a lot of the content of the VOTable standard. I would welcome at least a comment in the VOEvent document comparing the VOEvent and VOTable representation of tabular data and metadata, and explaining why the decision was made to reinvent rather than re-use.
Section 3.3.1.5: details of floating point representation in the Param/Field dataType are referenced in the non-existent reference [666], this must be fixed. If you're looking for somewhere to pinch standards for floating point value representation from, you could try (with care) XML Schema or VOTable.
Explicit consideration should be given the possibility of null values described by Field or Param elements: are nulls permitted for float dataTypes? for int dataTypes? How must they be represented?
References: as well as the non-existent ref [666], the ref [31] in section 3.3.1.6 is incorrect, it should be [32]. I haven't checked all the other references; please can the authors do so.
Param and Field elements in several of the examples appear to be missing dataType attributes (cnts in sec 3.3.2, seeing, time, mag, magerr in sec 4). The examples should be corrected. Since this is clearly an easy mistake to make, consider making the dataType attribute mandatory rather than defaulting to string, so that it is at least possible at validation time to detect whether values typed as strings are so typed deliberately.
Section 3.6.3: It sounds like the Concept element may offer the possibility to use terms from more than one controlled vocabulary. Would it be a good idea to add an attribute defining which vocabulary is in use, to aid parsing? e.g.
```
   <Concept vocabulary="ucd1+">process.variation.burst;em.opt</Concept>
   
```
Section 3.7.1: in one place the attribute value "supersede" is written when it should be "supersedes". The permitted values of the cite attribute are easy to get wrong since they are not gramatically very consistent - "followup", "supercede", "retract" might be a better set.
Spelling: atructure, suppliediven, thos, infromation, replaement
Character processing: the section(?) symbol is represented wrong in the PDF, e.g. in the last sentence before the section 1.1 (it also looks wrong in my browser, but that may be my problem).

-- MarkTaylor - 28 Apr 2011

Reply:

1. Examples of Independent Implementations:see new content at the top of the this page.

2. The schema can go in the appendix if the TCG requires it.

1. Text changed to indicate zero occurences of any of these are allowed. However the meaning of such minimalist VOEvents is not defined.

2. Document no longer talks of clicking.

4. Fixed

5. This element is intended for a short and simple table, and re-uses the ideas and syntax of the IVOA VOTable, but simplified and streamlined: this is appropriate because complex tables can be written as full VOTable and linked from the VOEvent. Specifically, these simplifications are: no support for hierarchy of tables (RESOURCE); no internal references (FieldRef and ParamRef); no provision for binary data, only XML; table cells can only be string, float, or int, in place of the arrays of 12 possible types and extensions; no formatting information contained in the Table, nor domain of the data (VALUES); no referencing between cells; there is no INFO element.

6. and 7. VOEvent supports IEEE 754-2008 floats, including signed zero, signed zeros, infinities, and special "not a number" values (NaNs). That standard also covers integers. Note that a null string cannot be converted to a float or int and will raise an error. http://en.wikipedia.org/wiki/IEEE_754-2008

8. Corrected these references

9. Fixed

10. Following the Reference element on this, the content of the examples showing the Concept element has been changed to a URI syntax. Thus, stars.supernova.Ia becomes http://ivoat.ivoa.net/stars.supernova.Ia.

11. and 12. Corrected spelling

-- RoyWilliams - 09 May 2011

Registry Working Group (Gretchen Greene, Pierre Le Sidaner)

3. In agreement with the recommendation for the VOEventStreamRegExt and VOEventServerRegExt registry extension specification naming convention - would like to see this specification effort included in the TCG roadmap

-- GretchenGreene - 10 May 2011

We think VOEvent v2.0 is ready for prime time. In the absence of substantive disagreement with this assertion, the PR will move to TCG review on 6 May 2011. -- RobSeaman - 2011-04-24

TCG Review Period: 9 May 2011 - until concluded

TCG members should add any comments under their name.

TCG Chair & Vice Chair (Christophe Arviset, Séverin Gaudet)

Following recent discussions, in particular at the TCG meeting in Naples, it would be good to address the following points :

1. Update the IVOA Architecture diagram for VOEvent and ensure that the introductory text makes explicit reference to adne explain the interdependencies with all of the IVOA standards mentioned on the diagram

2. Clarify / update sections 2.4.1 and 2.4.2 wrt VOEvent*RegExt registry standards related to VOEvent (to be discussed in Naples sessions)

3. Make a more explicit reference to the VOEvent schema, probably within the introdution as well, and be making a clear independent referene (currently it is kind of hidden in reference 20

4. It would be very useful to have page number and a TOC

Given that these points are addressed adequately, I'm happy to approve VOEvent 2.0

-- ChristopheArviset on 2011-05-17

Thanks for having implemented these changes into the 20110613 version, hence I approve VOEvent 2.0

-- ChristopheArviset on 2011-06-17

Reply:

(1) Text added above the Architecture figue, explaining connections. It says "VOEvent is an IVOA standard, which means that it fits into a rich matrix of other IVOA standards: the image below shows where VOEvent fits into the broader IVOA architecture. VOEvents inherit much of the structure and semantics of VOTable, including the UCD scheme for semantics of quantities. VOEvent takes space-time coordinates from the STC, and it uses the URI semantics of the IVOA Vocabulary effort. IVOA Identifiers are used for events and their parent streams and servers, and both these latter will be described by IVOA Resource Metadata and stored in the registry. "

(2) The RegExt suffix has been added

(3) The VOEvent schema has been included as a section 7.

(4) Sorry about the page numbers. We wrote in HTML, so this is the responsibility of the browser or other pagination device.

-- RoyWilliams - 28 May 2011

IVOA Chair & Vice Chair (Paolo Padovani, Ajit Kembhavi)

Applications Working Group (Tom Mcglynn, Mark Taylor)

As noted to Rob in person at Naples, I do not consider that my points 6 and 7 have been adequately answered. I'm talking here about representation of numeric values in the text of the XML document, i.e. how you write them down as a sequence of Unicode characters. As far as I can see(?) IEEE754 does not address that question, it only talks about how you serialize them as bit sequences. You may think this is obvious, but misunderstandings about whether, for example, "1e3" and "1d3" and "+1.00000000000000000000E+03" are valid representations of one thousand can and do cause interoperability problems. Concerning null values: if VOEvent really wants to disallow null entries in tables, the Apps WG will not veto it, but I strongly suspect that potential users of the VOEvent standard will sometimes wish to include blank entries as part of Table data. If this is disallowed, it should at least be mentioned explicitly in the same place you talk about numeric representation.

My item 13 has apparently not been addressed. You could just replace the character in question with the text "Section" if it's proving hard to fix.

-- MarkTaylor - 25 May 2011 (on the 20110509 version)

Thanks Mark. I had a slide on open issues at the plenary and VOEvent sessions. I was going to make a conversation of it at the TCG meeting, but that exercise turned into something else. Your issues will be addressed in the next draft and the TCG will be able to review the changes. We appreciate your attentive reading of the document.

-- RobSeaman - 2011-05-25

Here are the proposed definitions, including defaults, copied from Python docs (cited).

For dataType=float, the value must contain a possibly signed decimal or floating point number, possibly embedded in whitespace; it may also be [+|-]nan or [+|-]inf. If the value cannot be parsed this way, for example null string, it may return zero or NaN, but no exception should be thrown.

For dataType=int, the value must contain a possibly signed decimal number, possibly embedded in whitespace. Conversion of floating point numbers to integers truncates (towards zero). If the value cannot be parsed this way, for example null string, it will return zero, and no exception should be thrown.

Also the section symbols are now the proper html code.

-- RoyWilliams - 28 May 2011

Thanks to the authors for addressing my comments in the 2011-06-13 version. Apps WG now approves this document. -- MarkTaylor - 20 Jun 2011

Data Access Layer Working Group (Patrick Dowler, Mike Fitzpatrick)

A few minor comments and suggested readability changes:

1) P. 1 Abstract, para 3: The sentence "Subscribers, human or machine, receive immediate...." should be removed since it described the behavior of a particular type of client or use-case (two sentences later it says the standard does not address design issues like this). Alternatively, "...might receive...." would mitigate the declarative tone.

2) P. 7 Sec 2.2: Last sentence, suggest "remain under discussion...." rather than "debate"

3) P. 10 Sec 3.3.2: Suggest moving the last paragraph discussing the uniqueness of names to its own section (3.3.4) at the end of Sec 3.3 This would put the discussion after the

element is introduced rather than in the middle of the section.

4) P. 12 Sec 3.4.1: Text says "should contain" the and for the coordinates, these are required by the schema so suggest "must contain".

5) P. 13 Sec 3.4.2: In first sentence, suggest "...being described was made" add a comment such as "(or will be valid in the case of predictive packets)". The start of Sec 3.4.1 uses the phrase "...for which that location is valid" which is a bit more general than the strictly past-tense "was made".

6) P. 13 Sec 3.4.2: It isn't obvious from the text that when using the 'id' attribute to identify the observatory, that it is the application's responsibility to resolve this name to lat/lon values when the elements aren't used. Additionally, this is a free-form string with no enumerated values to provide reliable coords.

7) P. 19 Sec 5: Use of red text in the diagram isn't described. Are these just 'concepts' contained in the element or actual XML elements/attrs? What is meant by 'equivalent information'?

8) P. 21 Sec 6: [34] ...."externalauthentication" --> "external authentication"

Otherwise, I quite happily approve.

-- MikeFitzpatrick - 17 Jun 2011

#1, 2 and 8 changes made. #3 text left in position since suggestion would move text a long distance in the document. #4-7 are appropriate for future version addressing STC issues (perhaps after a more general STC review).

-- RobSeaman - 2011-06-27

Data Model Working Group (Jesus Salgado, Omar Laurino)

The document is well-structured and descriptive and it has been proved as useful and usable by the community. I approve it but I have some minor comments/questions (most probably not really applicable to version 2.0):

1. As mentioned by other people, there are some general formatting issues that are ongoing to be changed and that would be useful: TOC, page numbers and Justify Alignment.

2. This should be checked by the Sematics group but, in section 3.3.2, UCD arith.rate;photon.count could be, probably, better defined ("stat.max;arith.rate;phot.count"?, "stat.max;phot.flux"?)

3. Section 3.4.1: I imagine that the next point has been discussed in the past but, is it not negative to have so many options of AstoCoordSystem? Although other coordinate systems could be acceptable, would it be better to recommend the use of a certain coordinate system so the conversions can be done at publisher level with the correct level of accuracy?

4. In section 3.6.1: Is it possible to provide a clue in the text on how to estimate the value of the importance flag for a provider e.g. something statistical and/or default values? In other case, it could happen that some providers publish the events always as 1 and some other(modest ones) with lower numbers.

5. In section 3.9.5, the tag meaning is pointing to a URI and the example says that "this might indicate that this is a finding chart rather than a quick-look" How is this done? What is the expected content of this URI to provide this information? In the ObsCore effort we have a discussion on possible Data product.type (section 3.3.2.1 of ObsCore spec currently under review) and it was decided to generate a closed list of possible values of a certain utype. I am interested on that as it is possible that you have solved the same problem in a different way the could be re-used for other specs.

Apart from these questions and as I said before, I give my approval.

-- JesusSalgado - 14 Jun 2011

We appreciate these comments, but agree that they are not directly pertinent to v2.0. #1 was addressed by reformatting. #2 received no comment from the Semantics group and is exemplary only. #3 is an issue for some future discussion of STC. #4 has been unspecified since v1.0 and remains so in v2.0. #5 received discussion in the WG and the current language reflects the consensus.

-- RobSeaman - 2011-06-27

Grid & Web Services Working Group (Matthew Graham, Paul Harrison)

We approve this document.

--IVOA.MatthewGraham - 25 May 2011

Registry Working Group (Gretchen Greene, Pierre Le Sidaner)

Approved document.

--IVOA.GretchenGreene - 19 June 2011

Semantics Working Group (Sebastien Derriere, Norman Gray)

Document approved, congratulations to the authors for the effort on clarity. I really appreciated the overall clarity, Rob/Roy's subtle references and the VOEvent2 in a nutshell summary.

Minor typos in the last document version: