Difference: VTPRFC (10 vs. 11)

Revision 112016-11-28 - FrancoisBonnarel

 
META TOPICPARENT name="WebPreferences"

VTP 2.0 Proposed Recommendation: Request for Comments

Public discussion page for the IVOA VTP 2.0 Proposed Recommendation.

The latest version of the VTP Specification can be found at:

Reference Interoperable Implementations

Comments from the IVOA Community and TCG members during RFC period: 2016-05-31 - 2016-07-16

Comments from TCG members during the TCG Review Period: 2016-05-31 - 2016-07-16

WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or do not approve of the Standard.

IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory.

TCG Chair & Vice Chair ( _Matthew Graham, Pat Dowler )

Applications Working Group ( _Pierre Fernique, Tom Donaldson )


I found the document clear, easy to read, with a lot of useful examples and diagrams.

Concerning the relation and possible problems with other VO protocols, I just saw one potential difficulty with the last version of the Registry Identifier document. In VTP, each author, each subscriber and each broker uses a unique IVOID for its identification. Thus all of them must be fully registered in the VO registry. This constraint can be reasonable for authors and brokers, but could be a barrier for subscribers which just want to receive VOevents, and not necessary register them in the VO registry.

If this registration step is not a problem for the potential VO-event participants, I have no objection to promote this document in IVOA Rec status

--

Now, I remove my App working group hat, and as I read the document and try to understand VTP, I have two questions concerning the protocol VTP itself:

A) About the robustness of VTP, what could be the consequences of a temporary out-of-order of a broker notably in case of 2 brokers links (similar to figure 1 page 3).

I assume 2 connected brokers A and B, each on them with their own subscribers and authors:
0) The broker A stops (failure, maintenance, or whatever...)
1) All its subscribers will start their re-connection loop with a delay geometrical law (1s, 2s, 4s, ...10mn...).
2) When the broker A is restarted, most of the subscribers will be not immediately reconnected
(geometrical law consequence)
3) But the broker A will immediately reconnect to broker B
4) The broker B will send all the backlog messages to broker A (stacked during the broker A out-of-order time)
5) The broker A will immediately forward these messages to its "already" reconnected subscribers...
6) Consequences : most of the subscribers of broker A will just miss the backlog messages from broker B

To avoid this bad effect, I imagine that the broker A should wait atleast the max reconnection delay time to be sure that all its subscribers are ready to receive before re-subscribing itself to broker B. If I'm right, it could be important to notice this point in the document, and thus, explicitly provide the max reconnection delay time (10mn ? 1h ? 1 day ?)


B) I have also two remarks which are related to the scalability of VTP:

1) The protocol is simple - good thing in VO context - but probably quite limited in term of number of subscribers per brokers due to the TCP connection keep alive constraint. This constraint can be difficult to insure for the brokers (the number of TCP connections is sytem kernel dependent), and also for network equipments such as firewalls or NAT equipements which have to manage individually each opened TCP connections. I imagine that we are speaking about a few subscribers per broker (a few dozens, not really more). May be this point could be specified in the document.

2) It seems quite easy to write an VTP author, but the broker and the subscriber could be more difficult to implement notably due the lack of an unique identifier for a given VO-event (§8 in the document). As said in the document, that forces them to memorize all messages to be able to bit-for-bit compare them (*) for avoiding duplication or cycling effects. The document specifies that this comparison could be realized by an hash key method. But the hash method is not perfect in the sens that 2 different messages can potentially have the same hash key (no injectivity), notably if the content of the messages are globally similar. If VTP is used for a few messages, no problem. If this protocol is used more intensively, time to time a message will be ignored as wrong duplication. At this opposite, if this comparison task is not correctly implemented on the broker level there is a non negligible risk to flood the VO-event network. That also implies that there is no possibility to extend the content of a message, for instance by the brokers for incrementing a step counter in order to reduce the potential risk of cycling messages, or to store the path used by a VO-event message.

-- PierreFernique - 2016-09-23

Data Access Layer Working Group ( François Bonnarel, Marco Molinaro )

Added:
>
>
This document is well written and clear enough for the reader. Some points should be clarified anyway:

- How far is the IVOID of each node necessary to the whole process and what will be the registration process for that ?

- Reference to new IVOA identifiers specification and SSO specification should be included.

- answers to previous remarks on recovery procedure will be useful

As soon as these points are answered we support the recommendation of this specification.

- Apart form that a demonstrator could be useful to check the behaviour of a network of nodes with peculiar attention to what happens in case of broker failure and recovery.

-- FrancoisBonnarel MarcoMolinaro - 2016-11-28

 

Data Model Working Group ( _Mark Cresitello-Dittmar, Laurent Michel )

Introduction: it may be worth to give an order of magnitude of the expected VOEvent traffic in large infrascrutures (LSST...)

Acknoledgements: A first name would be welcome: ...Robert Seaman (NOAO). [J.] Swinbank....

Section 3.3: A brief sentence setting the difference between "VOEvent messages" and "Transport messages" or saying what "Transport messages" are would be helpful.

Section 4.2 and 7.x: What happens if a node does not respond to a message?

Section 6.2: Requesting the subscribers to include an IVOID in their responses supposes that they are registered in the registry. Is that true? May the <Origin> element be empty? This point must be clarified.

Section 8: Contrary to Apps, I've no objection about using hash codes to identify messages by their content; hash values are very different even if just one single bit has been changed.

Although the scope of the document is the transport protocol, some clarifications about the connection between a VOEvent infrastructure and the registry would be nice. Here are some questions I had upon while reading the document: Should brokers declare their added-value behaviour in the registry? How can an author locate a broker? and a subscriber?

Hereafter the clarifications I'm suggesting, I approve the document.

-- LaurentMichel - 2016-09-26

Grid & Web Services Working Group ( Brian Major, Giuliano Taffoni )

Section 8, De-duplication: The method of determining if messages are equal seems awkward. One could use, for example, a UUID that is generated when the event is constructed to avoid any ambiguity. Because each broker may choose a different algorithm for equality, they cannot communicate with each other about an event. (Even if that requirement isn't in the specification now, it could be later.)

Even better would be if the brokers could operate in a stateless manner. I.e. Not having to remember which events have come by. Was there consideration in building the history of brokers visited into the event itself?

In section 9, I think it would be appropriate to reference the IVOA Single Sign-On Profile document for more information on how to use digital signatures.

I approve this document.

-- BrianMajor - 2016-10-03

Registry Working Group ( _Markus Demleitner, Theresa Dower )

Sect. 6.1f use the term IVORN, which was deemed hopelessly ambiguous during the creation of Identifiers 2.0 (which had attempted to properly define it). So, we've deprecated it -- just call it IVOA Identifier or IVOID and be done with it (*).

Also in 6.1, you say "should normally include a timezone". You've probably seen the discussions about timestamp fromats in the context of UWS and DALI -- perhaps you could prevent future inconsistencies with the rest of the VO by saying "This [time and date] should include a ``Z'' timezone indicator. If the timezone indicator is missing, clients should assume UTC. The use of other timezone indicators (and, hence, timezones) is severely discouraged." (and similar in the following sections).

Still in 6.1, the sample document references a non IVOA-namespace and schema location. I realize it's probably too late to change the namespace. Perhaps you could still comment on this and at least use the ivoa.net schema location (http://ivoa.net/xml/Transport-v1.1.xsd, I think)?

I'm not terribly happy about 9.2, as the authentication scheme proposed isn't mentioned in the SSO document that we're reviewing almost simulataneously. If this wasn't advertised as "finished tech just being codified", I'd say it's a bit embarrassing to, in effect, have conflicting recommendations (SSO: "Auth schemes approved are..."; VTP: "...something we don't particularly care about."). Things being what they are: if this is deployed and implemented, so be it (except that then I think you should say what XML signature scheme actually is used). If it's not, however, can't you just put in "to be specified in 2.1"? [All this leaves aside the question of what kind of security is achieved by the protocol proposed -- over a plain TCP connection with all its susceptability to eavesdropping and hijacking, that's probably limited in the first place].

Typos and similar

p. 4 "...set of brokers which subscribe/r/ to each other's" (*)

"The Transport Protocol, hereafter VTP" -- perhaps actually put a "VOEvent" in front of "Transport" so the acronym matches? (*)

Provided at least the points marked with (*) are somehow addressed, we approve.

Semantics Working Group ( _Mireille Louys, Alberto Accomazzi )

Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )

Time Domain Interest Group ( _John Swinbank, Dave Morris )

Data Curation & Preservation Interest Group ( Françoise Genova )

Operations Interest Group ( _Tom McGlynn, Mark Taylor )

This seems like a protocol for which a validator should be both straighforward and important. This seems particularly true for the broker role -- where the handling of message loops would be a particular concern -- but simple validation of the Author and Subscriber roles would be helpful too.

Outside of the OIG's particular concerns, this seems to be a clearly written, easily understood and well bounded protocol. However the need to stay continuously connected seems dangerous. If the connection is interrupted for whatever reason there seems to be no way to recover a missed alert. A protocol that explicitly supported (but did not require) brokers providing subscribers with the set of recent events immediately after connection would seem more robust. If a broker optionally allowed users to ask about any events seen in the past hour (or some other TBD and possibly broker dependent time) then a subscriber could potentially recover any events they might have missed. Might also help with the message loops issue.

-- TomMcGlynn - 2016-09-27

Knowledge Discovery Interest Group ( Kaï Polsterer )

Theory Interest Group ( _Carlos Rodrigo )

Standards and Processes Committee ( Françoise Genova)


<--  
-->
 
This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback