Applications Messaging Minutes

(Minutes from the Apps4 and Apps5 sessions of the InterOpMay2007 meeting)

Sessions 4 and 5 of the Applications Working Group in Beijing were dedicated to discussing the emerging Applications Messaging protocol, tentatively named Simple Applications Messaging Protocol (SAMP). Each point in the PlasticOnePointOh list was presented (mostly by JohnTaylor) and discussion was invited. A substantial degree of agreement was reached.

Points addressed

A summary of the points addressed and the consensus which was reached on them follows.

Refactor registration API

Accepted. The following calls can be made by an application to the hub to prepare for messaging:

1. Actual registration, including receiving identifying ID
2. Notification to hub of metadata (name, icon, description, ...)
3. Declaration of messages which the application wishes to receive

Steps (2) and (3) may be called multiple times, in any order, with different arguments if the application wishes to change metadata/supported messages. The hub should send an appropriate notification message to interested third-party applications notifying them when any of the above three things happens.

Generalise metadata

Accepted. Applications will notify the hub of metadata describing themselves using key-value pairs. Certain keys (such as application name) will be reserved and recommended, but the list will be extensible by applications so that custom metadata items can be stored.

Message IDs

Accepted. The use of an identifier for each REQUEST-type message is required to support asynchronous messaging, so that responses can be tied up with messages sent earlier. It may not be essential for NOTIFY-type messages, but it's probably harmless in that case and may be useful to support (probably optional) message logging facilities.

Change the bootstrap method

Rejected. The proposal to change the bootstrap method to some port-based method was rejected. For now a mechanism similar to the existing PLASTIC one which writes a protected (600) file in the user's home directory will continue to be used. However, the format of this file should be modified so that it becomes more language-neutral than at present (the existing one relies on parsing rules based on Java properties serialisation). This does not preclude the use of other bootstrapping methods in future evolution of the messaging system.

"Security"

Accepted. A simple mechanism which prevents application spoofing will be used. This is not intended to provide any kind of encryption or defense against all forms of attack; it is expected that if more stringent security requirements arise they will be best addressed by use of an underlying communications transport which provides such facilities.

Anonymous Access

Rejected. There was a suggestion that it should be possible for applications to make Requests or Notifications to the hub without having previously registered. The motivation for this was to do with making it easy for simple applications (e.g. user-written scripts) to send one-shot messages. However, since registration is straightforward there doesn't seem to be any real requirement for this.

Drop Synchronous Messaging

Hybrid tentatively accepted. Outline hub and application APIs were proposed which combined synchronous and asynchronous messaging. Communication between the hub and the message recipient is always asynchronous. Communication between the sender and the hub is basically asynchronous, but the hub also provides a utility method which makes it appear synchronous. Notify (send and forget) type messages are also provided. This provides the best of both worlds: timeouts can be avoided, but applications which find it difficult to implement asynchronous messaging (for instance because they don't wish to run a server) can still receive message responses. Since this had not been presented in detail before the session, further consideration/refinement of the ideas may be in order.

Choice of Transport

It was agreed that Java RMI should be dropped, since it effectively requires hub implementations to be written in Java.

It was agreed that for the immediate future an implementation of SAMP using a single transport (see below) should be defined, documented and endorsed by the Apps WG. Of course anyone may write code to perform messaging over a different transport based on the ideas in SAMP, but this shall not be considered a SAMP implementation unless explicitly endorsed by the Apps WG. The Apps WG will consider such endorsements if there is some compelling reason why a new transport is required (presumably because the initial one is inadequate in some way), but will not endorse new implementations willy nilly. The thinking behind this is that proliferation of transports, other things being equal, acts as an impediment to interoperability.

The most obvious choice for the initial transport is XmlRpc, since PLASTIC already uses this as its language-neutral transport. However the suggestion to use JsonRpc or some other JSON-based transport instead was raised, since it is in some respects simpler (e.g. does not require any XML parsing, can work using HTTP GET). Further opinions on the best choice here (including choices not mentioned above) would be welcome on the mailing list.

mtypes

Accepted. There was some discussion of the pros and cons of using opaque URIs vs parsable hierarchical mtype strings (along the lines of UCDs) to identify the semantics of a message. It was agreed to move forward with mtypes. In order that the task of defining a list or hierarchy of mtypes does not slow down development of an initial SAMP standard and implementation it was agreed the priority in the first instance should be to come up with a list of mtypes which corresponds to the messages used in the existing vocabulary of PLASTIC messages.

Pass message parameters by name

Accepted.

Remove message parameter typing

Deferred. Fixing the type of parameter values passing through the messaging system does not provide the benefits that it does in a strongly-typed language (the hub cannot perform validatation since it does not know what type the parameters of each message ought to be). It also causes problems for untyped languages such as Perl, Python(?) and Tcl. However if typing is removed it becomes necessary to define exactly how, e.g. integers and floating point numbers are encoded and decoded into strings. There are also issues to do with encoding/decoding binary data, strings containing funny characters, lists and maps. No consensus emerged on this; a decision will be deferred until later in the process of specifying SAMP.

Annotations

Accepted with dissent The lengthy discussion on the mailing list concerning the usage and usefulness of annotating messages was reprised. Although agreement was not unanimous, a show of hands indicated a majority for including the possibility to annotate messages, using some yet-to-be-decided syntax. The point was made that for many types of message annotation was not necessary or desirable.

Next steps

These agreements, built on the ideas already instantiated in PLASTIC, should form the outline for a first version of SAMP. The points above need not however be absolutely binding; they can be reconsidered if further thought indicates that a better decision can be made.

A document describing SAMP will be written. It will comprise the following sections:

Architecture

Form of a message, language-neutral outline of hub and application APIs, ...)

Transport-specific implementation

Details of how applications can participate in messaging including bootstrap file format, actual hub and application APIs, .... Based on XML-RPC or possibly some other transport (see Transports discussion above)

mtypes

Initial description of how the mtype vocabulary is organised syntactically and semantically. To include at least mtypes which correspond to existing PLASTIC messages.

Volunteers to draft these sections are JohnTaylor, MarkTaylor and MikeFitzpatrick respectively. The drafts will be reviewed by a subgroup (mailing list) of the Apps WG with organisation and membership TBD.

The intention is to move this forward quickly. Before the Cambridge Interop (Sept2007) we hope to have a document in WD and working code, meaning both hubs and applications which communicate using the new protocol. In order not to disrupt usage of PLASTICised tools in the community, it will be necessary for some hubs to support both the existing PLASTIC protocol and the new SAMP protocol, and to translate between them, for some transitional period. This will allow communication between a mix of applications using the old PLASTIC and the new SAMP while applications take up the new standard.

Several participants indicated that they expected to implement the new messaging standard in applications when it was defined. Some volunteered to provide hub implementations in various languages (MarkTaylor: Java; JohnTaylor: Python?; MikeFitzpatrick: C?).

I guess I'll stick my hand in the air and volunteer to move my internal Perl hub (which exists sans-RMI for obvious reasons) over to the new standard -- AlasdairAllan - 30 May 2007

-- MarkTaylor - 23 May 2007