IVOA Document Standards RFC (Version 1.2)

This document acted as RFC centre for the IVOA Document Standards 1.2 Proposed Recommendation, 03 March 2009. Review period: 12 May 2009 to 11 June 2009

The PR Document has been updated. after RFC, and taking into account discussion at the TCG telecon of 16 September 2009. The PR text can be downloaded here. It is open for final TCG comments.

Final TCG Review period: 05 October 2009 to 04 November 2009. The comments should be posted at the end of this document.

Following comments from the TCG, the following revisions have been made:

1. Standard document format is PDF. Unfortunately, with the production of this very document we have encountered some problems with the preservation of hyperlinks. Hopefully this is simply a matter of finding the right tools to convert documents.

2. The need for at least two implementations has been strengthened, with explicit TCG approval required to waive this requirement if there are extenuating circumstances. If the TCG does not agree to waive the requirement, the Executive can take all circumstances into consideration and then make a final decision. The sense the authors have tried to convey is that having interoperable implementations is the norm and to be expected.

3. TCG members unfamiliar with the technical details of a standard may simply note "no dependencies" in their reviews.

Following a suggestion by N. Gray, an appendix has been added to the 20091002 PR version, containing recommended text for document status, in the 20091008 PR version.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Discussion about any of the comments or responses should be conducted on the Standing Committee on Standards & Processes mailing list, stdproc@ivoa.net.

For reference, here is a link to the first RFC (Version 1.1)

The comments (M. Deimletner, M. Taylor) posted on the page open for comments on V1.2 WD are copied below.

---

Comments

Minimal implementation rules

I'm not sure if this is one of these "not again" topics, but I'd really like so see something on the implementation side. Let me dream for a moment: What if the document said something like "A PR describing protocols, formats or similar computer processible artefacts can only become REC if (a) at least two independent implementations [fantasy: one of them in a C-linkable language; this would almost immediately allow bindings for basically all major languages] are available and (b) a validation suite is provided that allows implementors to verify the conformity of their implementation [for what it's worth]".

As a matter of fact, I think the validation suite would be worth more than all words and probably even formal specifications, since the behaviour defined by it is behaviour clients can rely on.

Now, I admit that if something like this were to become part of the standards process, more diligence is needed as to the wording, and certain details would have to be worked out (if I had my way, I'd require the implementations to be published under a DFSG-free licence, for starters). But I'm convinced the VO would be more fun to work and more robust with if rules like these were adopted. -- MarkusDemleitner - 17 Feb 2009

Me too

I welcome the introduction since v1.0 of a reference to validation tools in the WD->PR promotion rules (section 2.1). However, the language is pretty pusillanimous:

"Preferably, the WG should be able to demonstrate two interoperable implementations of each feature, and validation tools should be available. If the chair of the WG believes that broader review is critical, the chair may advance the document to Proposed Recommendation even without adequate implementation experience. In this case the document status section should indicate why the chair promoted the document directly to PR."

I have two issues with this text.

1. The language here makes it sound like two interoperable implementations and the validation tools are very much an optional extra. I would like the document authors to justify why these should not be made a requirement, or at the very least a condition for which explicit exemption (perhaps by the TCG) is required.
2. The phrasing seems disingenuous to me. The document may be advanced to PR if the chair believes (and how this 'belief' is to be assessed is another matter) that broader review is critical. That sounds just about plausible, since PR means a requirement for feedback from other WGs. However, the requirement for two implementations and validation tools is not reiterated at the PR->REC boundary. So there's nothing in the DocStd document to say that a document can't go from WD to REC with inadequate implementation experience simply because the WG chair believes it needed broader review while it was still a WD. If that is really what's meant (and I hope it is not) then it ought to be made explicit; if not, the implementation requirements should be reiterated more strongly in section 2.2.

I prepared the above comments before reading Markus's contribution above. Emboldened by him, I'll state it a little more strongly: why on earth shouldn't two interoperating implementations plus comprehensive validation suite be an absolute requirement? In my experience, writing the implementation is a breeze compared to thrashing out the standard. Unless of course the standard is so written as to be difficult or impossible to implement. Writing and testing an implementation is absolutely the best way to find out whether this is the case or not.

-- MarkTaylor - 18 Feb 2009

Having slept on it I think maybe I should backtrack slightly - if there is a comprehensive validation suite, I think that a single implementation is probably sufficient. Two independently written bits of interoperating software is a good test of a standard - either two implementations or an implementation and a validation suite (which is a kind of implementation in any case). Having a validation suite is a massive help when it comes to ensuring that later implementations are correct.

-- MarkTaylor - 19 Feb 2009

Having a good validation suite is very valuable, but is no substitute for having at least two interoperable implementations as a standard is developed. The problem is that a single implementation and a corresponding validation suite may be written by the same folks, and the fact they they agree with each other tells us little, and is not a reliable test of whether the standard is sufficiently understandable and complete to be implemented by others.

-- DougTody - 12 May 2009

I only want to reinforce (in case more votes are needed) what Markus and Mark already clearly stated: we have to have a validation service (mandatory and not recommended, as instead stated in paragraph 2.4): it is necessary (though obviously not sufficient) to ensure smooth operations. The advantages, at all levels, are so obvious that I do not see how we can live (or survive) without it.

-- AlbertoMicol - 13 May 2009

I too would like to support the idea that documents do not become standards without publicly available reusable implementations. Without wanting to embarrass Mark, I would say that the SAMP example could be a role model of how it should be done - there were reusable libraries published well before the document went to recommendation - this has resulted in enthusiastic uptake of SAMP. I can think of other examples of IVOA standards that took years of arguing over the document and years after that we still do not have publicly available software that implements the standard.

The whole purpose of these IVOA standards is to write interoperable software, and for the software engineer a software library is worth much more than a document, which has been pointed out elsewhere on this page, is generally only a description - the formal definition of a standard that can be reused by engineers is more often in schema or implementation libraries. Whilst I recognise that the IVOA has documents as its main "product", I think that it is essential that it does ensure that reusable software has been produced before promoting its documents as recommendations.

-- PaulHarrison - 21 May 2009

Response to the above comments on implementations and validation services (30 June 2009)

The members of the Committee are very sympathetic to these views. The problem we see with making such implementations mandatory is really one of scope and authority of the IVOA. The IVOA has no funding to support such activities, and thus the IVOA has no basis for demanding such things. If we were in position to fund such work, that would be one thing, but were are not; we rely on the goodwill and resources of the VO projects. If we make interoperable implementations and a validator mandatory, how do we make sure that they get developed? Who validates the validator? We have seen already that even our most experienced software engineers have written validators with errors in them.

Our view is therefore that this is a matter of scope/authority for the IVOA itself, and that we cannot mandate things that we cannot support. The IVOA "court" has no jurisdiction. However, we can try to strengthen the language a bit more.

-- BobHanisch on behalf of the Committee

Versioning

The document now states that there is an integer increment in the version number in the case where subsequent versions are not backward compatible.

However, the matter of how this impacts (web) service standards still remains open and needs addressing. To recap, any new version/revision of a (web) service standard will have to have an integer increment since it will be incompatible with previous versions: service request/response messages use namespaces to identify the version, e.g.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope">
  <soapenv:Header>
  <soapenv:Body>
    <PushToVoSpace xmlns="http://www.net.ivoa/xml/VOSpaceContract-v1.0">
...

The namespace on the message body - http://www.net.ivoa/xml/VOSpaceContract-v1.0 - indicates that this is a message to a VOSpace-1.0 compliant service. The same message to a VOSpace 1.1 service would utilise the namespace http://www.net.ivoa/xml/VOSpaceContract-v1.1 instead. A VOSpace-1.0 client cannot be used with a VOSpace-1.1 service and vice versa. So by the new rule VOSpace-1.1 should actually be VOSpace-2.0. The same argument applies, however, to minor changes, e.g. VOSpace-1.13 to VOSpace-1.14.

This brings up another point - how should associated schema, WSDLs, etc., in fact anything with a namespace, be versioned? With the version number of the associated document? For example, the change from v1.13 to v1.14 of a standard just reflects a format change in the WSDL document but this is "new incompatible" version. Should all service standards just employ integer versioning?

I also think that the language about supplementary resources needs to be strengthened. For a web service, the WSDL and any accompanying schema formally define the service interface and contract - these are really not supplemental in any sense but the machine-readable equivalent of the human-readable specification that the document represents. Additional implementations in these contexts will employ different namespaces and so will be not be interoperable. I do not think it is necessary to include the full text of such accompaniments in the human spec but they must be referenced and authoritative versions deposited with the IVOA Document Coordinator. We already have locations for such things: http://www.ivoa.net/xml, http://www.ivoa.net/wsdl and http://www.ivoa.net/rdf and these should be referenced in this document.

-- MatthewGraham - 12 May 2009

Response (30 June 2009)

The Committee agrees that the version numbering scheme is challenging when dealing with namespaces and WSDL, and leads to the conclusion that when IVOA standards describe web services or have associated XML schemas, with namespaces that when changed, cause software to break, then these changes must both be accompanied by an increment to the integer part of the document and the associated "supplementary" files. This would not affect most of the standards documents, and should not present any real logistical difficulty, as there are a sufficient number of integers available to support any number of revisions.

-- BobHanisch on behalf of the Committee

Sect 1.1: It would be useful to include the boilerplate in this section. While it may vary slightly in practice, I imagine most folk will simply lift this from another random standard, rather than being creative.

Response (30 June 2009)

It is not clear to the Committee just what is meant by "boilerplate" in this context.

-- BobHanisch on behalf of the Committee

Sect 1.2: I don't think it intends to, but the example progression of file names appears to suggest that a document appears solely as a PDF when it becomes a REC, and the HTML disappears.

Response (30 June 2009)

The use of HTML for the progression of documents and PDF for the REC was only to reinforce the idea that REC's should be available as printable PDF also. We do not think a change is required here, unless we want to show both formats explicitly.

-- BobHanisch on behalf of the Committee

Sect 1.5: In the case of WSDL files, schemas and I think most RDF files, the `supplementary resources' are not really supplementary, but as normative as any document text (as others have said). It might even be the case that all of these resources are normative. The issue, then, is whether anything has to be said about their normativity, and if so, where. I'd think that it would be sufficient to avoid making any statement about the normativity of other resources, but require the document to list the resources in question (perhaps in a special section?) and state whether they're normative or not. Or is this going into too much detail?

Response (30 June 2009)

The sense of the word "supplementary" that is intended here is that the files are not in-line in the text. Referring to my dictionary, I find supplementary defined using language such as something added to complete a thing, a section added to a book to give further information, and a separate section devoted to a particular topic. There is nothing that suggests that the information is optional or non-normative.

-- BobHanisch on behalf of the Committee

On that topic, it might be worth while requiring that the document is explicit about which sections are normative and which are informative. The Vocabularies document has a section in the preamble about this.

Response (30 June 2009)

We note that use of the terms "normative" and "informative" is not required for IVOA standards, and that there is not universal agreement on whether they are all that helpful. Moreover, essentially everything in this document is normative, except perhaps for example document names, but those are clearly labeled as examples.

-- BobHanisch on behalf of the Committee

Section 2.1: I'm another one who would prefer a pair of implementations to be mandatory, and probably prefer that they were independent, too. It'd probably be necessary to have some discretion, to handle the cases where it's not obvious what counts as an 'implementation'. For example, I've taken it that the Vocabularies document has four implementations, in the four vocabularies, but (a) one could take a different view of that, and (b) all four were produced by the document authors.

-- NormanGray - 24 May 2009

Response (30 June 2009)

See above for response regarding implementations.

-- BobHanisch on behalf of the Committee

After reading the draft I have two comments....

First, I think it would be helpful to standardize the location of these Wiki discussions. These can be relevant even after the standard is finalized. I often find it difficult to find these discussions and occasionally fail to comment simply because by the time I work my way through the E-mail or web links to locate the RFC page I been distracted by some other task. We want to make commenting as easy as possible.

Response (30 June 2009)

This is somewhat out of scope for this document, though the Committee agrees.

-- BobHanisch on behalf of the Committee

More significantly, I would like to put in my voice in support of the view that building a validation service should be a mandatory element of the standards development. Personally I think that having validators will greatly speed up the adoption of the protocols. I think there could be a significant degree of flexibility here just as there is for the development of the qualifying implementations. In a some cases (e.g., the current standard) validation may be inappropriate and in general I think it should be the purview of the working group to decide what level of validation is good enough (subject to review by the IVOA more generally when the document as the document goes through the promotion process).

While I believe validators would actually speed adoption of standards, some fear that this requirement would slow things down and there is particular interest that TAP not be delayed. If TAP is the real issue here then perhaps the requirement for validators could be delayed for six months or perhaps generally standards that are in an RFC before the standard process is updated could be grandfathered against new requirements. Then we'd have additional incentive to get TAP done quickly!

-- TomMcGlynn 11 June 2009

Response (30 June 2009)

See above for response regarding validators.

-- BobHanisch on behalf of the Committee

---

Comments from the TCG during the normal RFC period (starting 12 May 2009)

Applications (Tom McGlynn, Mark Taylor)

There has been no official response from the Working group with regard to any of the issues raised, notably with respect to whether validaters should be mandatory. This aspect of the standard will have a major impact on the quality of the tools and services provided by the VO. [TAM]

Data Access Layer (Keith Noddle, Jesus Salgado)

Data Model (Mireille Louys, AnitaRichards)

Grid&Web Sevices (Matthew Graham, Paul Harrison)

Registry (Ray Plante, Aurelien Stebe)

Semantics (Sebastien Derriere, Norman Gray)

VOEvent (Rob Seaman, Alasdair Allan)

Approved.

VO Query Language (Pedro Osuna, Yuji Shirasaki)

VOTable (François Ochsenbein)

Standard and Processes (Francoise Genova)

Astro RG (Masatoshi Ohishi)

I believe that the interface problems and/or the compatibility issues should be solved during the WD phase, not during the PR phase. The WG chairman could consult such issues with the TCG, and the TCG chair may ask relevant WG chairs to solve such issues. In these cases not all TCG members are not requested to review the WD. During the PR phase TCG may confirm that such issues have been fixed.

Current review process may serve too heavy burden to the TCG members, resulting in the very slow responses. (MasatoshiOhishi)

Data Curation & Preservation (Bob Hanisch)

I approve.

Theory (Herve Wozniak, Claudio Gheller)

Approved.

TCG (ChristopheArviset, Severin Gaudet)

I approve the document.

---

Comments from the TCG during the final TCG review (starting 2 October 2009, ending 4 November 2009)

Applications (Tom McGlynn, Mark Taylor)

Approved.

Data Access Layer (Keith Noddle, Jesus Salgado)

Approved - with the hope that the matter of reference implementations and validators continues to be discussed.

Data Model (Mireille Louys, AnitaRichards)

I approve the document. MireilleLouys

Grid&Web Sevices (Matthew Graham, Paul Harrison)

I approve, although I note that the Document promotion process summary makes no mention of reference implementations. MatthewGraham

Registry (Ray Plante, Aurelien Stebe)

Semantics (Sebastien Derriere, Norman Gray)

I approve the document. Good job -- SebastienDerriere

VOEvent (Rob Seaman, Alasdair Allan)

Approved - RobSeaman

VO Query Language (Pedro Osuna, Yuji Shirasaki)

I approve this document

VOTable (François Ochsenbein)

Approved.

Standard and Processes (Francoise Genova)

Approved

Astro RG (Masatoshi Ohishi)

The point I raised during the RFC period has not yet been solved. As I thought the problem (the "Panama Canal" problem, according to Fabio) has become more serious, I can NOT say that I agree.

The management of interface problems/compatibility issues is indeed a serious concern for IVOA, and an answer should have been posted to your comment. However this is in a sense too critical to find an answer solely through the Strandards Document. It was discussed during this Interop meeting both at the Exec level and at the TCG level. Cf also the Standards documents and process document produced by B. Hanisch and discussed during the meeting. The problem is thus tackled at the most general level of IVOA rather that at the level of the Standards document itself. In agreement with Bob's proposals, it was decided that each document should include a general introduction defining its scope, and that the TCG will establish a top-down view of the VO infrastructure/architecture in which components, subsystems and functions are defined and their relationship are shown. The role and relationship of standards documents should be shown in this diagram. A specific meeting of the TCG will be organised early 2010 for this purpose. I hope that this is a step forward in taking into account your very legitimate concern. Françoise Genova, on behalf of the SCSP, 12 November 2009

The current charter of TGC clearly states that it could establish a focus meeting to resolve possible conflicts among relevant WGs (e.g., the TAP problem). Thus I believe that it should be explicitly described in the Document and Standard Rec that the TCG should hold focus meetings to resolve such conflicts. A WG chair could consult with the TCG chair, pointing out that there could be some conflicts.

I would note that the ToR of the TCG and the content of the Document Standard Rec are strongly releted to each other and an update of the ToR of the TCG should be made to reflect these.

However, I can approve this Recommendation under the condition that the Recommendation will be immediately updated by taking into account these issues by the December 2010 InterOp meeting. (March 4th, 2010)

Data Curation & Preservation (Bob Hanisch)

Approved.

Theory (Herve Wozniak, Claudio Gheller)

Approved.

TCG (ChristopheArviset, Severin Gaudet)

I approve the document.