Difference: VOSpace21RFC (10 vs. 11)

Revision 112017-06-12 - MarkusDemleitner

 
META TOPICPARENT name="WebPreferences"

VOSpace v2.1 Proposed Recommendation: Request for Comments

RFC page for the IVOA VOSpace 2.1 Proposed Recommendation.

The latest VOSpace 2.1 Proposed Recommendation, with updates from feedback on this page, can be found at:

The original 20170405 PR can be found here:

Change Overview

The change notes since version 2.0:

From version 2.1-20170405:

  • Replaced the dated Custom Protocols section with a new Public Share Protocol section
  • Updates from working group feedback
From version 2.1-20150601:
  • Added REQUEST=redirect for further optimized pullFromVoSpace
  • Removed "search" and "find nodes" until use cases and implementations are presented
  • Changed pullToVoSpace, pushFromVoSpace to be client orchestrated.
  • Changed Transfer in XSD to accept parameters.
  • Recreated and expanded all webservice operation examples
  • Changed XSD element authType to SecurityMethod to match IVOA standard.
  • Fixed text inconsistencies
  • Clarified role of the node busy flag
  • Ported source to ivoatex format
From version 2.1-20140805:
  • Addition of optimized HTTP GET method of data transfer for pushToVoSpace, pullFromVoSpace
  • Addition of authType to Protocol in XML schema for transfer negotiation.
  • Added preliminary list of standard authType URIs
  • Deprecated view=data convenience method of data transfer
  • Corrections to minor XML format errors in the examples throughout the document.
An earlier VOSpace discussion page can be found here:

Comments from the IVOA Community during RFC period: 2017-04-06 - 2017-05-18

Implementation and Validators

Changed:
<
<
  • CADC and OATS-INAF implementations of VOSpace 2.1, both are based on the VOSpace source code from OpenCADC. One item remains to be fully 2.1 compliant: only redirect on a synctrans parameterized request when the parameter "request=redirect" is provided. This is described here:
>
>
  • CADC and OATS-INAF implementations of VOSpace 2.1, both are based on the VOSpace source code from OpenCADC. One item remains to be fully 2.1 compliant: only redirect on a synctrans parameterized request when the parameter "request=redirect" is provided. This is described here:
 Issue: v2.1 Compliance - Obey request=redirect parameter

Please note your implementation or validator in this section.

Added:
>
>
 

Comments from Pierre Fernique

Implementation references

  • Presently, it seems that there is one VOSpace 2.1 server working (CANFAR) declared in the VO registry. MSSL, CASU and WFAU seems to be out of order. Is there already VOSpace 2.1 clients ?
Abstract
  • p2 - It is a little bit surprising to have directly in the abstract a note about the (un)compatibility with the previous releases of the standard. May be a dedicated section, just after the Abstract, should be more adapted.

    => I agree. I have made the abstract more general and created a 'Previous Versions' subsection in the Introduction. -- BrianMajor - 2017-04-12

  • More largelly, the fact that the compatibility between clients and servers of different versions is rarelly insured questions me. This choice has always the same big impact. The consequence is to force the servers and the clients to implement VOSpace 1.0, 2.0 and 2.1 the time that all potential clients (resp. servers) have upgraded their codes. This remark is not limited to VOSpace, we have the same challenge for SIAv2 which is a totally new standard compared to SIAv1.

    => This is true. In the case of VOSpace, I think the version with the most implementations will gain support in the clients. We can try to share and make widely available client implementations for use by many. -- BrianMajor - 2017-04-12
Introduction
  • p5 - This section provides the analogy "node = file". Should we have to extend this analogy as "node = file or directory" ? This remark comes from our experience for implementing VOSpace client in Aladin. The role and functions on "directories" are not so clear in the previous release documents. And concretely, our experience shown us that the directory manipulations are generally not, or very lighted implemented. The situation could be improved, if we specify immediately that a node is potentially a directory (cf p11 - figure 2)

    => Good point. I have changed the text to say that a node can be thought of as a file or a directory. -- BrianMajor - 2017-04-12
Typical use of a VOSpace service
  • p5 - This subsection is important to understand the role of VOSpace, but I am wondering if it is a little bit too detailed as this level of the introduction (XML examples, ...)
VOSpace identifiers
  • p9 - The alternative '!' or '~' substitution characters should be justified. Why two substitutions characters ? Is it really required ? I suspect that it could be a source of potential bugs (cf. section 2.1 - identifier resolution). It seems that the response is provided p64 "Note that the # sign in PROTOCOL is URL encoded as %23 and that the alternate separator ‘~’ is used instead of ‘!’ in the TARGET".

    => You are correct in that the ~ was introduced so that it doesn't not have to be escaped in HTTP requests. I have added a 'Note' in the VOSpace Identifiers section describing this reasoning. As for whether it is required or not: I suspect that the original specification would have been better off using only ~ to begin with, but the ! character was preserved so as to not break existing implementations. Perhaps in version 3.0 we can deprecate the use of !. -- BrianMajor - 2017-04-12
  • P10 - The end of this section cites the standardID of this specification. May be, a short explanation of what is a "standardID" would be welcome to avoid the possible reader confusion with the node identifier (same section). This standardID is also described in page 36. Maybe this last definition should be sufficiente.

    => To make it more clear on the first reference to standardID on P10, I have added a short sentence explaining how it is used in the VO Registry. -- BrianMajor - 2017-04-12
VOSpace data model
  • p12 - "LinkNode" is described. But we do not know if we have to understand only dataLinkNode, or potentially ContainerLinkNode ?

    => There is this sentence: "The link target can be a URI that points to any type of resource, including other VOSpace Nodes (within the same VOSpace service or in another service), or external resources outside VOSpace altogether." I think this makes it clear that a link node can point to any type of node. -- BrianMajor - 2017-04-12
  • p13 - The support of all type of nodes is not required (end of the section). But does it means that a VOSPace could be not supporting hierarchy (flat view only) ?

    => That is correct. If container nodes are not supported then a heirarchy of nodes is not supported. -- BrianMajor - 2017-04-12
Property identifiers
  • p14 - URN is introduced here but not described. I suggest a end page note, or a RFC reference on URN.

    => I have added a reference to the RFC for URN Syntax in this section. -- BrianMajor - 2017-04-12
Property descriptions
  • p15 & 16 "Note that at the time of writing, the schema for registering PropertyDescriptions in the IVO registry has not been finalized" - This remark is repeated at various places in the document, but concretely, there is no progress on this point since VOSpace 1.0 (2007). Maybe these remarks should be simply removed.

    => Good idea. I've removed the three instances of that remark. -- BrianMajor - 2017-04-12
Standard properties
  • p16 - The standard properties identifiers are IVOID. But following the last Registry recommendation, any IVOID must be declared in the VO registry otherwise they are not valid. Actually, "ivo://ivoa.net/vospace/core" is not registered yet (or not accessible by the two active Full queryable resistries: euroVO and NAVO).
  • p17 - "ivo://ivoa.net/vospace/core#group...". This is the first (and the unique) mention of "group" notion. Surprisingly, there is no notion of "owner" (only creator, publisher and contributor which are probably not at the same level). It seems that this document does not described these notions (except briefly p36 "Nodes are considered to be owned by the user who created them"). May be, it could help the implementors to have a section describing these notions.
Cababilities
  • p19 - "Note that at the time of writing, the schema for registering Capability-Descriptions in the IVO registry has not been finalized" - same remark as p16

    => Done. -- BrianMajor - 2017-04-12
View descriptions
  • p24 - "Note that at the time of writing... " - same remark as p16

    => Done. -- BrianMajor - 2017-04-12
  • p25&26 - "pullToVoSpace" and "pushToVoSpace" are cited here, but never introduced before. May be a reference to the associated section would be welcome.
Public share protocol
  • p29 - A "standard ID" for "public share protocol" is provided in this section (new in VOSpace 2.1) => http://wiki.ivoa.net/twiki/bin/view/IVOA/VOSpacePublicShare. This ID is not compatible with regular IVOA standardID based on IVOID contrary to the other "standard protocols" defined in 3.5.3 (ex: ivo://ivoa.net/vospace/core#httpget). Is there a reason for this alternative nomenclature ?

    => Hmm... the idea behind the ID for the public share protocol was that the URI conveniently pointed to the page documenting the rules of the protocol. I personally like this concept as it encourages implementors to follow the pattern. However, I could be convinced to change it to the standard IVOID form. Additional thoughts from anyone? -- BrianMajor - 2017-04-13
Transfers
  • p30 - In the XML example, there are security method identifiers for which the vocabulary seems to take various origins ( ivo://ivoa.net/sso#tls-with-certificate, http://www.w3.org/Protocols/HTTP/1.0/spec/html#BasicAA). Reference to associated documents would help.

    => I've added a reference to the SSO document so these securityMethods can be more clear. -- BrianMajor - 2017-04-12
REST binding
  • p35 - "These names are part of the VOSpace 2.1 standard". It lets suppose that other elements in the document may be not relevant to VOSpace 2.1. I suggest to remove any reference to VOSpace 2.1 as this document is 2.1 specification (except for history list...)

    => Your comment made me realize that this section needed to be updated to match the recommendations of IVO Identifiers 2.0. There is now a new 2.1 synctrans endpoint that has the version number in the ID (not in path). I hope it is more clear now how the resource IDs work with the different versions of the specification. -- BrianMajor - 2017-04-14
  • p35 - ".auto", ".null" are cited in this section, but without explanation. The description of their respective role should be appreciate.

    => Removed from this section. -- BrianMajor - 2017-04-21
  • p36 - "For backwards compatibility support, services SHALL include the securityMethod element"... but as there is no real compatibility support (cf abstract), this sentence is a little bit surprising.

    => A 2.1 service is backwards compatible with a 2.0 client, so I think this sentence is still important to state. -- BrianMajor - 2017-04-21
Create Node
  • p41 - "If the Node xs:type...". This is the first citation of "xs:type" which seems to refer to node types.This notion should be defined somewhere before in the document, maybe page 11 in "Nodes and node types"

    => Good idea. I've added a short description of how to indentify node types in XML representations in "Nodes and node types" -- BrianMajor - 2017-04-21
  • p43 - In the example, (and the following examples), there are 2 paragraphes: Request:... Response:... but the Response paragraph also contains the HTTP request part of the dialog. Strickly speaking the "Response" starts at the ligne "HTTP/1.1 nnn...", the lines above ("PUT....") are the "Request". I would suggest to change the "Response" title by "HTTP dialog" which seems to be more appropriate.

    => I think I agree with you, but before I go and make all those changes, does anyone else have an opinion on this? -- BrianMajor - 2017-04-21
Transfering data
  • p55 - The difference between pushToVoSpace (sending data to a service) and pullFromVoSpace (reading data from a service) is not obvious. Same for pullToVoSpace (importing data into a service) and pushFromVoSpace (sending data from a service). I suppose that depends of the origin of the action, but it is just a speculation. As these 4 actions are a key point of VOSpace, small examples, or a small graphic would be appreciate.

    => Okay, I've added a bit more text for now on each of the four transfer protocols. I can add a graphics too if you think it needs more. -- BrianMajor - 2017-04-21
Change since last version
  • p82 - We have the list of the changes from the previous "version 2.1-20150601". The full history is provided in appendix D. It is probably preferable to have all the history in the same place.

    => Okay, I've consolidated these two sections. -- BrianMajor - 2017-04-12
  • p82 - a "(?TBD)" is still present

    => Thank you, removed. -- BrianMajor - 2017-04-12
Message schema
  • p82 - "VOSPace 2.0" must be replaced by "VOSpace 2.1" in the introduction sentence.

    => Done. -- BrianMajor - 2017-04-12
-- PierreFernique - 2017-04-11
Added:
>
>

Comments from Markus Demleitner

(1) The compliance matrix has links to findnodes and searches sections, which are commented out; hence the referencing fails. I think you should just remove the matrix lines.

(2) You're changing the target namespace of your schema; from what I can see in the spec, this should break 2.0 clients, as they will not find their elements any more. Are you sure yours isn't a classic case for the XML schema versioning note?

(3) I get a bit nervous about "uri: the vos:// identifier for the node, URI-encoded according to RFC3986 (Berners-Lee et al., 2005)." (p. 12) For one, I'm not sure if this is the right place to say anything about encoding, but more importantly: Are these URIs supposed to be comparable? If so, you should somewhere define how they're intended to be compared (e.g., authorities and schemes in general are case-insensitive -- are they here? What about normalising percent-encoded values?). Similarly for nodes and target, p. 13.

(4) "URI or URL" (e.g., sect. 3.2, p. 14): RFC3986 discourages a distinction between URI, URL, and URN, so I'd suggest to just write "URI" in these instances.

(5) The example in 3.3.5, "One use case for this would be a VOSpace 2.1 client talking to a service that implements both VOSpace 2.0 and VOSpace 2.1, where the client is acting on behalf of a third party agent that only understands VOSpace 2.0...", isn't quite convincing to me -- when you just change the minor version, shouldn't the 2.0 side be interoperable with any 2.1 side? What purpose does it then serve to discover a 2.0 service?

(6) p. 22, "uri: an optional boolean flag to indicate that the View preserves..." -- are you sure this is "uri"? If so, I think a few words on why you chose this name (it is a bit odd for a boolean flag, no?) was chosen might be helpful.

(7) Perhaps for the next version, I think having a single section on identifier rules that's then referenced from sections like 3.2.2, 3.3.2, 3.4.2 (just highlighting differences as necessary) will reduce repetition and make the document more readable.

(8) p. 26 "to the application/binary MIME type." After consulting my /etc/mime.types I don't think application/binary is defined. I think you mean application/octet-stream. I mention in passing that these guys aren't actually called MIME types but "media types". But since nobody calls them that I'll shut up about that particular point.

(9) p. 26 "the available views (the server) thinks is the most" -- why the parentheses?

(10) I'm a bit split on 3.5.4 -- on the one hand, this offloads what in effect is part of the spec to a wiki page, which I'd usually like to avoid, and the separation of concerns between what's specified on the wiki and what's in this main spec is unclear to me (what if we find contradictions later?). On the other hand, I see that implementors should be made aware of this protocol. Perhaps you can just have a section "Common external protocols defined so far", clarify their standard status ("If you provide the protocols with the following URIs, you should regard the referenced specifications as if they were part of this specification. On the other hand, incompatible changes to protocols listed here are forbidden by this specification.") and then just enumerate ("self-documenting") URLs?

(11) sect. 3.6.2 "limited to the UWS synchronous transfer endpoint" -- is there such a thing? I'd say the "UWS" needs to go from that statement.

(12) In several places, there still are hardcoded section references in the document (search for "section [0-9]"; example: "section 5.5" on p. 66, which I'm pretty sure doesn't point to where it should any more). They need be replaced by label-ref pairs (I'd have done that myself, but I don't understand the document well enough to confidently do that).

(13) With my hat as ivoatex author on, I remark that my recommendation is to keep physical lines shorter than 80 characters if at all possible; it makes for more informative diffs without further tooling.

-- MarkusDemleitner - 2017-06-12

 

Comments from TCG members during RFC period: 2017-04-06 - 2017-05-18

WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or do not approve of the Standard.

IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory.

TCG Chair & Vice Chair ( _Matthew Graham, Pat Dowler )

Applications Working Group ( _Pierre Fernique, Tom Donaldson )

Data Access Layer Working Group ( François Bonnarel, Marco Molinaro )

Data Model Working Group ( _Mark Cresitello-Dittmar, Laurent Michel )

VOSpace is a rather complex standard I just discoverd by reading the paper. I appreciate the section 1.4 which is an indispensable guide for the reader

Section 1.2:

Starting a document with a typical use case is a great idea, but this section is a bit too much detailed. Something more concise withe less XML would be still more valuable
P6: Similarly when a user....., they will (plurial mismatch?)

Section 3.1

It is said here the VOSpace may interpret data. This a very imprortant feature to me which should be introduced before the data model section (1.2 e.g.). This is not a data model feature by the way.

Thrown Exception

It is said in some places(3.4.4#Default Import View e.g.) that the service can throw exceptions. I do not see what does it mean in term of interoperatbility. Throwing or not exceptions is rather an implementation issue.


-- LaurentMichel - 2017-05-05

Grid & Web Services Working Group ( Brian Major, Giuliano Taffoni )

Registry Working Group ( _Markus Demleitner, Theresa Dower )

Semantics Working Group ( _Mireille Louys, Alberto Accomazzi )

Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )

Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick )

Data Curation & Preservation Interest Group ( Françoise Genova )

Operations Interest Group ( _Tom McGlynn, Mark Taylor )

Knowledge Discovery in Databases Interest Group ( _ Kai Polsterer_ )

Theory Interest Group ( _Carlos Rodrigo )

Standards and Processes Committee ( Françoise Genova) 


<--  
-->

META FILEATTACHMENT attachment="VOSpace.pdf" attr="" comment="" date="1492814535" name="VOSpace.pdf" path="VOSpace.pdf" size="1015442" user="BrianMajor" version="5"
View topic | History: r41 < r40 < r39 < r38 | More topic actions...
 
This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright © 2008-2025 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback