(r10) VOSpace21RFC < IVOA

IVOA Web>WebPreferences>VOSpace21RFC (revision 10) (raw view)~~EditAttach~~
---++ VOSpace v2.1 Proposed Recommendation: Request for Comments

RFC page for the IVOA VOSpace 2.1 Proposed Recommendation.

The latest VOSpace 2.1 Proposed Recommendation, with updates from feedback on this page, can be found at:
   * [[%ATTACHURL%/VOSpace.pdf][VOSpace-2.1.pdf]]
The original 20170405 PR can be found here:
   * [[http://www.ivoa.net/documents/VOSpace/20170405/][http://www.ivoa.net/documents/VOSpace/20170405/]]

---++ Change Overview

The change notes since version 2.0:

From version 2.1-20170405:
   * Replaced the dated Custom Protocols section with a new Public Share Protocol section
   * Updates from working group feedback
From version 2.1-20150601:
   * Added REQUEST=redirect for further optimized pullFromVoSpace
   * Removed "search" and "find nodes" until use cases and implementations are presented
   * Changed pullToVoSpace, pushFromVoSpace to be client orchestrated.
   * Changed Transfer in XSD to accept parameters.
   * Recreated and expanded all webservice operation examples
   * Changed XSD element authType to SecurityMethod to match IVOA standard.
   * Fixed text inconsistencies
   * Clarified role of the node busy flag
   * Ported source to ivoatex format
From version 2.1-20140805:
   * Addition of optimized HTTP GET method of data transfer for pushToVoSpace, pullFromVoSpace
   * Addition of authType to Protocol in XML schema for transfer negotiation.
   * Added preliminary list of standard authType URIs
   * Deprecated view=data convenience method of data transfer
   * Corrections to minor XML format errors in the examples throughout the document.
An earlier VOSpace discussion page can be found here:
   * [[VOSpace21Spec][http://wiki.ivoa.net/twiki/bin/view/IVOA/VOSpace21Spec]]

---++ Comments from the IVOA Community during RFC period: 2017-04-06 - 2017-05-18

---++ Implementation and Validators

   * CADC and OATS-INAF implementations of VOSpace 2.1, both are based on the VOSpace source code from [[https://github.com/opencadc/vos][OpenCADC]].  One item remains to be fully 2.1 compliant:  only redirect on a synctrans parameterized request when the parameter "request=redirect" is provided.  This is described here:
[[https://github.com/opencadc/vos/issues/29][Issue: v2.1 Compliance - Obey request=redirect parameter]]

Please note your implementation or validator in this section.
---++++ Comments from Pierre Fernique

*Implementation references*
   * Presently, it seems that there is one VOSpace 2.1 server working (CANFAR) declared in the VO registry. MSSL, CASU and WFAU seems to be out of order. Is there already VOSpace 2.1 clients ?
*Abstract*
   * p2 - It is a little bit surprising to have directly in the abstract a note about the (un)compatibility with the previous releases of the standard. May be a dedicated section, just after the Abstract, should be more adapted.<br /><span style="background-color: transparent;"><br />=&gt; I agree. I have made the abstract more general and created a 'Previous Versions' subsection in the Introduction. -- IVOA.BrianMajor - 2017-04-12 <br /><br /></span>
   * More largelly, the fact that the compatibility between clients and servers of different versions is rarelly insured questions me. This choice has always the same big impact. The consequence is to force the servers and the clients to implement VOSpace 1.0, 2.0 and 2.1 the time that all potential clients (resp. servers) have upgraded their codes. This remark is not limited to VOSpace, we have the same challenge for SIAv2 which is a totally new standard compared to SIAv1. <br /><br />=&gt; This is true. In the case of VOSpace, I think the version with the most implementations will gain support in the clients. We can try to share and make widely available client implementations for use by many. -- IVOA.BrianMajor - 2017-04-12
*Introduction*
   * p5 - This section provides the analogy "<em>node = file</em>". Should we have to extend this analogy as "<em>node = file or directory</em>" ? This remark comes from our experience for implementing VOSpace client in Aladin. The role and functions on "directories" are not so clear in the previous release documents. And concretely, our experience shown us that the directory manipulations are generally not, or very lighted implemented. The situation could be improved, if we specify immediately that a node is potentially a directory (cf p11 - figure 2) <br /><br />=&gt; Good point. I have changed the text to say that a node can be thought of as a file _or_ a directory. -- IVOA.BrianMajor - 2017-04-12
*Typical use of a VOSpace service*
   * p5 - This subsection is important to understand the role of VOSpace, but I am wondering if it is a little bit too detailed as this level of the introduction (XML examples, ...)
*VOSpace identifiers*
   * p9 - The alternative '!' or '~' substitution characters should be justified. Why two substitutions characters ? Is it really required ? I suspect that it could be a source of potential bugs (cf. section 2.1 - identifier resolution). It seems that the response is provided p64 "<em>Note that the # sign in PROTOCOL is URL encoded as %23 and that the alternate separator &lsquo;~&rsquo; is used instead of &lsquo;!&rsquo; in the TARGET</em>". <br /><br />=&gt; You are correct in that the ~ was introduced so that it doesn't not have to be escaped in HTTP requests. I have added a 'Note' in the VOSpace Identifiers section describing this reasoning. As for whether it is required or not: I suspect that the original specification would have been better off using only ~ to begin with, but the ! character was preserved so as to not break existing implementations. Perhaps in version 3.0 we can deprecate the use of !. -- IVOA.BrianMajor - 2017-04-12
   * P10 - The end of this section cites the standardID of this specification. May be, a short explanation of what is a "standardID" would be welcome to avoid the possible reader confusion with the node identifier (same section). This standardID is also described in page 36. Maybe this last definition should be sufficiente. <br /><br />=&gt; To make it more clear on the first reference to standardID on P10, I have added a short sentence explaining how it is used in the VO Registry. -- IVOA.BrianMajor - 2017-04-12
*VOSpace data model*
   * p12 - "LinkNode" is described. But we do not know if we have to understand only dataLinkNode, or potentially ContainerLinkNode ? <br /><br />=&gt; There is this sentence: "The link target can be a URI that points to any type of resource, including other VOSpace Nodes (within the same VOSpace service or in another service), or external resources outside VOSpace altogether." I think this makes it clear that a link node can point to any type of node. -- IVOA.BrianMajor - 2017-04-12
   * p13 - The support of all type of nodes is not required (end of the section). But does it means that a VOSPace could be not supporting hierarchy (flat view only) ? <br /><br />=&gt; That is correct. If container nodes are not supported then a heirarchy of nodes is not supported. -- IVOA.BrianMajor - 2017-04-12
*Property identifiers*
   * p14 - URN is introduced here but not described. I suggest a end page note, or a RFC reference on URN. <br /><br />=&gt; I have added a reference to the RFC for URN Syntax in this section. -- IVOA.BrianMajor - 2017-04-12
*Property descriptions*
   * p15 & 16 "<em>Note that at the time of writing, the schema for registering PropertyDescriptions in the IVO registry has not been finalized</em>" - This remark is repeated at various places in the document, but concretely, there is no progress on this point since VOSpace 1.0 (2007). Maybe these remarks should be simply removed. <br /><br />=&gt; Good idea. I've removed the three instances of that remark. -- IVOA.BrianMajor - 2017-04-12
*Standard properties*
   * p16 - The standard properties identifiers are IVOID. But following the last Registry recommendation, any IVOID must be declared in the VO registry otherwise they are not valid. Actually, "<em>ivo://ivoa.net/vospace/core</em>" is not registered yet (or not accessible by the two active Full queryable resistries: euroVO and NAVO).
   * p17 - "<em>ivo://ivoa.net/vospace/core#<strong>group</strong>...</em>". This is the first (and the unique) mention of "group" notion. Surprisingly, there is no notion of "owner" (only creator, publisher and contributor which are probably not at the same level). It seems that this document does not described these notions (except briefly p36 "<em>Nodes are considered to be owned by the user who created them</em>"). May be, it could help the implementors to have a section describing these notions.
*Cababilities*
   * p19 - "<em>Note that at the time of writing, the schema for registering Capability-Descriptions in the IVO registry has not been finalized</em>" - same remark as p16 <br /><br />=&gt; Done. -- IVOA.BrianMajor - 2017-04-12
*View descriptions*
   * p24 - "<em>Note that at the time of writing...</em> " - same remark as p16 <br /><br />=&gt; Done. -- IVOA.BrianMajor - 2017-04-12
   * p25&26 - "<em>pullToVoSpace</em>" and "<em>pushToVoSpace</em>" are cited here, but never introduced before. May be a reference to the associated section would be welcome.
*Public share protocol*
   * p29 - A "<em>standard ID</em>" for "<em>public share protocol</em>" is provided in this section (new in VOSpace 2.1) =&gt; http://wiki.ivoa.net/twiki/bin/view/IVOA/VOSpacePublicShare. This ID is not compatible with regular IVOA standardID based on IVOID contrary to the other "standard protocols" defined in 3.5.3 (ex: _ivo://ivoa.net/vospace/core#httpget_). Is there a reason for this alternative nomenclature ? <br /><br />=&gt; Hmm... the idea behind the ID for the public share protocol was that the URI conveniently pointed to the page documenting the rules of the protocol. I personally like this concept as it encourages implementors to follow the pattern. However, I could be convinced to change it to the standard IVOID form. Additional thoughts from anyone? -- IVOA.BrianMajor - 2017-04-13
*Transfers*
   * p30 - In the XML example, there are security method identifiers for which the vocabulary seems to take various origins ( _ivo://ivoa.net/sso#tls-with-certificate_, _http://www.w3.org/Protocols/HTTP/1.0/spec/html#BasicAA_). Reference to associated documents would help. <br /><br />=&gt; I've added a reference to the SSO document so these securityMethods can be more clear. -- IVOA.BrianMajor - 2017-04-12
*REST binding*
   * p35 - "<em>These names are part of the VOSpace 2.1 standard</em>". It lets suppose that other elements in the document may be not relevant to VOSpace 2.1. I suggest to remove any reference to VOSpace 2.1 as this document is 2.1 specification (except for history list...) <br /><br />=&gt; Your comment made me realize that this section needed to be updated to match the recommendations of IVO Identifiers 2.0. There is now a new 2.1 synctrans endpoint that has the version number in the ID (not in path). I hope it is more clear now how the resource IDs work with the different versions of the specification. -- IVOA.BrianMajor - 2017-04-14
   * p35 - "<em>.auto</em>", "<em>.null</em>" are cited in this section, but without explanation. The description of their respective role should be appreciate. <br /><br />=&gt; Removed from this section. -- IVOA.BrianMajor - 2017-04-21
   * p36 - "<em>For backwards compatibility support, services SHALL include the securityMethod element</em>"... but as there is no real compatibility support (cf abstract), this sentence is a little bit surprising. <br /><br />=&gt; A 2.1 service _is_ backwards compatible with a 2.0 client, so I think this sentence is still important to state. -- IVOA.BrianMajor - 2017-04-21
*Create Node*
   * p41 - "<em>If the Node xs:type...</em>". This is the first citation of "<em>xs:type</em>" which seems to refer to node types.This notion should be defined somewhere before in the document, maybe page 11 in "<em>Nodes and node types</em>"<br /><br />=&gt; Good idea. I've added a short description of how to indentify node types in XML representations in "Nodes and node types" -- IVOA.BrianMajor - 2017-04-21
   * p43 - In the example, (and the following examples), there are 2 paragraphes:<em> Request:... Response:...</em> but the Response paragraph also contains the HTTP request part of the dialog. Strickly speaking the "Response" starts at the ligne "<em>HTTP/1.1 nnn...</em>", the lines above ("PUT....") are the "Request". I would suggest to change the "Response" title by "<em>HTTP dialog</em>" which seems to be more appropriate. <br /><br />=&gt; I think I agree with you, but before I go and make all those changes, does anyone else have an opinion on this? -- IVOA.BrianMajor - 2017-04-21
*Transfering data*
   * p55 - The difference between <em>pushToVoSpace </em>(sending data to a service) and <em>pullFromVoSpace </em>(reading data from a service) is not obvious. Same for <em>pullToVoSpace </em>(importing data into a service) and <em>pushFromVoSpace </em>(sending data from a service). I suppose that depends of the origin of the action, but it is just a speculation. As these 4 actions are a key point of VOSpace, small examples, or a small graphic would be appreciate. <br /><br />=&gt; Okay, I've added a bit more text for now on each of the four transfer protocols. I can add a graphics too if you think it needs more. -- IVOA.BrianMajor - 2017-04-21
*Change since last version*
   * p82 - We have the list of the changes from the previous "<em>version 2.1-20150601</em>". The full history is provided in appendix D. It is probably preferable to have all the history in the same place. <br /><br />=&gt; Okay, I've consolidated these two sections. -- IVOA.BrianMajor - 2017-04-12
   * p82 - a "<em>(?TBD)</em>" is still present <br /><br />=&gt; Thank you, removed. -- IVOA.BrianMajor - 2017-04-12
*Message schema*
   * p82 - "<em>VOSPace 2.0</em>" must be replaced by "<em>VOSpace 2.1</em>" in the introduction sentence. <br /><br />=&gt; Done. -- IVOA.BrianMajor - 2017-04-12
-- IVOA.PierreFernique - 2017-04-11
---++ Comments from TCG members during RFC period: 2017-04-06 - 2017-05-18

WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or do not approve of the Standard.

IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory.

---+++ TCG Chair & Vice Chair ( _Matthew Graham, Pat Dowler_ )

---+++ Applications Working Group ( _Pierre Fernique, Tom Donaldson_ )

---+++ Data Access Layer Working Group ( _François Bonnarel, Marco Molinaro_ )

---+++ Data Model Working Group ( _Mark Cresitello-Dittmar, Laurent Michel_ )

VOSpace is a rather complex standard I just discoverd by reading the paper. I appreciate the section 1.4 which is an indispensable guide for the reader

*Section 1.2*:

Starting a document with a typical use case is a great idea, but this section is a bit too much detailed. Something more concise withe less XML would be still more valuable<br />P6: Similarly when *a* user....., *they* will (plurial mismatch?)

*Section 3.1*

It is said here the VOSpace may interpret data. This a very imprortant feature to me which should be introduced before the data model section (1.2 e.g.). This is not a data model feature by the way.

*Thrown Exception*

It is said in some places(3.4.4#Default Import View e.g.) that the service can throw exceptions. I do not see what does it mean in term of interoperatbility. Throwing or not exceptions is rather an implementation issue.

<strong><br /></strong>

-- IVOA.LaurentMichel - 2017-05-05
---+++ Grid & Web Services Working Group ( _Brian Major, Giuliano Taffoni_ )

---+++ Registry Working Group ( _Markus Demleitner, Theresa Dower_ )

---+++ Semantics Working Group ( _Mireille Louys, Alberto Accomazzi_ )

---+++ Education Interest Group ( _Massimo Ramella, Sudhanshu Barway_ )

---+++ Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick_ )

---+++ Data Curation & Preservation Interest Group ( _Françoise Genova_ )

---+++ Operations Interest Group ( _Tom McGlynn, Mark Taylor_ )

---+++ Knowledge Discovery in Databases Interest Group ( _ Kai Polsterer_ )

---+++ Theory Interest Group ( _Carlos Rodrigo_ )

---+++ Standards and Processes Committee ( _Françoise Genova_)

<br /> <!--
      * Set ALLOWTOPICRENAME = IVOA.TWikiAdminGroup
-->
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
pdf	VOSpace.pdf	r5 r4 r3 r2 r1	manage	991.6 K	2017-04-21 - 22:42	BrianMajor
Topic revision: r10 - 2017-05-09 - BrianMajor
IVOA
Log in or Register
IVOA.net
Wiki Home
WebChanges
WebTopicList
WebStatistics
Twiki Meta & Help
IVOA
Know
Main
Sandbox
TWiki
TWiki intro
TWiki tutorial
User registration
Notify me
Working Groups
Interest Groups
Committees
Stds&Procs
www.ivoa.net
Documents
Events
Members
XML Schema