VOspace10Spec21Archive < IVOA

IVOA Web>IvoaGridAndWebServices>VOSpace10Spec>VOspace10Spec21Archive (2006-07-19, PaulHarrison) (raw view)
%TOC%
---+ Archive of Change requests for V0.21 of VOSpace 1.0 draft
---++ Changes
---+++ Mandate and define at least one transport protocol
without this a compliant VOSpace will not be able to transfer data to another compliant VOSpace.

I recommend http be mandatory.

see detailed discussion in email thread http://www.ivoa.net/forum/vospace/0606/0097.htm

-- IVOA.PaulHarrison

   * Which version, http-1.0 or http-1.1 ?
   * Which methods http-get, http-put or both ?
      * For a space that stores public images, then http-1.1-get makes sense.
      * For a space that allows upload to sensitive database tables, then http-1.x-put does not have sufficient authentication.

-- IVOA.DaveMorris - 16 Jun 2006

I think that where VOSpace is acting as a http server then it would be good to mandate http-1.1 compliance - it does fix issues with 1.0 and afterall it is a 7 year old specification now....

As far as a compliance statement goes - how about 

(http-1.1-get or https-1.1-get) and (http-1.1-put or https-1.1-put)

-- IVOA.PaulHarrison - 19 Jun 2006

Making some form of HTTP mandatory makes it easier to write clients and harder to write services. VOSpace fails unless we get useful services, and services are already harder to write than clients.

-- IVOA.GuyRixon - 03 Jul 2006

we are already implicitly mandating http for the "control" interface - i.e. the web service - as far as implementors are concerned adding http-get would be not a great burden for the implementor - without a mandatory transport the success of the use case where a client sets up direct space to space transfers is not guaranteed, so the fallback would have to be copying the data to the client which understands the transport of each space and then transferring to the second space.

-- IVOA.PaulHarrison - 04 Jul 2006

---++++ Votes
| *name* | *vote*| *comment* |
| IVOA.PaulHarrison | +1 ||
| IVOA.DaveMorris | -1 ||
| IVOA.GuyRixon | -1 ||

_*result*_: not done

---+++ Specify as optional a small list of well known transport protocols
so that at least implementations will do the same thing for common protocols - this list to include
   * ftp
   * gridftp
   * file - direct access to the file system

-- IVOA.PaulHarrison

Agree with the list of common protocols.

Defined in a annex to the main specification, including a standard URI and details of the protocol specification (can be just short note and a reference to the external specification).

e.g.
<verbatim>
HTTP 1.1 get
URI
    ivo://....vospace/protocols/http-1.1-get
Description :
    Get data using the HTTP-1.1 GET method as defined in RFC2616.
    http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.3
</verbatim>


-- IVOA.DaveMorris - 16 Jun 2006

---++++ Votes
| *name* | *vote*| *comment* |
| IVOA.PaulHarrison | +1 ||
| IVOA.DaveMorris | +1 ||

_*Result*_: specified as registry entries


---+++ Specify the key names and meanings for a small group of "essential" property keys
The minimum set which I would say we need mandatory names are
   * vos.Owner
   * vos.ModificationDate
   * vos.Size
   * vos.MimeType

-- IVOA.PaulHarrison

Agree with the list of common properties.

Defined in a annex to the main specification, including a standard URI and details of what each property means and how it is represented.

e.g.
<verbatim>
Data created date
KEY
    vos.data.created.date
Description :
    A read-only property generated by the server.
    Indicating when the data contents were originally created.
    Formatted as a ISO 8601 date-time [yyyy-mm-ddTHH:MM:SS.SSS]
----
Data modified date
KEY
    vos.data.modified.date
Description :
    A read-only property generated by the server.
    Indicating when the data contents were last modified.
    Formatted as a ISO 8601 date-time [yyyy-mm-ddTHH:MM:SS.SSS]
</verbatim>

Note - As VOSpace-1.0 does not support append, the only way the created and modified dates will be different is if the server modifies the data underneath.

-- IVOA.DaveMorris - 16 Jun 2006

---++++ Votes
| *name* | *vote*| *comment* |
| IVOA.PaulHarrison | +1 ||
| IVOA.DaveMorris | +1 ||

_*Result*_: specified as registry entries


---+++ Clarify the use of the Format parameter in some calls
I believe that the original intention of Format was to specify a possible transformation of the data (mainly on export of table data from RDBMS based stores). In the current version of the document it reads as if this parameter is merely a description of the data - in which case mime-type suffices. In addition I am not sure if this parameter has any meaning for import operations.

---++++ Votes
| *name* | *vote*| *comment* |
| IVOA.PaulHarrison | +1 ||


_*Result*_: Format concept is now represented by the view concept

---+++ Change "exception" to "fault"
The document uses the term exception for the "faults" rather than "fault"
-- IVOA.PaulHarrison

---++++ Votes
| *name* | *vote*| *comment* |
| IVOA.PaulHarrison | +1 ||
| IVOA.DaveMorris | -1 | I can't see a significant benefit from changing this|

_*Result*_: not done

---+++ Consider adding an optional identifier list to the parameters for ListNodes
This would allow the client to specifly a subset of the VOSpace to be listed - in effect the behaviour would be similar to the "ls" command in unix. 

Reason for change: improved efficiency - if ListNodes always has to list the _whole_ VOSpace then it is a pretty blunt instrument, especially as the number of data objects in the space increases.

---++++ Use Case
Suppose that there is a 1.0 VOSpace containing 5000 data items and a client is currently interacting with a V2.0 VOSpace that has links to a small subset (e.g. 100 data items) - the client needs to make 100 getNodeProperties SOAP calls to the V1.0 space to get the latest metadata about the data objects - with an optional identifier list it can make one call to ListNodes.

-- IVOA.PaulHarrison 19 Jun 2006

---++++ Votes
| *name* | *vote*| *comment* |
| IVOA.PaulHarrison | +1 |proposer|
| IVOA.DaveMorris | -1 | I can't see a significant benefit from changing this|

_*Result*_: done

---+++ Reconsider getNodeProperties and setNodeProperties
the naming of these operations makes them appear as a pair, where in fact they are not - you get a Node out of the getNodeProperties and have to put a PropertyPairList into the setNodeProperties - If the change to ListNodes were made as above, then getNodeProperties would be redundant anyway.
-- IVOA.PaulHarrison

---++++ Votes
| *name* | *vote*| *comment* |
| IVOA.PaulHarrison | +1 |the benefit is improved clarity - in addition there is the slightly irritating issue (see below) that you cannot take the property list from the "get" and use it in the "set" because of read-only properties|
| IVOA.DaveMorris | -1 | I can't see a significant benefit from changing this|

_*Result*_: done

---+++ Semantics not clear for setNodeProperties
It is not clear
   * how to delete a property - does a null value of the property pair denote this?
   * does the whole set of node properties given as an argument replace the whole set for the node, or is a union operation performed.
   * interaction with the "read-only" properties in these scenarios...
-- IVOA.PaulHarrison

Agree, need to make this clearer.
   * Null value deletes property.
   * Operation is union not replace.
   * Server throws PermissionDenined for read-only properties.
      * what if only one of the list of properties is read-only? - need to signal which are the bad properties in the exception - also PermissionDenied could be confusing as this normally refers to the permissions on the data object which could potentially be changed in future versions of VOSpace - certain properties are fundamentally readOnly and so would 

-- IVOA.DaveMorris - 16 Jun 2006

---++++ Votes
| *name* | *vote*| *comment* |
| IVOA.PaulHarrison | +1 ||
| IVOA.DaveMorris | +1 ||

_*Result*_: done

---+++ Change names of parameters for moveNode and copyNode
Target is a confusing name for the "source" part of a move operation it sounds more like "destination" - recommend use "source" and "destination"
-- IVOA.PaulHarrison

Yep, source and destination are probably better. As long as we make it clear that these are internal locations, not references to external locations.

-- IVOA.DaveMorris - 16 Jun 2006

---++++ Votes
| *name* | *vote*| *comment* |
| IVOA.PaulHarrison | +1 ||
| IVOA.DaveMorris | +1 ||
| IVOA.GuyRixon | +1 ||

_*Result*_: done

---+++ Rename bulk data transfer operations
   * I stilll find the data transfer operations confusing- I think that the basic problem is that the push and pull verbs are opposite in meaning *and* viewed from opposite perspectives. I have a new set of proposals

| old name | new name"|
| pushDataToVoSpace | importDataClientPush|
| pullDataToVoSpace  | importDataServerPull|
| pushDataFromVoSpace | exportDataServerPush|
| pullDataFromVoSpace | exportDataClientPull|

which I think are better because
   1 the overall objective of the operation is the first verb
   1 the active party in the transfer is identified
   1 the direction of the transfer is then related to the active party.

-- IVOA.PaulHarrison - 13 Jun 2006

This is a change to the specification not the WSDL, move this to the specification change page.

-- IVOA.DaveMorris - 16 Jun 2006

I find the newly-proposed names _more_ confusing. It's not obvious to me whether "import" moves data into VOSpace or into the client of VOSpace; similarly  with "export".

-- IVOA.GuyRixon - 04 Jul 2006

---++++ Votes
| *name* | *vote*| *comment* |
| IVOA.PaulHarrison | +1 ||
| IVOA.MatthewGraham | 0 ||
| IVOA.DaveMorris | -1 | Change the specification document first |
| IVOA.GuyRixon | -1 | |

_*Result*_: not done

---+++ New <noop>GetPropertyKeys operation
This proposal is
basically because I am still a little worried about interoperability
problems with the completely untyped nature of the property-key pairs
- particularly as they are expected to carry some fundamental
metadata about the data objects in the current implementation. This
call would return the complete list of key names that have been used
in the VOSpace, which would then allow clients to attempt to be
consistent in the use of key names - it is not much but at least it
does provide a mechanism to voluntarily avoid complete anarchy.

Should return a list of the keys with an indication of which ones relate to  read-only properties.

-- IVOA.PaulHarrison - 13 Jun 2006

I agree with adding the method.
However, this should be changed in the specification first, not the WSDL. Move this to the specification change page.

-- IVOA.DaveMorris - 16 Jun 2006

Suggestion 1: this operation only returns the set of properties
understood by the service. Collating the other properties doesn't seem useful and might be expensive to implement.

Suggestion 2: for the properties of interest, return metadata roughly
equivalent to those for a CEA parameter:

   * Key (i.e. formal name)
   * Name to show in UI
   * Description
   * Whether writeable by client (probably as boolean attribute "readOnly")
   * UCD 1+
   * Unit

-- IVOA.GuyRixon - 04 Jul 2006

Suggestion 2 does have some appeal. Suggestion 1, however, I am not so keen on, the original intention of this proposal was to be able to find out about the "custom properties" that clients might be using, rather than simply the ones that the service "understands" - for the extra metadata to be supplied though it would imply that an extra "registerPropertyMetadata" operation would be needed before a client could set the property.

And I do not see this being at all expensive to implement - it is much easier to implement than storing the property values of each of the property instances for each of the stored data objects, and we already require that.


-- IVOA.PaulHarrison - 04 Jul 2006

---++++ Votes
| *name* | *vote*| *comment* |
| IVOA.PaulHarrison | +1 |proposer|
| IVOA.DaveMorris | +1 ||
| IVOA.GuyRixon | +1 |With details above|
| IVOA.MatthewGraham | +1 ||

_*Result*_: done

---+++ Re-assess the semantics of creating a server named data object.
see discussion on mailing list for more background http://www.ivoa.net/forum/vospace/0606/0095.htm
---++++ Votes
| *name* | *vote*| *comment* |

_*Result*_: done

---+++ Rename Status (of Node object) to TransferStatus

The Status member of the Node object is really referring to the transfer status.

---++++ Votes
| *name* | *vote*| *comment* |
| IVOA.PaulHarrison | +1 |proposer - benefit is increased clarity|
| IVOA.GuyRixon | +1 ||

_*Result*_:  new status that just indicates "busy" or not.
---+++ Consider interactions when two clients attempt operations on same data object
---++++use case

client 2 attempts to delete a file that client 1 is currently downloading via a "pull" transfer.

As specified at the moment, Client 1 could have a data transfer cut off with out any real knowledge of why, or client 2 could receive a PermissionDenied fault

Possible solutions 
   * add a DataInUse fault to the destructive manipulation methods
   * add a dataBeingRead to the list of Status values

This use case is actually quite an implementation challenge....involves close interaction with the states of the actual data streams for the transports.

---++++ Votes
| *name* | *vote*| *comment* |
| IVOA.PaulHarrison | +1 |proposer|
_*Result*_:  new status that just indicates "busy" or not.

---+++ Expand "expired" Node status to include "failed"

perhaps failed is a better name can then encompass the case where a partial data transfer has occured and failed half way, for some reason other than expiry

---++++ Votes
| *name* | *vote*| *comment* |
| IVOA.PaulHarrison | +1 |proposer|

---+++ Different type of list operation that returns only the uris...
Efficiency...

Proposed (and provisionally agreed by DaveMorris, MatthewGraham and GuyRixon) that we include a "level" parameter to select what is returned. The returned elements are always node elements but:

   * level="minimal" returns nodes with only the URI child;
   * level="standard" returns nodes with the properties as well;
   * level="extended" alls the return of derived types of nodes.

(Naming of the levels can be tweaked if necessary.)
"Derived types of nodes" allows implementors to design special nodes
(e.g. LdapDataNode) that have all the parts of a normal node and also extra children. The level parameter protects
the clients from parser grief when derived node-types are in play.

The URI child might become an attribute of the node element.

-- IVOA.GuyRixon - 04 Jul 2006

This design does not seem very OO to me - too may meanings are being attached to level - we are saying that the ListNodes operation returns a type Node, but for level="minimal" actually something is returned that is an "incomplete" node though it is still being called a node. It would seem cleaner to have a different operation ListIdentifiers that returned an identifier array for this case.

the level=extended seems not to be within scope of 1.0 - It also seems to be confusing the metadata about a node with the data in a node.


-- IVOA.PaulHarrison - 05 Jul 2006
---++++ Votes
| *name* | *vote*| *comment* |
| IVOA.PaulHarrison | +1 |proposer|
| IVOA.GuyRixon | +1 | With changes listed above |
_*Result*_:  done


---+++ Confusing that pullDataToVoSpace, pushDataFromVoSpace return transfer object.

The transfer object contains a location element which is intended to be the uri that pullDataFromVoSpace and pushDataToVoSpace supply as an *output* parameter, so it is confusing that pullDataToVoSpace, pushDataFromVoSpace return a uri for a transfer when for those methods the uri for transfer is an *input* parameter.

Recommend that the pullDataToVoSpace, pushDataFromVoSpace return only what is necessary for the client to know - this might mean removing the transfer object from the return list, or refactoring the transfer object - global analysis needed.

---++++ Votes
| *name* | *vote*| *comment* |
| IVOA.PaulHarrison | +1 |proposer|

---+++ Need to have describe how VOSpace should be registered
The registration of VOSpace servers will be crucial their operation - we need some discussion of this and preferably a VOResouce extension schema to accompany it.
 IVOA.PaulHarrison - 27 Jun 2006
---++++ Votes
| *name* | *vote*| *comment* |
| IVOA.PaulHarrison | +1 |proposer|
| IVOA.GuyRixon | +1 ||

_*Result*_: registry schema done


---+++ Transports operation needs to return the direction of operation 
The operation that returns the list of transports supported needs to return also the direction that the transport is supported in. e.g. a VOSpace server might support ftp as a client (in pullDataToVoSpace), but not as a server (pushDataToVoSpace). 
---++++ Votes
| *name* | *vote*| *comment* |
| IVOA.PaulHarrison | +1 |proposer|

_*Result*_:   protocols are listed in two categories, "accepts" and "provides"










<br/>
<!--
      * Set ALLOWTOPICRENAME = IVOA.TWikiAdminGroup
-->
Topic revision: r1 - 2006-07-19 - PaulHarrison
IVOA
Log in or Register
IVOA.net
Wiki Home
WebChanges
WebTopicList
WebStatistics
Twiki Meta & Help
IVOA
Know
Main
Sandbox
TWiki
TWiki intro
TWiki tutorial
User registration
Notify me
Working Groups
Interest Groups
Committees
Stds&Procs
www.ivoa.net
Documents
Events
Members
XML Schema