| ||||||||||||||||||||||||||||||||||||||||||||||
Changed: | ||||||||||||||||||||||||||||||||||||||||||||||
< < | VOSpace home page | |||||||||||||||||||||||||||||||||||||||||||||
> > | VOSpace home page | |||||||||||||||||||||||||||||||||||||||||||||
Discussion of the VOSpace 1.1 featuresThis is a discussion page for VOSpace-1.1 service features. Version 1.1 aims to extend the VOSpace-1.0 specification to include links and containers. The proposed mechanism is that we introduce two new node types:
This is somewhere where we can post proposals and to enable interested parties to discuss the changes. Please add your suggestion to this page and vote on other suggestions
Logical storage unitsA request has from our friends at SDSC to include references to the actual storage units that data is being deposited on. The use case is data replication so, for example, I want to move/copy a data object from a slow tape archive to an ultrafast disk but both hardware units are within the same VOSpace or I want to retrieve a data object from the ultrafast disk copy and not the slow tape one. I think that we can incorporate this easily into our existing data model. We will refer to hardware units as logical storage units with the implication that they are identified via a logical identifier (URI) that is set by the particular VOSpace implementation. To get the list of available storage units from a VOSpace, we will need a method: getLogicalStorageUnits() which will return a list of URIs. These URIs may be resolvable to a description of the storage unit. The logical storage unit identifier will be an optional argument in theI'm not sure there is a strong science use case for this. Turn your example round the other way, and what is the science use case for explicitly wanting to get the data from the slow tape store rather than the fast disk store ? Adding references to the storage units will add a whole load of complexity to VOSpace, that is already handled by other tools and services. As soon as we start to deal with things like replication, we will need to define the expected behaviour in a lot more detail than just simply adding references to logical storage units. Some of the question that we would need to answer (not a complete list) :
Votes
Alternative service interfacesIn reference to the above suggestion of adding references to logical storage units to support data replication. Why attempt to re-invent the wheel. If a VOSpace service is based on a SRB or iRODS system, then provide a way for the user to access the SRB or iRODS service interface directly. If a VOSpace service uses a different replication mechanism, then provide a way for the user to control the replication using that mechanism instead. The suggestion is we add a list of alternative service interfaces for accessing the node. These can either be aded to the existingprovides list, or in a specific list of alternative service capabilities.
In the specific example of data replication using SRB or iRODS.
If we define a URI that means 'access the data using the iRODS service interface'.
Then a VOSpace service that is based on a SRB or iRODS server can add the iRODS service interface in the provides list for a node.
<node uri="vos://xxxx"> .... <provides> .... <!-- Access the data using an iRODS service --> <view uri="ivo://irods.sdsc.edu/interface/irods-v0.9"> <endpoint>.....</endpoint> </view> </provides> </node>In effect the VOSpace service is saying, "the data replication for this node can be handled using the iRODs service API at [endpoint]. In order to implement this, we need to allow the view element to contain an endpoint URL.
A slight tweak to the VOSpace provides and view elements, and we get access to all of the iRODS service API for free.
Votes
Data access using DAL servicesThe sameview | endpoint syntax would also enable us to provide data access using other IVOA services.
If we had a container node that contained images.
Then the following example says that the images in the container can also be accessed
using a DAL SIAP service.
<node uri="vos://xxxx"> .... <provides> <!--+ | Provide access to the images in this container using a SIAP service. +--> <view uri="ivo://net.ivoa.dal/core/siap-1.0"> <endpoint>http://host/service</endpoint> </view> </provides> </node>This would enable an astronomer to create a container in vospace, drop some images into it, and then query the set of images using the SIAP interface. Votes
ContainerNodeFrom the introduction above : this cannot hold any data (no bytes) but can have .... views for container level formatting (aggregate zip/gzip) So, we have :
accepts and provides views, then to an external Actor a ContainerNode behaves the same way as a DataNode,
and does indeed appear to handle data.
So the definition becomes :
Votes
Separation of views and formatsI think it may be useful to re-visit how we represent views and data formats. In the current schema, we attempt to combine the two concepts in one view URI. We can either represent what the object is, or we can either represent how the data is formatted. However, I don't think the combined form is capable of representing both concepts clearly enough. Proposal is that we separate the two concepts, by representing the data format using a separate XML element and URI.<node uri="vos://xxxx"> .... <accepts> <!--+ | This node accepts images. +--> <view uri="ivo://vospace/core/view/image"> <!--+ | Formatted as PNG or FITS files. +--> <format uri="ivo://vospace/core/format/image-png"/> <format uri="ivo://vospace/core/format/image-fits"/> </view> </accepts> .... </node> Votes
Container viewsIf we registered a view URI that means "access the container as an aggregate" then we would be able to describe views of a container as an archive file. In this example, the server is saying it can accept a zip or tar archive, and will unpack the file to create the container contents on the server.<node uri="vos://xxxx"> .... <accepts> <!--+ | This (container) node can accept an archive file. +--> <view uri="ivo://vospace/core/view/container-archive"> <!--+ | Formatted as a ZIP or TAR file. +--> <format uri="ivo://vospace/core/format/archive-zip"/> <format uri="ivo://vospace/core/format/archive-tar"/> </view> </accepts> .... </node>In this example, the server can provide a zip or tar archive of the container contents. <node uri="vos://xxxx"> .... <provides> <!--+ | This (container) node can provide an archive file of the contents. +--> <view uri="ivo://vospace/core/view/container-archive"> <!--+ | Formatted as a ZIP or TAR file. +--> <format uri="ivo://vospace/core/format/archive-zip"/> <format uri="ivo://vospace/core/format/archive-tar"/> </view> </provides> .... </node> Nested views and formatsIf we allow archive formats to contain nested views and formats, then we can specify the type of files within the archive. In this example, the server can provide a zip or tar archive containing FITS or PNG image files.<node uri="vos://xxxx"> .... <provides> <!--+ | This (container) node can provide an archive file of the contents. +--> <view uri="ivo://vospace/core/view/container-archive"> <!--+ | Formatted as a ZIP file. +--> <format uri="ivo://vospace/core/format/archive-zip"> <!--+ | Containing images. +--> <view uri="ivo://vospace/core/view/image"> <!--+ | Formatted as PNG or FITS files. +--> <format uri="ivo://vospace/core/format/image-png"/> <format uri="ivo://vospace/core/format/image-fits"/> </view> </format> </view> </provides> .... </node> Votes
Generated filenamesAt the last IVOA meeting, we discussed service generated names. The fact thatvos://null does not work for containers, and the suggestion to use ".auto" or "/" as an alternative.
As far as I can remember, the decision at the time was to use a trailing "/" on the destination filename.
I would like to raise one possible issue.
If we have a command line client, based on the conventional behavior for most command line applications the user would probably expect the following command
cp data.xml vos://service/path/to create a file called vos://service/path/data.xml in the vospace, rather than generate a new filename for it vos://service/path/5116-8621
-- DaveMorris - 26 Sep 2007
Protocol and view parametersIn the current schema, protocol and View parameters are referred to by name.<view uri="ivo://net.ivoa.vospace/views#example"> <param name="user-dn">cn=xx,dc=yy,dc=zz</param> </view>In addition, the recommended practice is to register a set of View or Protocol definitions in the same registry document, using the # fragment identifier to select a specific View within the document.
Having used the # fragment identifier to select the View, this leaves us with a minor problem of how to refer
to a specific parameter.
In addition, looking forward to the possibility of refactoring VOSpace as a REST service in the future, it may be useful to
model a View using the same syntax as a Node with Property(s). At the moment, this would not be possible, as a Node Property is
identified by a URI, but a View parameter is identified by name.
The proposal is to change the schema to use the same syntax as Node Properties for both View and Protocol Parameters.
So the above example becomes :
<view uri="ivo://net.ivoa.vospace/views#example"> <property uri="ivo://net.ivoa.vospace/property#user-dn">cn=xx,dc=yy,dc=zz</property> </view>Changing the View param element into a standard property element. Defining the parameters as properties, outside the scope of a specific View or Protocol allows us to use the same definition for common properties that occur in different Views or Protocols, without having to re-define them each time. Votes
Core set of properties, protocols and viewsPart of the GWS session at the interop meeting will discuss what should be included in the core set of vospace properties, protocols and views. Discussion page for the core list is here -- DaveMorris - 27 Sep 2007 |
| ||||||||||||||||||||||||||||||||||||||||||||||
Changed: | ||||||||||||||||||||||||||||||||||||||||||||||
< < | Discussion of the VOSpace 1.1 specification | |||||||||||||||||||||||||||||||||||||||||||||
> > | Discussion of the VOSpace 1.1 features | |||||||||||||||||||||||||||||||||||||||||||||
Changed: | ||||||||||||||||||||||||||||||||||||||||||||||
< < | This is a discussion page for the VOSpace-1.1 service specification. | |||||||||||||||||||||||||||||||||||||||||||||
> > | This is a discussion page for VOSpace-1.1 service features. | |||||||||||||||||||||||||||||||||||||||||||||
Version 1.1 aims to extend the VOSpace-1.0 specification to include links and containers. The proposed mechanism is that we introduce two new node types:
This is somewhere where we can post proposals and to enable interested parties to discuss the changes. Please add your suggestion to this page and vote on other suggestions
Logical storage unitsA request has from our friends at SDSC to include references to the actual storage units that data is being deposited on. The use case is data replication so, for example, I want to move/copy a data object from a slow tape archive to an ultrafast disk but both hardware units are within the same VOSpace or I want to retrieve a data object from the ultrafast disk copy and not the slow tape one. I think that we can incorporate this easily into our existing data model. We will refer to hardware units as logical storage units with the implication that they are identified via a logical identifier (URI) that is set by the particular VOSpace implementation. To get the list of available storage units from a VOSpace, we will need a method: getLogicalStorageUnits() which will return a list of URIs. These URIs may be resolvable to a description of the storage unit. The logical storage unit identifier will be an optional argument in theI'm not sure there is a strong science use case for this. Turn your example round the other way, and what is the science use case for explicitly wanting to get the data from the slow tape store rather than the fast disk store ? Adding references to the storage units will add a whole load of complexity to VOSpace, that is already handled by other tools and services. As soon as we start to deal with things like replication, we will need to define the expected behaviour in a lot more detail than just simply adding references to logical storage units. Some of the question that we would need to answer (not a complete list) :
Votes
Alternative service interfacesIn reference to the above suggestion of adding references to logical storage units to support data replication. Why attempt to re-invent the wheel. If a VOSpace service is based on a SRB or iRODS system, then provide a way for the user to access the SRB or iRODS service interface directly. If a VOSpace service uses a different replication mechanism, then provide a way for the user to control the replication using that mechanism instead. The suggestion is we add a list of alternative service interfaces for accessing the node. These can either be aded to the existingprovides list, or in a specific list of alternative service capabilities.
In the specific example of data replication using SRB or iRODS.
If we define a URI that means 'access the data using the iRODS service interface'.
Then a VOSpace service that is based on a SRB or iRODS server can add the iRODS service interface in the provides list for a node.
<node uri="vos://xxxx"> .... <provides> .... <!-- Access the data using an iRODS service --> <view uri="ivo://irods.sdsc.edu/interface/irods-v0.9"> <endpoint>.....</endpoint> </view> </provides> </node>In effect the VOSpace service is saying, "the data replication for this node can be handled using the iRODs service API at [endpoint]. In order to implement this, we need to allow the view element to contain an endpoint URL.
A slight tweak to the VOSpace provides and view elements, and we get access to all of the iRODS service API for free.
Votes
Data access using DAL servicesThe sameview | endpoint syntax would also enable us to provide data access using other IVOA services.
If we had a container node that contained images.
Then the following example says that the images in the container can also be accessed
using a DAL SIAP service.
<node uri="vos://xxxx"> .... <provides> <!--+ | Provide access to the images in this container using a SIAP service. +--> <view uri="ivo://net.ivoa.dal/core/siap-1.0"> <endpoint>http://host/service</endpoint> </view> </provides> </node>This would enable an astronomer to create a container in vospace, drop some images into it, and then query the set of images using the SIAP interface. Votes
ContainerNodeFrom the introduction above : this cannot hold any data (no bytes) but can have .... views for container level formatting (aggregate zip/gzip) So, we have :
accepts and provides views, then to an external Actor a ContainerNode behaves the same way as a DataNode,
and does indeed appear to handle data.
So the definition becomes :
Votes
Separation of views and formatsI think it may be useful to re-visit how we represent views and data formats. In the current schema, we attempt to combine the two concepts in one view URI. We can either represent what the object is, or we can either represent how the data is formatted. However, I don't think the combined form is capable of representing both concepts clearly enough. Proposal is that we separate the two concepts, by representing the data format using a separate XML element and URI.<node uri="vos://xxxx"> .... <accepts> <!--+ | This node accepts images. +--> <view uri="ivo://vospace/core/view/image"> <!--+ | Formatted as PNG or FITS files. +--> <format uri="ivo://vospace/core/format/image-png"/> <format uri="ivo://vospace/core/format/image-fits"/> </view> </accepts> .... </node> Votes
Container viewsIf we registered a view URI that means "access the container as an aggregate" then we would be able to describe views of a container as an archive file. In this example, the server is saying it can accept a zip or tar archive, and will unpack the file to create the container contents on the server.<node uri="vos://xxxx"> .... <accepts> <!--+ | This (container) node can accept an archive file. +--> <view uri="ivo://vospace/core/view/container-archive"> <!--+ | Formatted as a ZIP or TAR file. +--> <format uri="ivo://vospace/core/format/archive-zip"/> <format uri="ivo://vospace/core/format/archive-tar"/> </view> </accepts> .... </node>In this example, the server can provide a zip or tar archive of the container contents. <node uri="vos://xxxx"> .... <provides> <!--+ | This (container) node can provide an archive file of the contents. +--> <view uri="ivo://vospace/core/view/container-archive"> <!--+ | Formatted as a ZIP or TAR file. +--> <format uri="ivo://vospace/core/format/archive-zip"/> <format uri="ivo://vospace/core/format/archive-tar"/> </view> </provides> .... </node> Nested views and formatsIf we allow archive formats to contain nested views and formats, then we can specify the type of files within the archive. In this example, the server can provide a zip or tar archive containing FITS or PNG image files.<node uri="vos://xxxx"> .... <provides> <!--+ | This (container) node can provide an archive file of the contents. +--> <view uri="ivo://vospace/core/view/container-archive"> <!--+ | Formatted as a ZIP file. +--> <format uri="ivo://vospace/core/format/archive-zip"> <!--+ | Containing images. +--> <view uri="ivo://vospace/core/view/image"> <!--+ | Formatted as PNG or FITS files. +--> <format uri="ivo://vospace/core/format/image-png"/> <format uri="ivo://vospace/core/format/image-fits"/> </view> </format> </view> </provides> .... </node> Votes
Generated filenamesAt the last IVOA meeting, we discussed service generated names. The fact thatvos://null does not work for containers, and the suggestion to use ".auto" or "/" as an alternative.
As far as I can remember, the decision at the time was to use a trailing "/" on the destination filename.
I would like to raise one possible issue.
If we have a command line client, based on the conventional behavior for most command line applications the user would probably expect the following command
cp data.xml vos://service/path/to create a file called vos://service/path/data.xml in the vospace, rather than generate a new filename for it vos://service/path/5116-8621
-- DaveMorris - 26 Sep 2007
Protocol and view parametersIn the current schema, protocol and View parameters are referred to by name.<view uri="ivo://net.ivoa.vospace/views#example"> <param name="user-dn">cn=xx,dc=yy,dc=zz</param> </view>In addition, the recommended practice is to register a set of View or Protocol definitions in the same registry document, using the # fragment identifier to select a specific View within the document.
Having used the # fragment identifier to select the View, this leaves us with a minor problem of how to refer
to a specific parameter.
In addition, looking forward to the possibility of refactoring VOSpace as a REST service in the future, it may be useful to
model a View using the same syntax as a Node with Property(s). At the moment, this would not be possible, as a Node Property is
identified by a URI, but a View parameter is identified by name.
The proposal is to change the schema to use the same syntax as Node Properties for both View and Protocol Parameters.
So the above example becomes :
<view uri="ivo://net.ivoa.vospace/views#example"> <property uri="ivo://net.ivoa.vospace/property#user-dn">cn=xx,dc=yy,dc=zz</property> </view>Changing the View param element into a standard property element. Defining the parameters as properties, outside the scope of a specific View or Protocol allows us to use the same definition for common properties that occur in different Views or Protocols, without having to re-define them each time. Votes
Core set of properties, protocols and viewsPart of the GWS session at the interop meeting will discuss what should be included in the core set of vospace properties, protocols and views. Discussion page for the core list is here -- DaveMorris - 27 Sep 2007<--
|
Discussion of the VOSpace 1.1 specificationThis is a discussion page for the VOSpace-1.1 service specification. Version 1.1 aims to extend the VOSpace-1.0 specification to include links and containers. The proposed mechanism is that we introduce two new node types:
This is somewhere where we can post proposals and to enable interested parties to discuss the changes. Please add your suggestion to this page and vote on other suggestions
Logical storage unitsA request has from our friends at SDSC to include references to the actual storage units that data is being deposited on. The use case is data replication so, for example, I want to move/copy a data object from a slow tape archive to an ultrafast disk but both hardware units are within the same VOSpace or I want to retrieve a data object from the ultrafast disk copy and not the slow tape one. I think that we can incorporate this easily into our existing data model. We will refer to hardware units as logical storage units with the implication that they are identified via a logical identifier (URI) that is set by the particular VOSpace implementation. To get the list of available storage units from a VOSpace, we will need a method: getLogicalStorageUnits() which will return a list of URIs. These URIs may be resolvable to a description of the storage unit. The logical storage unit identifier will be an optional argument in theI'm not sure there is a strong science use case for this. Turn your example round the other way, and what is the science use case for explicitly wanting to get the data from the slow tape store rather than the fast disk store ? Adding references to the storage units will add a whole load of complexity to VOSpace, that is already handled by other tools and services. As soon as we start to deal with things like replication, we will need to define the expected behaviour in a lot more detail than just simply adding references to logical storage units. Some of the question that we would need to answer (not a complete list) :
Votes
Alternative service interfacesIn reference to the above suggestion of adding references to logical storage units to support data replication. Why attempt to re-invent the wheel. If a VOSpace service is based on a SRB or iRODS system, then provide a way for the user to access the SRB or iRODS service interface directly. If a VOSpace service uses a different replication mechanism, then provide a way for the user to control the replication using that mechanism instead. The suggestion is we add a list of alternative service interfaces for accessing the node. These can either be aded to the existingprovides list, or in a specific list of alternative service capabilities.
In the specific example of data replication using SRB or iRODS.
If we define a URI that means 'access the data using the iRODS service interface'.
Then a VOSpace service that is based on a SRB or iRODS server can add the iRODS service interface in the provides list for a node.
<node uri="vos://xxxx"> .... <provides> .... <!-- Access the data using an iRODS service --> <view uri="ivo://irods.sdsc.edu/interface/irods-v0.9"> <endpoint>.....</endpoint> </view> </provides> </node>In effect the VOSpace service is saying, "the data replication for this node can be handled using the iRODs service API at [endpoint]. In order to implement this, we need to allow the view element to contain an endpoint URL.
A slight tweak to the VOSpace provides and view elements, and we get access to all of the iRODS service API for free.
Votes
Data access using DAL servicesThe sameview | endpoint syntax would also enable us to provide data access using other IVOA services.
If we had a container node that contained images.
Then the following example says that the images in the container can also be accessed
using a DAL SIAP service.
<node uri="vos://xxxx"> .... <provides> <!--+ | Provide access to the images in this container using a SIAP service. +--> <view uri="ivo://net.ivoa.dal/core/siap-1.0"> <endpoint>http://host/service</endpoint> </view> </provides> </node>This would enable an astronomer to create a container in vospace, drop some images into it, and then query the set of images using the SIAP interface. Votes
ContainerNodeFrom the introduction above : this cannot hold any data (no bytes) but can have .... views for container level formatting (aggregate zip/gzip) So, we have :
accepts and provides views, then to an external Actor a ContainerNode behaves the same way as a DataNode,
and does indeed appear to handle data.
So the definition becomes :
Votes
Separation of views and formatsI think it may be useful to re-visit how we represent views and data formats. In the current schema, we attempt to combine the two concepts in one view URI. We can either represent what the object is, or we can either represent how the data is formatted. However, I don't think the combined form is capable of representing both concepts clearly enough. Proposal is that we separate the two concepts, by representing the data format using a separate XML element and URI.<node uri="vos://xxxx"> .... <accepts> <!--+ | This node accepts images. +--> <view uri="ivo://vospace/core/view/image"> <!--+ | Formatted as PNG or FITS files. +--> <format uri="ivo://vospace/core/format/image-png"/> <format uri="ivo://vospace/core/format/image-fits"/> </view> </accepts> .... </node> Votes
Container viewsIf we registered a view URI that means "access the container as an aggregate" then we would be able to describe views of a container as an archive file. In this example, the server is saying it can accept a zip or tar archive, and will unpack the file to create the container contents on the server.<node uri="vos://xxxx"> .... <accepts> <!--+ | This (container) node can accept an archive file. +--> <view uri="ivo://vospace/core/view/container-archive"> <!--+ | Formatted as a ZIP or TAR file. +--> <format uri="ivo://vospace/core/format/archive-zip"/> <format uri="ivo://vospace/core/format/archive-tar"/> </view> </accepts> .... </node>In this example, the server can provide a zip or tar archive of the container contents. <node uri="vos://xxxx"> .... <provides> <!--+ | This (container) node can provide an archive file of the contents. +--> <view uri="ivo://vospace/core/view/container-archive"> <!--+ | Formatted as a ZIP or TAR file. +--> <format uri="ivo://vospace/core/format/archive-zip"/> <format uri="ivo://vospace/core/format/archive-tar"/> </view> </provides> .... </node> Nested views and formatsIf we allow archive formats to contain nested views and formats, then we can specify the type of files within the archive. In this example, the server can provide a zip or tar archive containing FITS or PNG image files.<node uri="vos://xxxx"> .... <provides> <!--+ | This (container) node can provide an archive file of the contents. +--> <view uri="ivo://vospace/core/view/container-archive"> <!--+ | Formatted as a ZIP file. +--> <format uri="ivo://vospace/core/format/archive-zip"> <!--+ | Containing images. +--> <view uri="ivo://vospace/core/view/image"> <!--+ | Formatted as PNG or FITS files. +--> <format uri="ivo://vospace/core/format/image-png"/> <format uri="ivo://vospace/core/format/image-fits"/> </view> </format> </view> </provides> .... </node> Votes
Generated filenamesAt the last IVOA meeting, we discussed service generated names. The fact thatvos://null does not work for containers, and the suggestion to use ".auto" or "/" as an alternative.
As far as I can remember, the decision at the time was to use a trailing "/" on the destination filename.
I would like to raise one possible issue.
If we have a command line client, based on the conventional behavior for most command line applications the user would probably expect the following command
cp data.xml vos://service/path/to create a file called vos://service/path/data.xml in the vospace, rather than generate a new filename for it vos://service/path/5116-8621
-- DaveMorris - 26 Sep 2007
Protocol and view parametersIn the current schema, protocol and View parameters are referred to by name.<view uri="ivo://net.ivoa.vospace/views#example"> <param name="user-dn">cn=xx,dc=yy,dc=zz</param> </view>In addition, the recommended practice is to register a set of View or Protocol definitions in the same registry document, using the # fragment identifier to select a specific View within the document.
Having used the # fragment identifier to select the View, this leaves us with a minor problem of how to refer
to a specific parameter.
In addition, looking forward to the possibility of refactoring VOSpace as a REST service in the future, it may be useful to
model a View using the same syntax as a Node with Property(s). At the moment, this would not be possible, as a Node Property is
identified by a URI, but a View parameter is identified by name.
The proposal is to change the schema to use the same syntax as Node Properties for both View and Protocol Parameters.
So the above example becomes :
<view uri="ivo://net.ivoa.vospace/views#example"> <property uri="ivo://net.ivoa.vospace/property#user-dn">cn=xx,dc=yy,dc=zz</property> </view>Changing the View param element into a standard property element. Defining the parameters as properties, outside the scope of a specific View or Protocol allows us to use the same definition for common properties that occur in different Views or Protocols, without having to re-define them each time. Votes
| ||||||||||||||||||||||||||||||||||||||||||||||||
Added: | ||||||||||||||||||||||||||||||||||||||||||||||||
> > |
Core set of properties, protocols and viewsPart of the GWS session at the interop meeting will discuss what should be included in the core set of vospace properties, protocols and views. Discussion page for the core list is here -- DaveMorris - 27 Sep 2007 | |||||||||||||||||||||||||||||||||||||||||||||||
<--
|
Discussion of the VOSpace 1.1 specificationThis is a discussion page for the VOSpace-1.1 service specification. Version 1.1 aims to extend the VOSpace-1.0 specification to include links and containers. The proposed mechanism is that we introduce two new node types:
This is somewhere where we can post proposals and to enable interested parties to discuss the changes. Please add your suggestion to this page and vote on other suggestions
Logical storage unitsA request has from our friends at SDSC to include references to the actual storage units that data is being deposited on. The use case is data replication so, for example, I want to move/copy a data object from a slow tape archive to an ultrafast disk but both hardware units are within the same VOSpace or I want to retrieve a data object from the ultrafast disk copy and not the slow tape one. I think that we can incorporate this easily into our existing data model. We will refer to hardware units as logical storage units with the implication that they are identified via a logical identifier (URI) that is set by the particular VOSpace implementation. To get the list of available storage units from a VOSpace, we will need a method: getLogicalStorageUnits() which will return a list of URIs. These URIs may be resolvable to a description of the storage unit. The logical storage unit identifier will be an optional argument in theI'm not sure there is a strong science use case for this. Turn your example round the other way, and what is the science use case for explicitly wanting to get the data from the slow tape store rather than the fast disk store ? Adding references to the storage units will add a whole load of complexity to VOSpace, that is already handled by other tools and services. As soon as we start to deal with things like replication, we will need to define the expected behaviour in a lot more detail than just simply adding references to logical storage units. Some of the question that we would need to answer (not a complete list) :
Votes
Alternative service interfacesIn reference to the above suggestion of adding references to logical storage units to support data replication. Why attempt to re-invent the wheel. If a VOSpace service is based on a SRB or iRODS system, then provide a way for the user to access the SRB or iRODS service interface directly. If a VOSpace service uses a different replication mechanism, then provide a way for the user to control the replication using that mechanism instead. The suggestion is we add a list of alternative service interfaces for accessing the node. These can either be aded to the existingprovides list, or in a specific list of alternative service capabilities.
In the specific example of data replication using SRB or iRODS.
If we define a URI that means 'access the data using the iRODS service interface'.
Then a VOSpace service that is based on a SRB or iRODS server can add the iRODS service interface in the provides list for a node.
<node uri="vos://xxxx"> .... <provides> .... <!-- Access the data using an iRODS service --> <view uri="ivo://irods.sdsc.edu/interface/irods-v0.9"> <endpoint>.....</endpoint> </view> </provides> </node>In effect the VOSpace service is saying, "the data replication for this node can be handled using the iRODs service API at [endpoint]. In order to implement this, we need to allow the view element to contain an endpoint URL.
A slight tweak to the VOSpace provides and view elements, and we get access to all of the iRODS service API for free.
Votes
Data access using DAL servicesThe sameview | endpoint syntax would also enable us to provide data access using other IVOA services.
If we had a container node that contained images.
Then the following example says that the images in the container can also be accessed
using a DAL SIAP service.
<node uri="vos://xxxx"> .... <provides> <!--+ | Provide access to the images in this container using a SIAP service. +--> <view uri="ivo://net.ivoa.dal/core/siap-1.0"> <endpoint>http://host/service</endpoint> </view> </provides> </node>This would enable an astronomer to create a container in vospace, drop some images into it, and then query the set of images using the SIAP interface. Votes
ContainerNodeFrom the introduction above : this cannot hold any data (no bytes) but can have .... views for container level formatting (aggregate zip/gzip) So, we have :
accepts and provides views, then to an external Actor a ContainerNode behaves the same way as a DataNode,
and does indeed appear to handle data.
So the definition becomes :
Votes
Separation of views and formatsI think it may be useful to re-visit how we represent views and data formats. In the current schema, we attempt to combine the two concepts in one view URI. We can either represent what the object is, or we can either represent how the data is formatted. However, I don't think the combined form is capable of representing both concepts clearly enough. Proposal is that we separate the two concepts, by representing the data format using a separate XML element and URI.<node uri="vos://xxxx"> .... <accepts> <!--+ | This node accepts images. +--> <view uri="ivo://vospace/core/view/image"> <!--+ | Formatted as PNG or FITS files. +--> <format uri="ivo://vospace/core/format/image-png"/> <format uri="ivo://vospace/core/format/image-fits"/> </view> </accepts> .... </node> Votes
Container viewsIf we registered a view URI that means "access the container as an aggregate" then we would be able to describe views of a container as an archive file. In this example, the server is saying it can accept a zip or tar archive, and will unpack the file to create the container contents on the server.<node uri="vos://xxxx"> .... <accepts> <!--+ | This (container) node can accept an archive file. +--> <view uri="ivo://vospace/core/view/container-archive"> <!--+ | Formatted as a ZIP or TAR file. +--> <format uri="ivo://vospace/core/format/archive-zip"/> <format uri="ivo://vospace/core/format/archive-tar"/> </view> </accepts> .... </node>In this example, the server can provide a zip or tar archive of the container contents. <node uri="vos://xxxx"> .... <provides> <!--+ | This (container) node can provide an archive file of the contents. +--> <view uri="ivo://vospace/core/view/container-archive"> <!--+ | Formatted as a ZIP or TAR file. +--> <format uri="ivo://vospace/core/format/archive-zip"/> <format uri="ivo://vospace/core/format/archive-tar"/> </view> </provides> .... </node> Nested views and formatsIf we allow archive formats to contain nested views and formats, then we can specify the type of files within the archive. In this example, the server can provide a zip or tar archive containing FITS or PNG image files.<node uri="vos://xxxx"> .... <provides> <!--+ | This (container) node can provide an archive file of the contents. +--> <view uri="ivo://vospace/core/view/container-archive"> <!--+ | Formatted as a ZIP file. +--> <format uri="ivo://vospace/core/format/archive-zip"> <!--+ | Containing images. +--> <view uri="ivo://vospace/core/view/image"> <!--+ | Formatted as PNG or FITS files. +--> <format uri="ivo://vospace/core/format/image-png"/> <format uri="ivo://vospace/core/format/image-fits"/> </view> </format> </view> </provides> .... </node> Votes
Generated filenamesAt the last IVOA meeting, we discussed service generated names. The fact thatvos://null does not work for containers, and the suggestion to use ".auto" or "/" as an alternative.
As far as I can remember, the decision at the time was to use a trailing "/" on the destination filename.
I would like to raise one possible issue.
If we have a command line client, based on the conventional behavior for most command line applications the user would probably expect the following command
cp data.xml vos://service/path/to create a file called vos://service/path/data.xml in the vospace, rather than generate a new filename for it vos://service/path/5116-8621
-- DaveMorris - 26 Sep 2007 | ||||||||||||||||||||||||||||||||||||||||||
Added: | ||||||||||||||||||||||||||||||||||||||||||
> > |
Protocol and view parametersIn the current schema, protocol and View parameters are referred to by name.<view uri="ivo://net.ivoa.vospace/views#example"> <param name="user-dn">cn=xx,dc=yy,dc=zz</param> </view>In addition, the recommended practice is to register a set of View or Protocol definitions in the same registry document, using the # fragment identifier to select a specific View within the document.
Having used the # fragment identifier to select the View, this leaves us with a minor problem of how to refer
to a specific parameter.
In addition, looking forward to the possibility of refactoring VOSpace as a REST service in the future, it may be useful to
model a View using the same syntax as a Node with Property(s). At the moment, this would not be possible, as a Node Property is
identified by a URI, but a View parameter is identified by name.
The proposal is to change the schema to use the same syntax as Node Properties for both View and Protocol Parameters.
So the above example becomes :
<view uri="ivo://net.ivoa.vospace/views#example"> <property uri="ivo://net.ivoa.vospace/property#user-dn">cn=xx,dc=yy,dc=zz</property> </view>Changing the View param element into a standard property element. Defining the parameters as properties, outside the scope of a specific View or Protocol allows us to use the same definition for common properties that occur in different Views or Protocols, without having to re-define them each time. Votes
| |||||||||||||||||||||||||||||||||||||||||
<--
|
Discussion of the VOSpace 1.1 specificationThis is a discussion page for the VOSpace-1.1 service specification. Version 1.1 aims to extend the VOSpace-1.0 specification to include links and containers. The proposed mechanism is that we introduce two new node types:
This is somewhere where we can post proposals and to enable interested parties to discuss the changes. Please add your suggestion to this page and vote on other suggestions
Logical storage unitsA request has from our friends at SDSC to include references to the actual storage units that data is being deposited on. The use case is data replication so, for example, I want to move/copy a data object from a slow tape archive to an ultrafast disk but both hardware units are within the same VOSpace or I want to retrieve a data object from the ultrafast disk copy and not the slow tape one. I think that we can incorporate this easily into our existing data model. We will refer to hardware units as logical storage units with the implication that they are identified via a logical identifier (URI) that is set by the particular VOSpace implementation. To get the list of available storage units from a VOSpace, we will need a method: getLogicalStorageUnits() which will return a list of URIs. These URIs may be resolvable to a description of the storage unit. The logical storage unit identifier will be an optional argument in theI'm not sure there is a strong science use case for this. Turn your example round the other way, and what is the science use case for explicitly wanting to get the data from the slow tape store rather than the fast disk store ? Adding references to the storage units will add a whole load of complexity to VOSpace, that is already handled by other tools and services. As soon as we start to deal with things like replication, we will need to define the expected behaviour in a lot more detail than just simply adding references to logical storage units. Some of the question that we would need to answer (not a complete list) :
Votes
Alternative service interfacesIn reference to the above suggestion of adding references to logical storage units to support data replication. Why attempt to re-invent the wheel. If a VOSpace service is based on a SRB or iRODS system, then provide a way for the user to access the SRB or iRODS service interface directly. If a VOSpace service uses a different replication mechanism, then provide a way for the user to control the replication using that mechanism instead. The suggestion is we add a list of alternative service interfaces for accessing the node. These can either be aded to the existingprovides list, or in a specific list of alternative service capabilities.
In the specific example of data replication using SRB or iRODS.
If we define a URI that means 'access the data using the iRODS service interface'.
Then a VOSpace service that is based on a SRB or iRODS server can add the iRODS service interface in the provides list for a node.
<node uri="vos://xxxx"> .... <provides> .... <!-- Access the data using an iRODS service --> <view uri="ivo://irods.sdsc.edu/interface/irods-v0.9"> <endpoint>.....</endpoint> </view> </provides> </node>In effect the VOSpace service is saying, "the data replication for this node can be handled using the iRODs service API at [endpoint]. In order to implement this, we need to allow the view element to contain an endpoint URL.
A slight tweak to the VOSpace provides and view elements, and we get access to all of the iRODS service API for free.
Votes
Data access using DAL servicesThe sameview | endpoint syntax would also enable us to provide data access using other IVOA services.
If we had a container node that contained images.
Then the following example says that the images in the container can also be accessed
using a DAL SIAP service.
<node uri="vos://xxxx"> .... <provides> <!--+ | Provide access to the images in this container using a SIAP service. +--> <view uri="ivo://net.ivoa.dal/core/siap-1.0"> <endpoint>http://host/service</endpoint> </view> </provides> </node>This would enable an astronomer to create a container in vospace, drop some images into it, and then query the set of images using the SIAP interface. Votes
ContainerNodeFrom the introduction above : this cannot hold any data (no bytes) but can have .... views for container level formatting (aggregate zip/gzip) So, we have :
accepts and provides views, then to an external Actor a ContainerNode behaves the same way as a DataNode,
and does indeed appear to handle data.
So the definition becomes :
Votes
Separation of views and formatsI think it may be useful to re-visit how we represent views and data formats. In the current schema, we attempt to combine the two concepts in one view URI. We can either represent what the object is, or we can either represent how the data is formatted. However, I don't think the combined form is capable of representing both concepts clearly enough. Proposal is that we separate the two concepts, by representing the data format using a separate XML element and URI.<node uri="vos://xxxx"> .... <accepts> <!--+ | This node accepts images. +--> <view uri="ivo://vospace/core/view/image"> <!--+ | Formatted as PNG or FITS files. +--> <format uri="ivo://vospace/core/format/image-png"/> <format uri="ivo://vospace/core/format/image-fits"/> </view> </accepts> .... </node> Votes
Container viewsIf we registered a view URI that means "access the container as an aggregate" then we would be able to describe views of a container as an archive file. In this example, the server is saying it can accept a zip or tar archive, and will unpack the file to create the container contents on the server.<node uri="vos://xxxx"> .... <accepts> <!--+ | This (container) node can accept an archive file. +--> <view uri="ivo://vospace/core/view/container-archive"> <!--+ | Formatted as a ZIP or TAR file. +--> <format uri="ivo://vospace/core/format/archive-zip"/> <format uri="ivo://vospace/core/format/archive-tar"/> </view> </accepts> .... </node>In this example, the server can provide a zip or tar archive of the container contents. <node uri="vos://xxxx"> .... <provides> <!--+ | This (container) node can provide an archive file of the contents. +--> <view uri="ivo://vospace/core/view/container-archive"> <!--+ | Formatted as a ZIP or TAR file. +--> <format uri="ivo://vospace/core/format/archive-zip"/> <format uri="ivo://vospace/core/format/archive-tar"/> </view> </provides> .... </node> Nested views and formatsIf we allow archive formats to contain nested views and formats, then we can specify the type of files within the archive. In this example, the server can provide a zip or tar archive containing FITS or PNG image files.<node uri="vos://xxxx"> .... <provides> <!--+ | This (container) node can provide an archive file of the contents. +--> <view uri="ivo://vospace/core/view/container-archive"> <!--+ | Formatted as a ZIP file. +--> <format uri="ivo://vospace/core/format/archive-zip"> <!--+ | Containing images. +--> <view uri="ivo://vospace/core/view/image"> <!--+ | Formatted as PNG or FITS files. +--> <format uri="ivo://vospace/core/format/image-png"/> <format uri="ivo://vospace/core/format/image-fits"/> </view> </format> </view> </provides> .... </node> Votes
| ||||||||||||||||||||||||||||||||||||||||||
Added: | ||||||||||||||||||||||||||||||||||||||||||
> > | Generated filenamesAt the last IVOA meeting, we discussed service generated names. The fact thatvos://null does not work for containers, and the suggestion to use ".auto" or "/" as an alternative.
As far as I can remember, the decision at the time was to use a trailing "/" on the destination filename.
I would like to raise one possible issue.
If we have a command line client, based on the conventional behavior for most command line applications the user would probably expect the following command
cp data.xml vos://service/path/to create a file called vos://service/path/data.xml in the vospace, rather than generate a new filename for it vos://service/path/5116-8621
-- DaveMorris - 26 Sep 2007 | |||||||||||||||||||||||||||||||||||||||||
<--
|
Discussion of the VOSpace 1.1 specificationThis is a discussion page for the VOSpace-1.1 service specification. Version 1.1 aims to extend the VOSpace-1.0 specification to include links and containers. The proposed mechanism is that we introduce two new node types:
This is somewhere where we can post proposals and to enable interested parties to discuss the changes. Please add your suggestion to this page and vote on other suggestions
Logical storage unitsA request has from our friends at SDSC to include references to the actual storage units that data is being deposited on. The use case is data replication so, for example, I want to move/copy a data object from a slow tape archive to an ultrafast disk but both hardware units are within the same VOSpace or I want to retrieve a data object from the ultrafast disk copy and not the slow tape one. I think that we can incorporate this easily into our existing data model. We will refer to hardware units as logical storage units with the implication that they are identified via a logical identifier (URI) that is set by the particular VOSpace implementation. To get the list of available storage units from a VOSpace, we will need a method: getLogicalStorageUnits() which will return a list of URIs. These URIs may be resolvable to a description of the storage unit. The logical storage unit identifier will be an optional argument in theI'm not sure there is a strong science use case for this. Turn your example round the other way, and what is the science use case for explicitly wanting to get the data from the slow tape store rather than the fast disk store ? Adding references to the storage units will add a whole load of complexity to VOSpace, that is already handled by other tools and services. As soon as we start to deal with things like replication, we will need to define the expected behaviour in a lot more detail than just simply adding references to logical storage units. Some of the question that we would need to answer (not a complete list) :
| |||||||||||||||||||||||||
Added: | |||||||||||||||||||||||||
> > | -- DaveMorris - 25 Sep 2007 | ||||||||||||||||||||||||
Votes
Alternative service interfacesIn reference to the above suggestion of adding references to logical storage units to support data replication. Why attempt to re-invent the wheel. If a VOSpace service is based on a SRB or iRODS system, then provide a way for the user to access the SRB or iRODS service interface directly. If a VOSpace service uses a different replication mechanism, then provide a way for the user to control the replication using that mechanism instead. The suggestion is we add a list of alternative service interfaces for accessing the node. These can either be aded to the existingprovides list, or in a specific list of alternative service capabilities.
In the specific example of data replication using SRB or iRODS.
If we define a URI that means 'access the data using the iRODS service interface'.
Then a VOSpace service that is based on a SRB or iRODS server can add the iRODS service interface in the provides list for a node.
<node uri="vos://xxxx"> .... <provides> .... <!-- Access the data using an iRODS service --> <view uri="ivo://irods.sdsc.edu/interface/irods-v0.9"> <endpoint>.....</endpoint> </view> </provides> </node>In effect the VOSpace service is saying, "the data replication for this node can be handled using the iRODs service API at [endpoint]. In order to implement this, we need to allow the view element to contain an endpoint URL.
A slight tweak to the VOSpace provides and view elements, and we get access to all of the iRODS service API for free. | |||||||||||||||||||||||||
Added: | |||||||||||||||||||||||||
> > | Votes
| ||||||||||||||||||||||||
Added: | |||||||||||||||||||||||||
> > | Data access using DAL servicesThe sameview | endpoint syntax would also enable us to provide data access using other IVOA services.
If we had a container node that contained images.
Then the following example says that the images in the container can also be accessed
using a DAL SIAP service.
<node uri="vos://xxxx"> .... <provides> <!--+ | Provide access to the images in this container using a SIAP service. +--> <view uri="ivo://net.ivoa.dal/core/siap-1.0"> <endpoint>http://host/service</endpoint> </view> </provides> </node>This would enable an astronomer to create a container in vospace, drop some images into it, and then query the set of images using the SIAP interface. | ||||||||||||||||||||||||
Votes
ContainerNodeFrom the introduction above : this cannot hold any data (no bytes) but can have .... views for container level formatting (aggregate zip/gzip) So, we have :
accepts and provides views, then to an external Actor a ContainerNode behaves the same way as a DataNode,
and does indeed appear to handle data.
So the definition becomes :
Votes
Separation of views and formatsI think it may be useful to re-visit how we represent views and data formats. In the current schema, we attempt to combine the two concepts in one view URI. We can either represent what the object is, or we can either represent how the data is formatted. However, I don't think the combined form is capable of representing both concepts clearly enough. Proposal is that we separate the two concepts, by representing the data format using a separate XML element and URI.<node uri="vos://xxxx"> .... <accepts> <!--+ | This node accepts images. +--> <view uri="ivo://vospace/core/view/image"> <!--+ | Formatted as PNG or FITS files. +--> <format uri="ivo://vospace/core/format/image-png"/> <format uri="ivo://vospace/core/format/image-fits"/> </view> </accepts> .... </node> Votes
Container viewsIf we registered a view URI that means "access the container as an aggregate" then we would be able to describe views of a container as an archive file. In this example, the server is saying it can accept a zip or tar archive, and will unpack the file to create the container contents on the server.<node uri="vos://xxxx"> .... <accepts> <!--+ | This (container) node can accept an archive file. +--> <view uri="ivo://vospace/core/view/container-archive"> <!--+ | Formatted as a ZIP or TAR file. +--> <format uri="ivo://vospace/core/format/archive-zip"/> <format uri="ivo://vospace/core/format/archive-tar"/> </view> </accepts> .... </node>In this example, the server can provide a zip or tar archive of the container contents. <node uri="vos://xxxx"> .... <provides> <!--+ | This (container) node can provide an archive file of the contents. +--> <view uri="ivo://vospace/core/view/container-archive"> <!--+ | Formatted as a ZIP or TAR file. +--> <format uri="ivo://vospace/core/format/archive-zip"/> <format uri="ivo://vospace/core/format/archive-tar"/> </view> </provides> .... </node> Nested views and formatsIf we allow archive formats to contain nested views and formats, then we can specify the type of files within the archive. In this example, the server can provide a zip or tar archive containing FITS or PNG image files.<node uri="vos://xxxx"> .... <provides> <!--+ | This (container) node can provide an archive file of the contents. +--> <view uri="ivo://vospace/core/view/container-archive"> <!--+ | Formatted as a ZIP file. +--> <format uri="ivo://vospace/core/format/archive-zip"> <!--+ | Containing images. +--> <view uri="ivo://vospace/core/view/image"> <!--+ | Formatted as PNG or FITS files. +--> <format uri="ivo://vospace/core/format/image-png"/> <format uri="ivo://vospace/core/format/image-fits"/> </view> </format> </view> </provides> .... </node> Votes
<--
|
Discussion of the VOSpace 1.1 specificationThis is a discussion page for the VOSpace-1.1 service specification. Version 1.1 aims to extend the VOSpace-1.0 specification to include links and containers. The proposed mechanism is that we introduce two new node types:
This is somewhere where we can post proposals and to enable interested parties to discuss the changes. Please add your suggestion to this page and vote on other suggestions
Logical storage unitsA request has from our friends at SDSC to include references to the actual storage units that data is being deposited on. The use case is data replication so, for example, I want to move/copy a data object from a slow tape archive to an ultrafast disk but both hardware units are within the same VOSpace or I want to retrieve a data object from the ultrafast disk copy and not the slow tape one. I think that we can incorporate this easily into our existing data model. We will refer to hardware units as logical storage units with the implication that they are identified via a logical identifier (URI) that is set by the particular VOSpace implementation. To get the list of available storage units from a VOSpace, we will need a method: getLogicalStorageUnits() which will return a list of URIs. These URIs may be resolvable to a description of the storage unit. The logical storage unit identifier will be an optional argument in theI'm not sure there is a strong science use case for this. Turn your example round the other way, and what is the science use case for explicitly wanting to get the data from the slow tape store rather than the fast disk store ? Adding references to the storage units will add a whole load of complexity to VOSpace, that is already handled by other tools and services. As soon as we start to deal with things like replication, we will need to define the expected behaviour in a lot more detail than just simply adding references to logical storage units. Some of the question that we would need to answer (not a complete list) :
Votes
Alternative service interfacesIn reference to the above suggestion of adding references to logical storage units to support data replication. Why attempt to re-invent the wheel. If a VOSpace service is based on a SRB or iRODS system, then provide a way for the user to access the SRB or iRODS service interface directly. If a VOSpace service uses a different replication mechanism, then provide a way for the user to control the replication using that mechanism instead. The suggestion is we add a list of alternative service interfaces for accessing the node. These can either be aded to the existingprovides list, or in a specific list of alternative service capabilities.
In the specific example of data replication using SRB or iRODS.
If we define a URI that means 'access the data using the iRODS service interface'.
Then a VOSpace service that is based on a SRB or iRODS server can add the iRODS service interface in the provides list for a node.
<node uri="vos://xxxx"> .... <provides> .... | |||||||||||||
Changed: | |||||||||||||
< < | <-- iRODS service (version 0.0) --> | ||||||||||||
> > | <-- Access the data using an iRODS service --> | ||||||||||||
| |||||||||||||
Added: | |||||||||||||
> > | In order to implement this, we need to allow the view element to contain an endpoint URL. | ||||||||||||
A slight tweak to the VOSpace provides and view elements, and we get access to all of the iRODS service API for free. | |||||||||||||
Added: | |||||||||||||
> > | |||||||||||||
Votes
ContainerNodeFrom the introduction above : this cannot hold any data (no bytes) but can have .... views for container level formatting (aggregate zip/gzip) So, we have :
accepts and provides views, then to an external Actor a ContainerNode behaves the same way as a DataNode,
and does indeed appear to handle data.
So the definition becomes :
| |||||||||||||
Changed: | |||||||||||||
< < | Again - Why explicitly add clauses that exclude things that don't break anything ? | ||||||||||||
> > | Votes | ||||||||||||
Added: | |||||||||||||
> > |
Separation of views and formatsI think it may be useful to re-visit how we represent views and data formats. In the current schema, we attempt to combine the two concepts in one view URI. We can either represent what the object is, or we can either represent how the data is formatted. However, I don't think the combined form is capable of representing both concepts clearly enough. Proposal is that we separate the two concepts, by representing the data format using a separate XML element and URI.<node uri="vos://xxxx"> .... <accepts> <!--+ | This node accepts images. +--> <view uri="ivo://vospace/core/view/image"> <!--+ | Formatted as PNG or FITS files. +--> <format uri="ivo://vospace/core/format/image-png"/> <format uri="ivo://vospace/core/format/image-fits"/> </view> </accepts> .... </node> Votes
Container viewsIf we registered a view URI that means "access the container as an aggregate" then we would be able to describe views of a container as an archive file. In this example, the server is saying it can accept a zip or tar archive, and will unpack the file to create the container contents on the server.<node uri="vos://xxxx"> .... <accepts> <!--+ | This (container) node can accept an archive file. +--> <view uri="ivo://vospace/core/view/container-archive"> <!--+ | Formatted as a ZIP or TAR file. +--> <format uri="ivo://vospace/core/format/archive-zip"/> <format uri="ivo://vospace/core/format/archive-tar"/> </view> </accepts> .... </node>In this example, the server can provide a zip or tar archive of the container contents. <node uri="vos://xxxx"> .... <provides> <!--+ | This (container) node can provide an archive file of the contents. +--> <view uri="ivo://vospace/core/view/container-archive"> <!--+ | Formatted as a ZIP or TAR file. +--> <format uri="ivo://vospace/core/format/archive-zip"/> <format uri="ivo://vospace/core/format/archive-tar"/> </view> </provides> .... </node> Nested views and formatsIf we allow archive formats to contain nested views and formats, then we can specify the type of files within the archive. In this example, the server can provide a zip or tar archive containing FITS or PNG image files.<node uri="vos://xxxx"> .... <provides> <!--+ | This (container) node can provide an archive file of the contents. +--> <view uri="ivo://vospace/core/view/container-archive"> <!--+ | Formatted as a ZIP file. +--> <format uri="ivo://vospace/core/format/archive-zip"> <!--+ | Containing images. +--> <view uri="ivo://vospace/core/view/image"> <!--+ | Formatted as PNG or FITS files. +--> <format uri="ivo://vospace/core/format/image-png"/> <format uri="ivo://vospace/core/format/image-fits"/> </view> </format> </view> </provides> .... </node> | ||||||||||||
Votes
<--
|
Discussion of the VOSpace 1.1 specificationThis is a discussion page for the VOSpace-1.1 service specification. Version 1.1 aims to extend the VOSpace-1.0 specification to include links and containers. The proposed mechanism is that we introduce two new node types:
This is somewhere where we can post proposals and to enable interested parties to discuss the changes. Please add your suggestion to this page and vote on other suggestions
Logical storage unitsA request has from our friends at SDSC to include references to the actual storage units that data is being deposited on. The use case is data replication so, for example, I want to move/copy a data object from a slow tape archive to an ultrafast disk but both hardware units are within the same VOSpace or I want to retrieve a data object from the ultrafast disk copy and not the slow tape one. I think that we can incorporate this easily into our existing data model. We will refer to hardware units as logical storage units with the implication that they are identified via a logical identifier (URI) that is set by the particular VOSpace implementation. To get the list of available storage units from a VOSpace, we will need a method: getLogicalStorageUnits() which will return a list of URIs. These URIs may be resolvable to a description of the storage unit. The logical storage unit identifier will be an optional argument in the | ||||||||||||||||||||||
Added: | ||||||||||||||||||||||
> > |
I'm not sure there is a strong science use case for this. Turn your example round the other way, and what is the science use case for explicitly wanting to get the data from the slow tape store rather than the fast disk store ? Adding references to the storage units will add a whole load of complexity to VOSpace, that is already handled by other tools and services. As soon as we start to deal with things like replication, we will need to define the expected behaviour in a lot more detail than just simply adding references to logical storage units. Some of the question that we would need to answer (not a complete list) :
Votes
Alternative service interfacesIn reference to the above suggestion of adding references to logical storage units to support data replication. Why attempt to re-invent the wheel. If a VOSpace service is based on a SRB or iRODS system, then provide a way for the user to access the SRB or iRODS service interface directly. If a VOSpace service uses a different replication mechanism, then provide a way for the user to control the replication using that mechanism instead. The suggestion is we add a list of alternative service interfaces for accessing the node. These can either be aded to the existingprovides list, or in a specific list of alternative service capabilities.
In the specific example of data replication using SRB or iRODS.
If we define a URI that means 'access the data using the iRODS service interface'.
Then a VOSpace service that is based on a SRB or iRODS server can add the iRODS service interface in the provides list for a node.
<node uri="vos://xxxx"> .... <provides> .... <!-- iRODS service (version 0.0) --> <view uri="ivo://irods.sdsc.edu/interface/irods-v0.9"> <endpoint>.....</endpoint> </view> </provides> </node>In effect the VOSpace service is saying, "the data replication for this node can be handled using the iRODs service API at [endpoint]. A slight tweak to the VOSpace provides and view elements, and we get access to all of the iRODS service API for free.
Votes
ContainerNodeFrom the introduction above : this cannot hold any data (no bytes) but can have .... views for container level formatting (aggregate zip/gzip) So, we have :
accepts and provides views, then to an external Actor a ContainerNode behaves the same way as a DataNode,
and does indeed appear to handle data.
So the definition becomes :
Votes
| |||||||||||||||||||||
<--
|
Discussion of the VOSpace 1.1 specificationThis is a discussion page for the VOSpace-1.1 service specification. Version 1.1 aims to extend the VOSpace-1.0 specification to include links and containers. The proposed mechanism is that we introduce two new node types:
This is somewhere where we can post proposals and to enable interested parties to discuss the changes. Please add your suggestion to this page and vote on other suggestions
| ||||||||
Added: | ||||||||
> > |
Logical storage unitsA request has from our friends at SDSC to include references to the actual storage units that data is being deposited on. The use case is data replication so, for example, I want to move/copy a data object from a slow tape archive to an ultrafast disk but both hardware units are within the same VOSpace or I want to retrieve a data object from the ultrafast disk copy and not the slow tape one. I think that we can incorporate this easily into our existing data model. We will refer to hardware units as logical storage units with the implication that they are identified via a logical identifier (URI) that is set by the particular VOSpace implementation. To get the list of available storage units from a VOSpace, we will need a method: getLogicalStorageUnits() which will return a list of URIs. These URIs may be resolvable to a description of the storage unit. The logical storage unit identifier will be an optional argument in the | |||||||
<--
|
Discussion of the VOSpace 1.1 specificationThis is a discussion page for the VOSpace-1.1 service specification. | ||||||||
Changed: | ||||||||
< < | Version 1.1 aims to extend the VOSpace-1.0 specification to include links and containers. | |||||||
> > | Version 1.1 aims to extend the VOSpace-1.0 specification to include links and containers. The proposed mechanism is that we introduce two new node types: | |||||||
Added: | ||||||||
> > |
| |||||||
At the May 2007 interop, we identified the following items as requiring resolution for VOSpace 1.1:
This is somewhere where we can post proposals and to enable interested parties to discuss the changes. Please add your suggestion to this page and vote on other suggestions
<--
|
Discussion of the VOSpace 1.1 specificationThis is a discussion page for the VOSpace-1.1 service specification. Version 1.1 aims to extend the VOSpace-1.0 specification to include links and containers. | ||||||||
Added: | ||||||||
> > |
At the May 2007 interop, we identified the following items as requiring resolution for VOSpace 1.1:
| |||||||
This is somewhere where we can post proposals and to enable interested parties to discuss the changes. Please add your suggestion to this page and vote on other suggestions
<--
|
| ||||||||
Added: | ||||||||
> > | VOSpace home page | |||||||
Discussion of the VOSpace 1.1 specificationThis is a discussion page for the VOSpace-1.1 service specification. Version 1.1 aims to extend the VOSpace-1.0 specification to include links and containers.This is somewhere where we can post proposals and to enable interested parties to discuss the changes. Please add your suggestion to this page and vote on other suggestions
<--
|
Discussion of the VOSpace 1.1 specificationThis is a discussion page for the VOSpace-1.1 service specification. | ||||||||
Changed: | ||||||||
< < | This is somewhere where we can post proposals and to enable interested parties to discuss the different versions. | |||||||
> > | Version 1.1 aims to extend the VOSpace-1.0 specification to include links and containers. | |||||||
Added: | ||||||||
> > |
This is somewhere where we can post proposals and to enable interested parties to discuss the changes. | |||||||
Please add your suggestion to this page and vote on other suggestions
<--
|
Discussion of the VOSpace 1.1 specificationThis is a discussion page for the VOSpace-1.1 service specification. This is somewhere where we can post proposals and to enable interested parties to discuss the different versions. Please add your suggestion to this page and vote on other suggestions
<--
|