AstroGrid FileStore

Dave Morris

AstroGrid

What Is FileStore ?

• It's a simple service that stores files
• Clients can get and put files
• Clients can ask the service to get and put files
• Uses SOAP for control messages, but data is transferred by direct connection

File identifiers

• Each file has a unique identifier, issued by the FileStore service
• Identifiers are based on the service IVO identifier, plus a unique file number
• e.g. ivo://ast.cam.ac.uk/filestore#1234-5678
• Identifiers are globally unique
• The service identifier is unique within the registry
• e.g. ivo://ast.cam.ac.uk/filestore
• The file number is unique within the service
• e.g. 1234-5678

Resolving identifiers

• Given a file identifier, clients can use the registry to locate a file
• e.g. ivo://ast.cam.ac.uk/filestore#1234-5678
• The client can resolve the FileStore service in the registry using the identifier
  
  service identifier >> registry entry
• The client can resolve the service endpoint from registry entry
  
  registry entry >> service endpoint
• The client can ask the service for access to the file
  
  service >> access URL

Client access

• It is possible for clients to contact the FileStore directly
• However, FileStore (VoStore) was designed to work in conjunction with the FileManager service (VoSpace)
• A client would normally contact a FileStore via a FileManager

Import init (async)

Client has data it wants to send to FileStore

• Client contacts FileStore and requests an import by calling importInit()
• Client may provide additional properties about the data e.g.
  • The MIME type
  • The preferred protocol and method **
  • The IVO identifier of a callback listener **
• FileStore allocates a new (empty) container for the data
• FileStore replies to client with
  • The identifier for the new container
  • The location to send the data to, and the protocol and method to use
• ---- SOAP call completed ----
• Client uses the URL to send the data to the FileStore
• FileStore notifies listener if callback address was specified in the request **

Import data (sync)

Client wants FileStore to fetch data from an external location

• Client contacts FileStore and requests an import by calling importData()
• Client must specify
  • The location of the data (URL)
  • The transfer protocol and method to use
• Client may provide additional properties about the data e.g.
  • The MIME type
• FileStore creates a new container and fetches the data from the URL
• FileStore replies to client with
  • The identifier for the new container
  • Additional properties describing the data it recieved e.g.
    • The size of the data
    • The date/time the transfer completed
• ---- SOAP call completed ----

Import data (async) **

Client wants FileStore to fetch data from an external location

• Client contacts FileStore and requests an import by calling importData()
• Client must specify
  • The location of the data (URL)
  • The transfer protocol and method to use
• Client may provide additional properties e.g.
  • The MIME type
  • the IVO identifier of a callback listener service
• FileStore creates a new container for the data and schedules the transfer to occur later
• FileStore replies to client with
  • The identifier for the new container
• ---- SOAP call completed ----
• FileStore fetches data from the URL and stores it in the container
• FileStore notifies listener if callback address was specified in the request **

Export init (async)

Client wants access to data in FileStore

• Client contacts FileStore and requests an export by calling exportInit()
• Client must specify
  • The identifier of the container
• Client can provide additional properties, including
  • The MIME type to use for the transfer **
  • The preferred protocol and method **
• FileStore replies to client with
  • The URL location to fetch the data from
  • The protocol and method to use
• ---- SOAP call completed ----
• The client uses the URL to access the data

Export data (sync)

Client wants FileStore to send data to a remote location

• Client contacts FileStore and requests an export by calling exportData()
• Client must specify
  • The identifier of the container
  • The location to send the data to
  • The transfer protocol and method to use
• Client may provide additional properties, including
  • The MIME type to use for the transfer **
• FileStore sends the data to the specified URL
• FileStore replies to client with
  • A set of properties describing the data that was sent
• ---- SOAP call completed ----

Export data (async)

Client wants FileStore to send data to a remote location

• Client contacts FileStore and requests an export by calling exportData()
• Client must specify
  • The identifier of the container
  • The location to send the data to
  • The transfer protocol and method to use
• Client may provide additional properties, including
  • The MIME type to use for the transfer**
  • The IVO identifier of a callback listener service **
• FileStore replies to client with
  • A set of properties describing the data that will be sent
• ---- SOAP call completed ----
• FileStore sends the data to the specified URL
• FileStore notifies listener if callback address was specified in the request **