VOSpace Brainstorm Meeting

This is a set of notes made during a meeting to brainstorm ideas on VOSpace. Those present in the meeting were:

• WilliamOMullane, GuyRixon, KeithNoddle, Dave Morris, NoelWinstanley, TonyLinde

and on the following telecon:

• RoyWilliams, JohnGood, MatthewGraham

See also:

Presentations

• MySpace structure & scenarios (Dave Morris): FileStore.html, FileManager.html
• Security aspects (GuyRixon): ppt, pdf

Meeting Notes

Dave's FileStore presn

• in Export: MIME type allowed for data conversion (eg FITS to jpeg)
  • as yet no negotiation
• question of about how dumb is the VOStore
  • contains metadata per item but does not interpret them
  • but must do so to implement security
• MyDB vs MySpace:
  • MySpace returns exact file put in
  • MyDB will translate into database and back again
  • so need to include flag that says 'store exactly'
• VOSpace vs VOStore
  • Wil: implement only VOStore first
  • AG: FileStore is dumb and FileManager has all intelligence (incl who owns items)
• coffee discussion
  • need layer above FileStore to implement security
  • VOBroker
  • but AG could just plug this into its FileStore --> VOStore

Dave's FileManager presn

• do we want to introduce replica management into the spec at this stage
• for now, AG only implements http protocol for filestore
  • but could implement other protocols
• need to define minimum set of protocols
• if http, need to ensure security
• need protocol negotiation
  • minimum:
    • SOAP with DIME attachment,
    • HTTP
      • needs to allow security
• VOSpace will support N protocols
  • will address many underlying filestores
  • client agent can state which protocol it wants to use
  • VOSpace will only pass back stores which support that protocol

Guy's Security presn

• Wil: users don't care about how security is implemented
  • Guy: implementers of VOStores will
• SAML as language for tickets?
  • yes, but not only option
• issue of which tickets are needed for an activity
  • requires policy store/check
• look at Shibboleth / PERMIS

Later discussion

• decided we do not need to standardise on VOSpace for now, just VOStore (see Agreements below)

Roy's Use Cases

1. Collaborative shared space
   A geographically-separated group is working on image-cleaning algorithms. Each member of the group can see the results from others and uploads the images they have built.
   The group can (if they wish) allow only group members to see the images.
2. Distributed file system
   A group can share data objects with each person using the client they prefer. Some like Linux, some Mac, some Microsoft. They can mount a filesystem, they can use a web browser, they can use scripted command line.
3. Object Listing
   To see a listing of all the data objects that are in the VOSpace owned by a given user or group, together with metadata such as size, mime-type, date, permissions, etc.
4. Asynchronous services
   A scientist uses NVO services asynchonously, so that results do not come back to the client as a service response packet, but rather they are put into VOStore. The results can be picked up later through a variety of client tools.
5. Workflow
   A user wishes to connect NVO services in a sequence, so that the output of one is the input for another. This can be implemented if one service writes a data object into VOStore, passes a VOStore reference to the other, who can then read and use the data object.
6. Distributed CasJobs
   CasJobs (http://casjobs.sdss.org/CasJobs/) is a browser-based environment for processing database tables. Users get an account, and results can be persisted in the "home space" that comes with the account. We would like to have geographically-distributed servers, so that a user sees the same home space whichever server is being used.
7. Widening the permission
   A user has created a private data object in a VOStore, and wishes to make it available to a wider audience by changing the permissions.
8. International
   A scientist has created a data table on the Astrogrid "MySpace" system, and would like to crossmatch it with another table that is in the NVO VOStore system.

Telecon

Agenda

1. run through discussions and agreements of meeting
2. use cases
3. VOStore specifics for:
   1. WebDAV
   2. SRB

Notes:

• re VOStore data return format
  • ?whether need this in the capability definition
  • three layers: VOStore, VOSpace, physical
    • will only standardise on VO levels
• subset or views of data
  • cannot include parameters in name of item
  • could do this by defining view name as item and change the view in the underlying database by another means
• naming
  • will use ivo://vostore_registry_id#vostore_identifier
  • Alternative: use endpoint of VOStore service instead of IVOID above. This would be problematic if using http as endpoint as cannot then discern web service from http; not recommended.
  • resolve AG ivorn
    • look up bit before # in registry and find service
    • send service bit after the #
• creating data item
  • do we allow space reservation
• transport protocol
  • must support http put/get
• security, identification, authentication
  • identity of person
    • can use ivoa identifier for community and look up user with id after #
  • can use certificates to sign into portal but back end authentication using tickets
  • tickets are signed SAML assertions
    • like proxy certificates

Meeting Agreements

• still only need VOSpace and VOStore layers
  • VOStore is higher level than AG FileStore
• do not need VOSpace protocol at this point
  • just need common definition of VOStore
    • includes list method
    • includes callback api (to know when asynch action finished)
  • projects implement own VOSpace
    • will look to standardise afterwards
• VOStore will advertise its capabilities (metadata in registry)
  • eg ability to return data in given format
    • AG filestore (currently) will only return in format uploaded
      • AG will extend filestore to support VOStore
    • MyDB will allow return in VOTable, CSV, FITS
• ticket-based security looks like way to go
  • subject to trial (see below)

Telecon agreements

• will not include ability to reserve space in VOStore
  • can be added to V2 later if necessary
• all VOStores must support http put/get as a minimum transport protocol
• will only work at data object level (not lower than that)

Action list

• Wil: send Dave latest version of VOStore: end of this week
• Dave: update VOStore doc with new methods: before May Interop
  • synch/asynch
  • init & data
  • get & put
• Guy/Wil: get prototype security mechanism working: before May Interop
• Dave/Wil: add Roy's use cases to VOStore document: before May Interop
  • expand one or two to expose how VOStore will work

Issues

• Group permissions: not addressed now
  • perhaps need to implement this in VOSpace

VOSpace Brainstorm Meeting

This is a set of notes made during a meeting to brainstorm ideas on VOSpace. Those present in the meeting were:

• WilliamOMullane, GuyRixon, KeithNoddle, Dave Morris, NoelWinstanley, TonyLinde

and on the following telecon:

• RoyWilliams, JohnGood, MatthewGraham

See also:

• http://www.ivoa.net/internal/IVOA/IvoaGridAndWebServices/vostore0.15.pdf

Presentations

• MySpace structure & scenarios (Dave Morris): FileStore.html, FileManager.html
• Security aspects (GuyRixon): ppt, pdf

Meeting Notes

Dave's FileStore presn

• in Export: MIME type allowed for data conversion (eg FITS to jpeg)
  • as yet no negotiation
• question of about how dumb is the VOStore
  • contains metadata per item but does not interpret them
  • but must do so to implement security
• MyDB vs MySpace:
  • MySpace returns exact file put in
  • MyDB will translate into database and back again
  • so need to include flag that says 'store exactly'
• VOSpace vs VOStore
  • Wil: implement only VOStore first
  • AG: FileStore is dumb and FileManager has all intelligence (incl who owns items)
• coffee discussion
  • need layer above FileStore to implement security
  • VOBroker
  • but AG could just plug this into its FileStore --> VOStore

Dave's FileManager presn

• do we want to introduce replica management into the spec at this stage
• for now, AG only implements http protocol for filestore
  • but could implement other protocols
• need to define minimum set of protocols
• if http, need to ensure security
• need protocol negotiation
  • minimum:
    • SOAP with DIME attachment,
    • HTTP
      • needs to allow security
• VOSpace will support N protocols
  • will address many underlying filestores
  • client agent can state which protocol it wants to use
  • VOSpace will only pass back stores which support that protocol

Guy's Security presn

• Wil: users don't care about how security is implemented
  • Guy: implementers of VOStores will
• SAML as language for tickets?
  • yes, but not only option
• issue of which tickets are needed for an activity
  • requires policy store/check
• look at Shibboleth / PERMIS

Later discussion

• decided we do not need to standardise on VOSpace for now, just VOStore (see Agreements below)

Roy's Use Cases

1. Collaborative shared space
   A geographically-separated group is working on image-cleaning algorithms. Each member of the group can see the results from others and uploads the images they have built.
   The group can (if they wish) allow only group members to see the images.
2. Distributed file system
   A group can share data objects with each person using the client they prefer. Some like Linux, some Mac, some Microsoft. They can mount a filesystem, they can use a web browser, they can use scripted command line.
3. Object Listing
   To see a listing of all the data objects that are in the VOSpace owned by a given user or group, together with metadata such as size, mime-type, date, permissions, etc.
4. Asynchronous services
   A scientist uses NVO services asynchonously, so that results do not come back to the client as a service response packet, but rather they are put into VOStore. The results can be picked up later through a variety of client tools.
5. Workflow
   A user wishes to connect NVO services in a sequence, so that the output of one is the input for another. This can be implemented if one service writes a data object into VOStore, passes a VOStore reference to the other, who can then read and use the data object.
6. Distributed CasJobs
   CasJobs (http://casjobs.sdss.org/CasJobs/) is a browser-based environment for processing database tables. Users get an account, and results can be persisted in the "home space" that comes with the account. We would like to have geographically-distributed servers, so that a user sees the same home space whichever server is being used.
7. Widening the permission
   A user has created a private data object in a VOStore, and wishes to make it available to a wider audience by changing the permissions.
8. International
   A scientist has created a data table on the Astrogrid "MySpace" system, and would like to crossmatch it with another table that is in the NVO VOStore system.

Telecon

Agenda

1. run through discussions and agreements of meeting
2. use cases
3. VOStore specifics for:
   1. WebDAV
   2. SRB

Notes:

• re VOStore data return format
  • ?whether need this in the capability definition
  • three layers: VOStore, VOSpace, physical
    • will only standardise on VO levels
• subset or views of data
  • cannot include parameters in name of item
  • could do this by defining view name as item and change the view in the underlying database by another means
• naming
  • will use ivo://vostore_registry_id#vostore_identifier
  • Alternative: use endpoint of VOStore service instead of IVOID above. This would be problematic if using http as endpoint as cannot then discern web service from http; not recommended.
  • resolve AG ivorn
    • look up bit before # in registry and find service
    • send service bit after the #
• creating data item
  • do we allow space reservation
• transport protocol
  • must support http put/get
• security, identification, authentication
  • identity of person
    • can use ivoa identifier for community and look up user with id after #
  • can use certificates to sign into portal but back end authentication using tickets
  • tickets are signed SAML assertions
    • like proxy certificates

Meeting Agreements

• still only need VOSpace and VOStore layers
  • VOStore is higher level than AG FileStore
• do not need VOSpace protocol at this point
  • just need common definition of VOStore
    • includes list method
    • includes callback api (to know when asynch action finished)
  • projects implement own VOSpace
    • will look to standardise afterwards
• VOStore will advertise its capabilities (metadata in registry)
  • eg ability to return data in given format
    • AG filestore (currently) will only return in format uploaded
      • AG will extend filestore to support VOStore
    • MyDB will allow return in VOTable, CSV, FITS
• ticket-based security looks like way to go
  • subject to trial (see below)

Telecon agreements

• will not include ability to reserve space in VOStore
  • can be added to V2 later if necessary
• all VOStores must support http put/get as a minimum transport protocol
• will only work at data object level (not lower than that)

Action list

• Wil: send Dave latest version of VOStore: end of this week
• Dave: update VOStore doc with new methods: before May Interop
  • synch/asynch
  • init & data
  • get & put
• Guy/Wil: get prototype security mechanism working: before May Interop
• Dave/Wil: add Roy's use cases to VOStore document: before May Interop
  • expand one or two to expose how VOStore will work

Issues

• Group permissions: not addressed now
  • perhaps need to implement this in VOSpace

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

VOSpace Brainstorm Meeting

This is a set of notes made during a meeting to brainstorm ideas on VOSpace. Those present in the meeting were:

• WilliamOMullane, GuyRixon, KeithNoddle, Dave Morris, NoelWinstanley, TonyLinde

and on the following telecon:

• RoyWilliams, JohnGood, MatthewGraham

See also:

• http://www.ivoa.net/internal/IVOA/IvoaGridAndWebServices/vostore0.15.pdf

Presentations

• MySpace structure & scenarios (Dave Morris): FileStore.html, FileManager.html
• Security aspects (GuyRixon): ppt, pdf

Meeting Notes

Dave's FileStore presn

• in Export: MIME type allowed for data conversion (eg FITS to jpeg)
  • as yet no negotiation
• question of about how dumb is the VOStore
  • contains metadata per item but does not interpret them
  • but must do so to implement security
• MyDB vs MySpace:
  • MySpace returns exact file put in
  • MyDB will translate into database and back again
  • so need to include flag that says 'store exactly'
• VOSpace vs VOStore
  • Wil: implement only VOStore first
  • AG: FileStore is dumb and FileManager has all intelligence (incl who owns items)
• coffee discussion
  • need layer above FileStore to implement security
  • VOBroker
  • but AG could just plug this into its FileStore --> VOStore

Dave's FileManager presn

• do we want to introduce replica management into the spec at this stage
• for now, AG only implements http protocol for filestore
  • but could implement other protocols
• need to define minimum set of protocols
• if http, need to ensure security
• need protocol negotiation
  • minimum:
    • SOAP with DIME attachment,
    • HTTP
      • needs to allow security
• VOSpace will support N protocols
  • will address many underlying filestores
  • client agent can state which protocol it wants to use
  • VOSpace will only pass back stores which support that protocol

Guy's Security presn

• Wil: users don't care about how security is implemented
  • Guy: implementers of VOStores will
• SAML as language for tickets?
  • yes, but not only option
• issue of which tickets are needed for an activity
  • requires policy store/check
• look at Shibboleth / PERMIS

Later discussion

• decided we do not need to standardise on VOSpace for now, just VOStore (see Agreements below)

Roy's Use Cases

1. Collaborative shared space
   A geographically-separated group is working on image-cleaning algorithms. Each member of the group can see the results from others and uploads the images they have built.
   The group can (if they wish) allow only group members to see the images.
2. Distributed file system
   A group can share data objects with each person using the client they prefer. Some like Linux, some Mac, some Microsoft. They can mount a filesystem, they can use a web browser, they can use scripted command line.
3. Object Listing
   To see a listing of all the data objects that are in the VOSpace owned by a given user or group, together with metadata such as size, mime-type, date, permissions, etc.
4. Asynchronous services
   A scientist uses NVO services asynchonously, so that results do not come back to the client as a service response packet, but rather they are put into VOStore. The results can be picked up later through a variety of client tools.
5. Workflow
   A user wishes to connect NVO services in a sequence, so that the output of one is the input for another. This can be implemented if one service writes a data object into VOStore, passes a VOStore reference to the other, who can then read and use the data object.
6. Distributed CasJobs
   CasJobs (http://casjobs.sdss.org/CasJobs/) is a browser-based environment for processing database tables. Users get an account, and results can be persisted in the "home space" that comes with the account. We would like to have geographically-distributed servers, so that a user sees the same home space whichever server is being used.
7. Widening the permission
   A user has created a private data object in a VOStore, and wishes to make it available to a wider audience by changing the permissions.
8. International
   A scientist has created a data table on the Astrogrid "MySpace" system, and would like to crossmatch it with another table that is in the NVO VOStore system.

Telecon

Agenda

1. run through discussions and agreements of meeting
2. use cases
3. VOStore specifics for:
   1. WebDAV
   2. SRB

Notes:

• re VOStore data return format
  • ?whether need this in the capability definition
  • three layers: VOStore, VOSpace, physical
    • will only standardise on VO levels
• subset or views of data
  • cannot include parameters in name of item
  • could do this by defining view name as item and change the view in the underlying database by another means
• naming
  • will use ivo://vostore_registry_id#vostore_identifier
  • Alternative: use endpoint of VOStore service instead of IVOID above. This would be problematic if using http as endpoint as cannot then discern web service from http; not recommended.
  • resolve AG ivorn
    • look up bit before # in registry and find service
    • send service bit after the #
• creating data item
  • do we allow space reservation
• transport protocol
  • must support http put/get
• security, identification, authentication
  • identity of person
    • can use ivoa identifier for community and look up user with id after #
  • can use certificates to sign into portal but back end authentication using tickets
  • tickets are signed SAML assertions
    • like proxy certificates

• still only need VOSpace and VOStore layers
  • VOStore is higher level than AG FileStore
• do not need VOSpace protocol at this point
  • just need common definition of VOStore
    • includes list method
    • includes callback api (to know when asynch action finished)
  • projects implement own VOSpace
    • will look to standardise afterwards
• VOStore will advertise its capabilities (metadata in registry)
  • eg ability to return data in given format
    • AG filestore (currently) will only return in format uploaded
      • AG will extend filestore to support VOStore
    • MyDB will allow return in VOTable, CSV, FITS
• ticket-based security looks like way to go
  • subject to trial (see below)

Telecon agreements

• will not include ability to reserve space in VOStore
  • can be added to V2 later if necessary
• all VOStores must support http put/get as a minimum transport protocol
• will only work at data object level (not lower than that)

Action list

• Wil: send Dave latest version of VOStore: end of this week
• Dave: update VOStore doc with new methods: before May Interop
  • synch/asynch
  • init & data
  • get & put
• Guy/Wil: get prototype security mechanism working: before May Interop
• Dave/Wil: add Roy's use cases to VOStore document: before May Interop
  • expand one or two to expose how VOStore will work

Issues

• Group permissions: not addressed now
  • perhaps need to implement this in VOSpace

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

VOSpace Brainstorm Meeting

This is a set of notes made during a meeting to brainstorm ideas on VOSpace. Those present in the meeting were:

• WilliamOMullane, GuyRixon, KeithNoddle, Dave Morris, NoelWinstanley, TonyLinde

and on the following telecon:

• RoyWilliams, JohnGood, MatthewGraham

See also:

• http://www.ivoa.net/internal/IVOA/IvoaGridAndWebServices/vostore0.15.pdf

Presentations

• MySpace structure & scenarios (Dave Morris): FileStore.html, FileManager.html
• Security aspects (GuyRixon): ppt, pdf

Meeting Notes

Dave's FileStore presn

• in Export: MIME type allowed for data conversion (eg FITS to jpeg)
  • as yet no negotiation
• question of about how dumb is the VOStore
  • contains metadata per item but does not interpret them
  • but must do so to implement security
• MyDB vs MySpace:
  • MySpace returns exact file put in
  • MyDB will translate into database and back again
  • so need to include flag that says 'store exactly'
• VOSpace vs VOStore
  • Wil: implement only VOStore first
  • AG: FileStore is dumb and FileManager has all intelligence (incl who owns items)
• coffee discussion
  • need layer above FileStore to implement security
  • VOBroker
  • but AG could just plug this into its FileStore --> VOStore

Dave's FileManager presn

• do we want to introduce replica management into the spec at this stage
• for now, AG only implements http protocol for filestore
  • but could implement other protocols
• need to define minimum set of protocols
• if http, need to ensure security
• need protocol negotiation
  • minimum:
    • SOAP with DIME attachment,
    • HTTP
      • needs to allow security
• VOSpace will support N protocols
  • will address many underlying filestores
  • client agent can state which protocol it wants to use
  • VOSpace will only pass back stores which support that protocol

Guy's Security presn

• Wil: users don't care about how security is implemented
  • Guy: implementers of VOStores will
• SAML as language for tickets?
  • yes, but not only option
• issue of which tickets are needed for an activity
  • requires policy store/check
• look at Shibboleth / PERMIS

Later discussion

• decided we do not need to standardise on VOSpace for now, just VOStore (see Agreements below)

Roy's Use Cases

1. Collaborative shared space
   A geographically-separated group is working on image-cleaning algorithms. Each member of the group can see the results from others and uploads the images they have built.
   The group can (if they wish) allow only group members to see the images.
2. Distributed file system
   A group can share data objects with each person using the client they prefer. Some like Linux, some Mac, some Microsoft. They can mount a filesystem, they can use a web browser, they can use scripted command line.
3. Object Listing
   To see a listing of all the data objects that are in the VOSpace owned by a given user or group, together with metadata such as size, mime-type, date, permissions, etc.
4. Asynchronous services
   A scientist uses NVO services asynchonously, so that results do not come back to the client as a service response packet, but rather they are put into VOStore. The results can be picked up later through a variety of client tools.
5. Workflow
   A user wishes to connect NVO services in a sequence, so that the output of one is the input for another. This can be implemented if one service writes a data object into VOStore, passes a VOStore reference to the other, who can then read and use the data object.
6. Distributed CasJobs
   CasJobs (http://casjobs.sdss.org/CasJobs/) is a browser-based environment for processing database tables. Users get an account, and results can be persisted in the "home space" that comes with the account. We would like to have geographically-distributed servers, so that a user sees the same home space whichever server is being used.
7. Widening the permission
   A user has created a private data object in a VOStore, and wishes to make it available to a wider audience by changing the permissions.
8. International
   A scientist has created a data table on the Astrogrid "MySpace" system, and would like to crossmatch it with another table that is in the NVO VOStore system.

Telecon

Agenda

1. run through discussions and agreements of meeting
2. use cases
3. VOStore specifics for:
   1. WebDAV
   2. SRB

Notes:

• re VOStore data return format
  • ?whether need this in the capability definition
  • three layers: VOStore, VOSpace, physical
    • will only standardise on VO levels
• subset or views of data
  • cannot include parameters in name of item
  • could do this by defining view name as item and change the view in the underlying database by another means
• naming
  • will use ivo://vostore_registry_id#vostore_identifier

• • resolve AG ivorn
    • look up bit before # in registry and find service
    • send service bit after the #
• creating data item
  • do we allow space reservation
• transport protocol
  • must support http put/get
• security, identification, authentication
  • identity of person
    • can use ivoa identifier for community and look up user with id after #
  • can use certificates to sign into portal but back end authentication using tickets
  • tickets are signed SAML assertions
    • like proxy certificates

Morning Agreements

• still only need VOSpace and VOStore layers
  • VOStore is higher level than AG FileStore
• do not need VOSpace protocol at this point
  • just need common definition of VOStore
    • includes list method
    • includes callback api (to know when asynch action finished)
  • projects implement own VOSpace
    • will look to standardise afterwards
• VOStore will advertise its capabilities (metadata in registry)
  • eg ability to return data in given format
    • AG filestore (currently) will only return in format uploaded
      • AG will extend filestore to support VOStore
    • MyDB will allow return in VOTable, CSV, FITS
• ticket-based security looks like way to go
  • subject to trial (see below)

Telecon agreements

• will not include ability to reserve space in VOStore
  • can be added to V2 later if necessary
• all VOStores must support http put/get as a minimum transport protocol
• will only work at data object level (not lower than that)

Action list

• Wil: send Dave latest version of VOStore: end of this week
• Dave: update VOStore doc with new methods: before May Interop
  • synch/asynch
  • init & data
  • get & put
• Guy/Wil: get prototype security mechanism working: before May Interop
• Dave/Wil: add Roy's use cases to VOStore document: before May Interop
  • expand one or two to expose how VOStore will work

Issues

• Group permissions: not addressed now
  • perhaps need to implement this in VOSpace

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

VOSpace Brainstorm Meeting

This is a set of notes made during a meeting to brainstorm ideas on VOSpace. Those present in the meeting were:

• WilliamOMullane, GuyRixon, KeithNoddle, Dave Morris, NoelWinstanley, TonyLinde

and on the following telecon:

• RoyWilliams, JohnGood, MatthewGraham

See also:

• http://www.ivoa.net/internal/IVOA/IvoaGridAndWebServices/vostore0.15.pdf

Presentations

• MySpace structure & scenarios (Dave Morris): FileStore.html, FileManager.html
• Security aspects (GuyRixon): ppt, pdf

Meeting Notes

Dave's FileStore presn

• in Export: MIME type allowed for data conversion (eg FITS to jpeg)
  • as yet no negotiation
• question of about how dumb is the VOStore
  • contains metadata per item but does not interpret them
  • but must do so to implement security
• MyDB vs MySpace:
  • MySpace returns exact file put in
  • MyDB will translate into database and back again
  • so need to include flag that says 'store exactly'
• VOSpace vs VOStore
  • Wil: implement only VOStore first
  • AG: FileStore is dumb and FileManager has all intelligence (incl who owns items)
• coffee discussion
  • need layer above FileStore to implement security
  • VOBroker
  • but AG could just plug this into its FileStore --> VOStore

Dave's FileManager presn

• do we want to introduce replica management into the spec at this stage
• for now, AG only implements http protocol for filestore
  • but could implement other protocols
• need to define minimum set of protocols
• if http, need to ensure security
• need protocol negotiation
  • minimum:
    • SOAP with DIME attachment,
    • HTTP
      • needs to allow security
• VOSpace will support N protocols
  • will address many underlying filestores
  • client agent can state which protocol it wants to use
  • VOSpace will only pass back stores which support that protocol

Guy's Security presn

• Wil: users don't care about how security is implemented
  • Guy: implementers of VOStores will
• SAML as language for tickets?
  • yes, but not only option
• issue of which tickets are needed for an activity
  • requires policy store/check
• look at Shibboleth / PERMIS

Later discussion

• decided we do not need to standardise on VOSpace for now, just VOStore (see Agreements below)

Roy's Use Cases

1. Collaborative shared space
   A geographically-separated group is working on image-cleaning algorithms. Each member of the group can see the results from others and uploads the images they have built.
   The group can (if they wish) allow only group members to see the images.
2. Distributed file system
   A group can share data objects with each person using the client they prefer. Some like Linux, some Mac, some Microsoft. They can mount a filesystem, they can use a web browser, they can use scripted command line.
3. Object Listing
   To see a listing of all the data objects that are in the VOSpace owned by a given user or group, together with metadata such as size, mime-type, date, permissions, etc.
4. Asynchronous services
   A scientist uses NVO services asynchonously, so that results do not come back to the client as a service response packet, but rather they are put into VOStore. The results can be picked up later through a variety of client tools.
5. Workflow
   A user wishes to connect NVO services in a sequence, so that the output of one is the input for another. This can be implemented if one service writes a data object into VOStore, passes a VOStore reference to the other, who can then read and use the data object.
6. Distributed CasJobs
   CasJobs (http://casjobs.sdss.org/CasJobs/) is a browser-based environment for processing database tables. Users get an account, and results can be persisted in the "home space" that comes with the account. We would like to have geographically-distributed servers, so that a user sees the same home space whichever server is being used.
7. Widening the permission
   A user has created a private data object in a VOStore, and wishes to make it available to a wider audience by changing the permissions.
8. International
   A scientist has created a data table on the Astrogrid "MySpace" system, and would like to crossmatch it with another table that is in the NVO VOStore system.

Telecon

Agenda

1. run through discussions and agreements of meeting
2. use cases
3. VOStore specifics for:
   1. WebDAV
   2. SRB

Notes:

• still only need VOSpace and VOStore layers
  • VOStore is higher level than AG FileStore
• do not need VOSpace protocol at this point
  • just need common definition of VOStore
    • includes list method
    • includes callback api (to know when asynch action finished)
  • projects implement own VOSpace
    • will look to standardise afterwards

• • eg ability to return data in given format
    • AG filestore (currently) will only return in format uploaded
      • AG will extend filestore to support VOStore
    • MyDB will allow return in VOTable, CSV, FITS
• ticket-based security looks like way to go
  • subject to trial (see below)

Action list

• Wil: send Dave latest version of VOStore: end of this week
• Dave: update VOStore doc with new methods: before May Interop
  • synch/asynch
  • init & data
  • get & put
• Guy/Wil: get prototype security mechanism working: before May Interop

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

VOSpace Brainstorm Meeting

This is a set of notes made during a meeting to brainstorm ideas on VOSpace. Those present in the meeting were:

• WilliamOMullane, GuyRixon, KeithNoddle, Dave Morris, NoelWinstanley, TonyLinde

and on the following telecon:

• RoyWilliams, JohnGood, MatthewGraham

See also:

• http://www.ivoa.net/internal/IVOA/IvoaGridAndWebServices/vostore0.15.pdf

Presentations

• MySpace structure & scenarios (Dave Morris): FileStore.html, FileManager.html
• Security aspects (GuyRixon): ppt, pdf

Dave's FileStore presn

• in Export: MIME type allowed for data conversion (eg FITS to jpeg)
  • as yet no negotiation
• question of about how dumb is the VOStore
  • contains metadata per item but does not interpret them
  • but must do so to implement security
• MyDB vs MySpace:
  • MySpace returns exact file put in
  • MyDB will translate into database and back again
  • so need to include flag that says 'store exactly'
• VOSpace vs VOStore
  • Wil: implement only VOStore first
  • AG: FileStore is dumb and FileManager has all intelligence (incl who owns items)
• coffee discussion
  • need layer above FileStore to implement security
  • VOBroker
  • but AG could just plug this into its FileStore --> VOStore

Dave's FileManager presn

• do we want to introduce replica management into the spec at this stage
• for now, AG only implements http protocol for filestore
  • but could implement other protocols
• need to define minimum set of protocols
• if http, need to ensure security
• need protocol negotiation
  • minimum:
    • SOAP with DIME attachment,
    • HTTP
      • needs to allow security
• VOSpace will support N protocols
  • will address many underlying filestores
  • client agent can state which protocol it wants to use
  • VOSpace will only pass back stores which support that protocol

Guy's Security presn

• Wil: users don't care about how security is implemented
  • Guy: implementers of VOStores will
• SAML as language for tickets?
  • yes, but not only option
• issue of which tickets are needed for an activity
  • requires policy store/check
• look at Shibboleth / PERMIS

Later discussion

• decided we do not need to standardise on VOSpace for now, just VOStore (see Agreements below)

Arguments

• whether we need transformation (RESOLVED with VOStore capabilities)
  • mandatory if you are putting from file to database
  • metadata must indicate capabilities of store

Agreements

• still only need VOSpace and VOStore layers
  • VOStore is higher level than AG FileStore
• do not need VOSpace protocol at this point
  • just need common definition of VOStore
    • includes list method
    • includes callback api (to know when asynch action finished)
  • projects implement own VOSpace
    • will look to standardise afterwards
• VOStore will advertise its capabilities
  • eg ability to return data in given format
    • AG filestore (currently) will only return in format uploaded
      • AG will extend filestore to support VOStore
    • MyDB will allow return in VOTable, CSV, FITS
• ticket-based security looks like way to go
  • subject to trial (see below)

Action list

• Wil: send Dave latest version of VOStore: end of this week
• Dave: update VOStore doc with new methods: before May Interop
  • synch/asynch
  • init & data
  • get & put
• Guy/Wil: get prototype security mechanism working: before May Interop

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

VOSpace Brainstorm Meeting

• WilliamOMullane, GuyRixon, KeithNoddle, Dave Morris, NoelWinstanley, TonyLinde

• RoyWilliams, JohnGood, MatthewGraham

Presentations

• MySpace structure & scenarios (Dave Morris): FileStore.html, FileManager.html

Notes

Dave's FileStore presn

• in Export: MIME type allowed for data conversion (eg FITS to jpeg)
  • as yet no negotiation
• question of about how dumb is the VOStore
  • contains metadata per item but does not interpret them
  • but must do so to implement security
• MyDB vs MySpace:
  • MySpace returns exact file put in
  • MyDB will translate into database and back again
  • so need to include flag that says 'store exactly'
• VOSpace vs VOStore
  • Wil: implement only VOStore first
  • AG: FileStore is dumb and FileManager has all intelligence (incl who owns items)
• coffee discussion
  • need layer above FileStore to implement security
  • VOBroker
  • but AG could just plug this into its FileStore --> VOStore

Dave's FileManager presn

• do we want to introduce replica management into the spec at this stage
• for now, AG only implements http protocol for filestore
  • but could implement other protocols
• need to define minimum set of protocols
• if http, need to ensure security
• need protocol negotiation
  • minimum:
    • SOAP with DIME attachment,
    • HTTP
      • needs to allow security
• VOSpace will support N protocols
  • will address many underlying filestores
  • client agent can state which protocol it wants to use
  • VOSpace will only pass back stores which support that protocol

Guy's Security presn

Arguments

• whether we need transformation (RESOLVED with VOStore capabilities)
  • mandatory if you are putting from file to database
  • metadata must indicate capabilities of store

Agreements

• • VOStore is higher level than AG FileStore

• VOStore will advertise its capabilities
  • eg ability to return data in given format
    • AG filestore (currently) will only return in format uploaded
      • AG will extend filestore to support VOStore
    • MyDB will allow return in VOTable, CSV, FITS

Action list

• • synch/asynch
  • init & data

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

VOSpace Brainstorm Meeting

• WilliamOMullane, GuyRixon, KeithNoddle, Dave Morris, NoelWinstanley, TonyLinde

• RoyWilliams, JohnGood, MatthewGraham

Presentations

• MySpace structure & scenarios (Dave Morris): FileStore.html, FileManager.html

Notes

Dave's FileStore presn

• in Export: MIME type allowed for data conversion (eg FITS to jpeg)
  • as yet no negotiation
• question of about how dumb is the VOStore
  • contains metadata per item but does not interpret them
  • but must do so to implement security
• MyDB vs MySpace:
  • MySpace returns exact file put in
  • MyDB will translate into database and back again
  • so need to include flag that says 'store exactly'
• VOSpace vs VOStore
  • Wil: implement only VOStore first
  • AG: FileStore is dumb and FileManager has all intelligence (incl who owns items)
• coffee discussion
  • need layer above FileStore to implement security
  • VOBroker
  • but AG could just plug this into its FileStore --> VOStore

Dave's FileManager presn

• do we want to introduce replica management into the spec at this stage
• for now, AG only implements http protocol for filestore
  • but could implement other protocols
• need to define minimum set of protocols
• if http, need to ensure security
• need protocol negotiation
  • minimum:
    • SOAP with DIME attachment,
    • HTTP
      • needs to allow security
• VOSpace will support N protocols
  • will address many underlying filestores
  • client agent can state which protocol it wants to use
  • VOSpace will only pass back stores which support that protocol

Guy's Security presn

Arguments

• whether we need transformation (RESOLVED with VOStore capabilities)
  • mandatory if you are putting from file to database
  • metadata must indicate capabilities of store

Agreements

• still only need VOSpace and VOStore protocol levels
  • VOStore is higher level than AG FileStore
• VOStore will advertise its capabilities
  • eg ability to return data in given format
    • AG filestore (currently) will only return in format uploaded
      • AG will extend filestore to support VOStore
    • MyDB will allow return in VOTable, CSV, FITS

Action list

• Dave: update VOStore doc with new methods
  • synch/asynch
  • init & data

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

VOSpace Brainstorm Meeting

• WilliamOMullane, GuyRixon, KeithNoddle, Dave Morris, NoelWinstanley, TonyLinde

• RoyWilliams, JohnGood, MatthewGraham

Presentations

• MySpace structure & scenarios (Dave Morris): FileStore.html, FileManager.html
• Security aspects (GuyRixon): [[][.ppt]]

Notes

Dave's FileStore presn

• in Export: MIME type allowed for data conversion (eg FITS to jpeg)
  • as yet no negotiation
• question of about how dumb is the VOStore
  • contains metadata per item but does not interpret them
  • but must do so to implement security
• MyDB vs MySpace:
  • MySpace returns exact file put in
  • MyDB will translate into database and back again

• VOSpace vs VOStore
  • Wil: implement only VOStore first
  • AG: FileStore is dumb and FileManager has all intelligence (incl who owns items)
• coffee discussion
  • need layer above FileStore to implement security
  • VOBroker
  • but AG could just plug this into its FileStore --> VOStore

Dave's FileManager presn

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

VOSpace Brainstorm Meeting

• WilliamOMullane, GuyRixon, KeithNoddle, Dave Morris, NoelWinstanley, TonyLinde

• RoyWilliams, JohnGood, MatthewGraham

Presentations

• MySpace structure & scenarios (Dave Morris): FileStore.html, FileManager.html
• Security aspects (GuyRixon): [[][.ppt]]

Notes

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

VOSpace Brainstorm Meeting

• WilliamOMullane, GuyRixon, KeithNoddle, Dave Morris, NoelWinstanley, TonyLinde

• RoyWilliams, JohnGood, MatthewGraham

Presentations

• MySpace structure & scenarios (Dave Morris): FileStore.html, FileManager.html
• Security aspects (GuyRixon): [[][.ppt]]

Notes

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->