Difference: InterOpOct2003Registry (1 vs. 14)

Revision 142007-01-19 - BrunoRino

 
META TOPICPARENT name="IvoaResReg"
This page will hold presentations, discussions and results of the Registry WG meeting at InterOpOct2003.

Presentations

Day 1:

Day 2:

Notes

many thanks to Anita for these notes from the Day 1 meeting

Registry IVOA notes  AMS Richards amsr@jb.man.ac.uk

11:00 2003 October 16

NB two Jan 2004 demos
- AAS
- AVO Munich 27-28 Jan

---------------------------------------------------------------
A IDENTIFIERS
Ray Plante See document on wiki V0.2

Register ID with some global authority to ensure uniqueness etc.

1.
Issues to be resolved through practice:

*
Mirrors/versions have different identifiers - harvesters need to know mirror etc. info too.

*
Permanence - can identifiers be recycled?

*
'ivo' xml format implies resource is _registered_ somewhere ADEC (organisation of NASA astrophysical data centres) adopted IVOA syntax. US data publishing lined to (refereed) journal papers

*
Need standard for logical identifiers - need to automatically point to relevant version. Are Identifiers 'logical' or static? Is there a contradiction here?


2.
FAQs
Who assigns IDs?	Publishing registry (check to avoid
			duplication)

What is ID content?	Conventions to encourage uniqueness but
			decision ultimately up to authority behind ID
			(as long as it is in valid characters).

Where register?		One place (so must have all registries
			accessible via global registry)

3.
Discussion
How to manage global requirements espec uniquness of ID?
How police?
Trust - duplicate won't be replicated so it won't work - practical disincentive. Need good explanation for data providers Learn what mechanisms are needed in practice e.g. Jan 2004 demo

**DECISION: Support recommendation to adopt IVOA Identifiers V0.2

---------------------------------------------------------------
B RESOURCE METADATA
Bob Hanisch see wiki doc V0.82

See Changes

creator removed from list of required elements,
**address/tel. should be removed

Title - verbose (e.g. Hubble Space Telescope) required ShortName optional

Keywords - ideally one source i.e. IAU Thesaurus
			  meanwhile will be more than one but must have namespace(?)
			  with definitions

Wavelength coverage - mm-wave added, submm may be needed?
				Suggestion - submm instead of mm?
							 e.g. 10 micron to 1 mm ????
				Can't be perfect, can overlap as necessarily

UCDs	  - will there be a problem with getting them two ways?
			  But they aren't compulsory. Having UCDs in registry if
			  required allows quicker searches in some cases. More
			  important, it is allows it to be in model.

Data Quality - agree outline of section, discuss details in light of
			 practice (e.g. distinguish quality of data
			 v. metadata).
			 Data provider assigns A B or C

**DECISION: Support recommendation to adopt IVOA RM v0.82

---------------------------------------------------------------

C
VOResource

Ray Plante v0.8.4

Covers most concepts in RM Working Doc
Extensions will add additional metdata for particular classes of resources as required.

NB See XML Schema style guide provided by Ray Plante

1
Metadata

Minimum required metadata just 7-8 elements

*
Identity
Service metadata - how to invoke/access, virtual address

*
Community

*
Data service - collection
		  - sky service
		  - tabular sky service

etc.

2
*Issues*

Use of STC metadata schema (Rots) ultimate goal
- Use temporary simple version for now

Using/reusing VOTable elements? for table columns, service parameters

>From discussion:
Duplication of services - how resolve

Coverage etc. - may need to introduce upper and lower limits?

Input registration of services, data collections etc.
		- need documentation and templates
		- for VO testers, basic xml templates probably most useful
		- easy web interface for wider input

3
What can we register?

4
Standardisation
- of each schema
- as a data model
- don't need PR version to use a schema (what does PR mean??) *must stabilise schemas*

---------------------------------------------------------------

D WEB SERVICES

Elizabeth Auden

1
How do services talk to each other?
Language not important as long as accessible to web services.

Queries
	Simple query
	Complex Query
	Cone search
	SIA
	+?
Admin

Query language - XQuery? Others also.

Changed:
<
<
- IvoaVOQL etc
>
>
- VOQL etc
  Returns AstroGrid uss XML just not Next VOTable (already in some use)

Whose problem is taking to back end e.g. data providers' BDs? Not IVOA?

Not everyone needs to use same language behind the scenes

2 Discussion

2a Does cone search need to be in Registry? Yes - can't assume it is always built in, need to register where to find specialised search/extraction services - many services are not built in to VOs e.g. extraacting spectra from x-ray data...

2b See Elizabeth's notes What do registry interfaces need to be able to do? i) Check authenticity ID with global registries ii) Register authority ID with global registry iii) Query the registry and get answer (xml-based in VOResource schema format) iv) Harvest the registry through an interface (incorporates replication?) - web service or httpGET? OAI (uses httpGET? - need one method for Jan 2004! * OAI * v) Define each registry interface web service method in WDSL vi) Documentation for data entry tools vii) Administer registry through add, delete, edit

**RECOMMENDATION: NVO, AstroGrid etc. registries share via OAI for Jan 2004


E REGISTRY HARVESTING

Keith Noddle

1. A registry is itself a resource holding metadata about itself.

- What level - global, local etc? - is registry open to query?

Definitions: Harvest gather metadata Replicate data

Basic security checks - need to follow path.

Do not want to have to choose which registry to search - need automatic referal from local to full registry when required - choice made automatically

2. Demo

a) Use HTTPGet (OAI) b) Specify 'since' date ie all changes since.... ???

3. Future consideration: - replication and harvesting - HTTPGet v. Web services or both - resource attributes - suppress updates if local resource doesn't want DoNotReplicate - delete DeleteResource - how propagate? - etc. ...Deprecated? Use of wrappers

4. Summary AAS Demo Jan2004

**DECISION Get on with it!


F DEMO ACTION Tony Linde

1. Standards Identifier, RM, Schema doc's to be adopted as above

2. Future Web service call interface Standard query language - SQL and XQuery needed Harvesting/replication

2. Demos a) Schema Will be based on Schema 0.9 ready ?20 Oct b) Harvesting Use OAI to return all resources since (date) as XML c) Goals Not - passing between registry projects But - registries must replicate ID and authority Maybe - external services e.g. cone search, data query, ACE

Are present schemas adequate for calling such services in an automated fashion

Need one standard way of getting metadata from a service. Add to schema how you get the full metadata for a resource (ie not just what is in registry) Need to investigate further.

d) Organisation of demos i) Who will publish registries AstroGrid AVO NVO GAVO CDS? + ? - Notify via mailing list

ii) What sort?

META FILEATTACHMENT attr="h" comment="" date="1066310134" name="IVOARegistry.ppt" path="IVOA Registry.ppt" size="260096" user="TonyLinde" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066310371" name="IVOARegistry.pdf" path="IVOA Registry.pdf" size="449628" user="TonyLinde" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312861" name="IdentifiersOct03.ppt" path="IdentifiersOct03.ppt" size="207360" user="RayPlante" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312899" name="IdentifiersOct03.pdf" path="IdentifiersOct03.pdf" size="100878" user="RayPlante" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312936" name="RMXMLOct03.ppt" path="RMXMLOct03.ppt" size="242176" user="RayPlante" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312973" name="RMXMLOct03.pdf" path="RMXMLOct03.pdf" size="117393" user="RayPlante" version="1.1"
META FILEATTACHMENT attr="" comment="Elizabeth Auden's slides: Registry Interface Calls" date="1066313930" name="IVOA_RegInterfaceCalls2.ppt" path="C:\Astrogrid\Presentations\IVOA_RegInterfaceCalls2.ppt" size="70144" user="TWikiGuest" version="/opsw/avo/rcs/bin/rlog -h /home/web/ivoa/docs/internal/IVOA/InterOpOct2003Registry/IVOA_RegInterfaceCalls2.ppt,v"
META FILEATTACHMENT attr="" comment="Resource metadata revisions" date="1066315583" name="RegistriesRM.ppt" path="C:\Documents and Settings\hanisch1.STSCI\My Documents\Talks_and_Papers\adass03\RegistriesRM.ppt" size="184320" user="BobHanisch" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066640080" name="IVOARegistry2.ppt" path="IVOA Registry 2.ppt" size="218624" user="TonyLinde" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066640103" name="IVOARegistry2.PDF" path="IVOA Registry2.PDF" size="495909" user="TonyLinde" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066652003" name="RegHarvest.pdf" path="RegHarvest.pdf" size="132964" user="KeithNoddle" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066652022" name="RegHarvest.sxi" path="RegHarvest.sxi" size="43002" user="KeithNoddle" version="1.1"

Revision 132007-01-17 - BrunoRino

 
META TOPICPARENT name="IvoaResReg"
Changed:
<
<
This page will hold presentations, discussions and results of the Registry WG meeting at InterOpOct2003.

Presentations

Day 1:

Day 2:

Notes

many thanks to Anita for these notes from the Day 1 meeting

Registry IVOA notes  AMS Richards amsr@jb.man.ac.uk

11:00 2003 October 16

NB two Jan 2004 demos
- AAS
- AVO Munich 27-28 Jan

---------------------------------------------------------------
A IDENTIFIERS
Ray Plante See document on wiki V0.2

Register ID with some global authority to ensure uniqueness etc.

1.
Issues to be resolved through practice:

*
Mirrors/versions have different identifiers - harvesters need to know mirror etc. info too.

*
Permanence - can identifiers be recycled?

*
'ivo' xml format implies resource is _registered_ somewhere ADEC (organisation of NASA astrophysical data centres) adopted IVOA syntax. US data publishing lined to (refereed) journal papers

*
Need standard for logical identifiers - need to automatically point to relevant version. Are Identifiers 'logical' or static? Is there a contradiction here?


2.
FAQs
Who assigns IDs?	Publishing registry (check to avoid
			duplication)

What is ID content?	Conventions to encourage uniqueness but
			decision ultimately up to authority behind ID
			(as long as it is in valid characters).

Where register?		One place (so must have all registries
			accessible via global registry)

3.
Discussion
How to manage global requirements espec uniquness of ID?
How police?
Trust - duplicate won't be replicated so it won't work - practical disincentive. Need good explanation for data providers Learn what mechanisms are needed in practice e.g. Jan 2004 demo

**DECISION: Support recommendation to adopt IVOA Identifiers V0.2

---------------------------------------------------------------
B RESOURCE METADATA
Bob Hanisch see wiki doc V0.82

See Changes

creator removed from list of required elements,
**address/tel. should be removed

Title - verbose (e.g. Hubble Space Telescope) required ShortName optional

Keywords - ideally one source i.e. IAU Thesaurus
			  meanwhile will be more than one but must have namespace(?)
			  with definitions

Wavelength coverage - mm-wave added, submm may be needed?
				Suggestion - submm instead of mm?
							 e.g. 10 micron to 1 mm ????
				Can't be perfect, can overlap as necessarily

UCDs	  - will there be a problem with getting them two ways?
			  But they aren't compulsory. Having UCDs in registry if
			  required allows quicker searches in some cases. More
			  important, it is allows it to be in model.

Data Quality - agree outline of section, discuss details in light of
			 practice (e.g. distinguish quality of data
			 v. metadata).
			 Data provider assigns A B or C

**DECISION: Support recommendation to adopt IVOA RM v0.82

---------------------------------------------------------------

C
VOResource

Ray Plante v0.8.4

Covers most concepts in RM Working Doc
Extensions will add additional metdata for particular classes of resources as required.

NB See XML Schema style guide provided by Ray Plante

1
Metadata

Minimum required metadata just 7-8 elements

*
Identity
Service metadata - how to invoke/access, virtual address

*
Community

*
Data service - collection
		  - sky service
		  - tabular sky service

etc.

2
*Issues*

Use of STC metadata schema (Rots) ultimate goal
- Use temporary simple version for now

Using/reusing VOTable elements? for table columns, service parameters

>From discussion:
Duplication of services - how resolve

Coverage etc. - may need to introduce upper and lower limits?

Input registration of services, data collections etc.
		- need documentation and templates
		- for VO testers, basic xml templates probably most useful
		- easy web interface for wider input

3
What can we register?

4
Standardisation
- of each schema
- as a data model
- don't need PR version to use a schema (what does PR mean??) *must stabilise schemas*

---------------------------------------------------------------

D WEB SERVICES

Elizabeth Auden

1
How do services talk to each other?
Language not important as long as accessible to web services.

Queries
	Simple query
	Complex Query
	Cone search
	SIA
	+?
Admin

Query language - XQuery? Others also.
			 - VOQL etc

Returns
	AstroGrid uss XML just not
	Next VOTable (already in some use)

Whose problem is taking to back end e.g. data providers' BDs? Not IVOA?

Not everyone needs to use same language behind the scenes

2
Discussion

2a
Does cone search need to be in Registry? Yes - can't assume it is always built in, need to register where to find specialised search/extraction services - many services are not built in to VOs e.g. extraacting spectra from x-ray data...

2b
See Elizabeth's notes
What do registry interfaces need to be able to do?
i)	Check authenticity ID with global registries
ii)  Register authority ID with global registry
iii) Query the registry and get answer (xml-based in
	  VOResource schema format)
iv)  Harvest the registry through an interface (incorporates
	  replication?)  - web service or httpGET? OAI (uses httpGET?
	  - need one method for Jan 2004! ** OAI **
v)	Define each registry interface web service method in WDSL
vi)  Documentation for data entry tools
vii) Administer registry through add, delete, edit

**RECOMMENDATION: NVO, AstroGrid etc. registries share via OAI for
	Jan 2004

---------------------------------------------------------------

E REGISTRY HARVESTING

Keith Noddle

1.
A registry is itself a resource holding metadata about itself.

- What level - global, local etc? - is registry open to query?

Definitions:
_Harvest_ gather metadata
_Replicate_ data

Basic security checks - need to follow path.

Do not want to have to choose which registry to search
- need automatic referal from local to full registry when required
- choice made automatically

2.
Demo

a) Use HTTPGet (OAI)
b) Specify 'since' date ie all changes since....
???

3.
Future consideration:
- replication and harvesting
  - HTTPGet v. Web services or both
- resource attributes
  - suppress updates if local resource doesn't want DoNotReplicate
  - delete DeleteResource - how propagate?
  - etc. ...Deprecated?
Use of wrappers

4.
Summary
AAS Demo Jan2004

**DECISION Get on with it!

---------------------------------------------------------------
F
DEMO ACTION
Tony Linde

1.
Standards
Identifier, RM, Schema doc's to be adopted as above

2.
Future
Web service call interface
Standard query language - SQL and XQuery needed Harvesting/replication

2.
Demos
a) Schema
	Will be based on Schema 0.9 ready ?20 Oct
b) Harvesting
	Use OAI to return all resources since (date) as XML
c) Goals
Not	- passing between registry projects
But	- registries must replicate ID and authority
Maybe	- external services e.g. cone search, data query, ACE

Are present schemas adequate for calling such services _in an automated fashion_

Need one standard way of getting metadata from a service.
*Add to schema* how you get the full metadata for a resource (ie not just what is in registry) Need to investigate further.

d) Organisation of demos
i)	Who will publish registries
	  AstroGrid AVO NVO GAVO CDS? + ?
	  - _Notify via mailing list_

ii)  What sort?


>
>
This page will hold presentations, discussions and results of the Registry WG meeting at InterOpOct2003.

Presentations

Day 1:

Day 2:

Notes

many thanks to Anita for these notes from the Day 1 meeting

Registry IVOA notes  AMS Richards amsr@jb.man.ac.uk

11:00 2003 October 16

NB two Jan 2004 demos
- AAS
- AVO Munich 27-28 Jan

---------------------------------------------------------------
A IDENTIFIERS
Ray Plante See document on wiki V0.2

Register ID with some global authority to ensure uniqueness etc.

1.
Issues to be resolved through practice:

*
Mirrors/versions have different identifiers - harvesters need to know mirror etc. info too.

*
Permanence - can identifiers be recycled?

*
'ivo' xml format implies resource is _registered_ somewhere ADEC (organisation of NASA astrophysical data centres) adopted IVOA syntax. US data publishing lined to (refereed) journal papers

*
Need standard for logical identifiers - need to automatically point to relevant version. Are Identifiers 'logical' or static? Is there a contradiction here?


2.
FAQs
Who assigns IDs?	Publishing registry (check to avoid
			duplication)

What is ID content?	Conventions to encourage uniqueness but
			decision ultimately up to authority behind ID
			(as long as it is in valid characters).

Where register?		One place (so must have all registries
			accessible via global registry)

3.
Discussion
How to manage global requirements espec uniquness of ID?
How police?
Trust - duplicate won't be replicated so it won't work - practical disincentive. Need good explanation for data providers Learn what mechanisms are needed in practice e.g. Jan 2004 demo

**DECISION: Support recommendation to adopt IVOA Identifiers V0.2

---------------------------------------------------------------
B RESOURCE METADATA
Bob Hanisch see wiki doc V0.82

See Changes

creator removed from list of required elements,
**address/tel. should be removed

Title - verbose (e.g. Hubble Space Telescope) required ShortName optional

Keywords - ideally one source i.e. IAU Thesaurus
			  meanwhile will be more than one but must have namespace(?)
			  with definitions

Wavelength coverage - mm-wave added, submm may be needed?
				Suggestion - submm instead of mm?
							 e.g. 10 micron to 1 mm ????
				Can't be perfect, can overlap as necessarily

UCDs	  - will there be a problem with getting them two ways?
			  But they aren't compulsory. Having UCDs in registry if
			  required allows quicker searches in some cases. More
			  important, it is allows it to be in model.

Data Quality - agree outline of section, discuss details in light of
			 practice (e.g. distinguish quality of data
			 v. metadata).
			 Data provider assigns A B or C

**DECISION: Support recommendation to adopt IVOA RM v0.82

---------------------------------------------------------------

C
VOResource

Ray Plante v0.8.4

Covers most concepts in RM Working Doc
Extensions will add additional metdata for particular classes of resources as required.

NB See XML Schema style guide provided by Ray Plante

1
Metadata

Minimum required metadata just 7-8 elements

*
Identity
Service metadata - how to invoke/access, virtual address

*
Community

*
Data service - collection
		  - sky service
		  - tabular sky service

etc.

2
*Issues*

Use of STC metadata schema (Rots) ultimate goal
- Use temporary simple version for now

Using/reusing VOTable elements? for table columns, service parameters

>From discussion:
Duplication of services - how resolve

Coverage etc. - may need to introduce upper and lower limits?

Input registration of services, data collections etc.
		- need documentation and templates
		- for VO testers, basic xml templates probably most useful
		- easy web interface for wider input

3
What can we register?

4
Standardisation
- of each schema
- as a data model
- don't need PR version to use a schema (what does PR mean??) *must stabilise schemas*

---------------------------------------------------------------

D WEB SERVICES

Elizabeth Auden

1
How do services talk to each other?
Language not important as long as accessible to web services.

Queries
	Simple query
	Complex Query
	Cone search
	SIA
	+?
Admin

Query language - XQuery? Others also.
			 - IvoaVOQL etc

Returns
	AstroGrid uss XML just not
	Next VOTable (already in some use)

Whose problem is taking to back end e.g. data providers' BDs? Not IVOA?

Not everyone needs to use same language behind the scenes

2
Discussion

2a
Does cone search need to be in Registry? Yes - can't assume it is always built in, need to register where to find specialised search/extraction services - many services are not built in to VOs e.g. extraacting spectra from x-ray data...

2b
See Elizabeth's notes
What do registry interfaces need to be able to do?
i)	Check authenticity ID with global registries
ii)  Register authority ID with global registry
iii) Query the registry and get answer (xml-based in
	  VOResource schema format)
iv)  Harvest the registry through an interface (incorporates
	  replication?)  - web service or httpGET? OAI (uses httpGET?
	  - need one method for Jan 2004! ** OAI **
v)	Define each registry interface web service method in WDSL
vi)  Documentation for data entry tools
vii) Administer registry through add, delete, edit

**RECOMMENDATION: NVO, AstroGrid etc. registries share via OAI for
	Jan 2004

---------------------------------------------------------------

E REGISTRY HARVESTING

Keith Noddle

1.
A registry is itself a resource holding metadata about itself.

- What level - global, local etc? - is registry open to query?

Definitions:
_Harvest_ gather metadata
_Replicate_ data

Basic security checks - need to follow path.

Do not want to have to choose which registry to search
- need automatic referal from local to full registry when required
- choice made automatically

2.
Demo

a) Use HTTPGet (OAI)
b) Specify 'since' date ie all changes since....
???

3.
Future consideration:
- replication and harvesting
  - HTTPGet v. Web services or both
- resource attributes
  - suppress updates if local resource doesn't want DoNotReplicate
  - delete DeleteResource - how propagate?
  - etc. ...Deprecated?
Use of wrappers

4.
Summary
AAS Demo Jan2004

**DECISION Get on with it!

---------------------------------------------------------------
F
DEMO ACTION
Tony Linde

1.
Standards
Identifier, RM, Schema doc's to be adopted as above

2.
Future
Web service call interface
Standard query language - SQL and XQuery needed Harvesting/replication

2.
Demos
a) Schema
	Will be based on Schema 0.9 ready ?20 Oct
b) Harvesting
	Use OAI to return all resources since (date) as XML
c) Goals
Not	- passing between registry projects
But	- registries must replicate ID and authority
Maybe	- external services e.g. cone search, data query, ACE

Are present schemas adequate for calling such services _in an automated fashion_

Need one standard way of getting metadata from a service.
*Add to schema* how you get the full metadata for a resource (ie not just what is in registry) Need to investigate further.

d) Organisation of demos
i)	Who will publish registries
	  AstroGrid AVO NVO GAVO CDS? + ?
	  - _Notify via mailing list_

ii)  What sort?


 
META FILEATTACHMENT attr="h" comment="" date="1066310134" name="IVOARegistry.ppt" path="IVOA Registry.ppt" size="260096" user="TonyLinde" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066310371" name="IVOARegistry.pdf" path="IVOA Registry.pdf" size="449628" user="TonyLinde" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312861" name="IdentifiersOct03.ppt" path="IdentifiersOct03.ppt" size="207360" user="RayPlante" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312899" name="IdentifiersOct03.pdf" path="IdentifiersOct03.pdf" size="100878" user="RayPlante" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312936" name="RMXMLOct03.ppt" path="RMXMLOct03.ppt" size="242176" user="RayPlante" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312973" name="RMXMLOct03.pdf" path="RMXMLOct03.pdf" size="117393" user="RayPlante" version="1.1"
META FILEATTACHMENT attr="" comment="Elizabeth Auden's slides: Registry Interface Calls" date="1066313930" name="IVOA_RegInterfaceCalls2.ppt" path="C:\Astrogrid\Presentations\IVOA_RegInterfaceCalls2.ppt" size="70144" user="TWikiGuest" version="/opsw/avo/rcs/bin/rlog -h /home/web/ivoa/docs/internal/IVOA/InterOpOct2003Registry/IVOA_RegInterfaceCalls2.ppt,v"
META FILEATTACHMENT attr="" comment="Resource metadata revisions" date="1066315583" name="RegistriesRM.ppt" path="C:\Documents and Settings\hanisch1.STSCI\My Documents\Talks_and_Papers\adass03\RegistriesRM.ppt" size="184320" user="BobHanisch" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066640080" name="IVOARegistry2.ppt" path="IVOA Registry 2.ppt" size="218624" user="TonyLinde" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066640103" name="IVOARegistry2.PDF" path="IVOA Registry2.PDF" size="495909" user="TonyLinde" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066652003" name="RegHarvest.pdf" path="RegHarvest.pdf" size="132964" user="KeithNoddle" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066652022" name="RegHarvest.sxi" path="RegHarvest.sxi" size="43002" user="KeithNoddle" version="1.1"

Revision 122003-10-21 - MarcoLeoni

 
META TOPICPARENT name="IvoaResReg"
This page will hold presentations, discussions and results of the Registry WG meeting at InterOpOct2003.

Presentations

Day 1:

Changed:
<
<
>
>
 

Day 2:

Notes

many thanks to Anita for these notes from the Day 1 meeting

Registry IVOA notes  AMS Richards amsr@jb.man.ac.uk

11:00 2003 October 16

NB two Jan 2004 demos
- AAS
- AVO Munich 27-28 Jan

---------------------------------------------------------------
A IDENTIFIERS
Ray Plante See document on wiki V0.2

Register ID with some global authority to ensure uniqueness etc.

1.
Issues to be resolved through practice:

*
Mirrors/versions have different identifiers - harvesters need to know mirror etc. info too.

*
Permanence - can identifiers be recycled?

*
'ivo' xml format implies resource is _registered_ somewhere ADEC (organisation of NASA astrophysical data centres) adopted IVOA syntax. US data publishing lined to (refereed) journal papers

*
Need standard for logical identifiers - need to automatically point to relevant version. Are Identifiers 'logical' or static? Is there a contradiction here?


2.
FAQs
Who assigns IDs?	Publishing registry (check to avoid
			duplication)

What is ID content?	Conventions to encourage uniqueness but
			decision ultimately up to authority behind ID
			(as long as it is in valid characters).

Where register?		One place (so must have all registries
			accessible via global registry)

3.
Discussion
How to manage global requirements espec uniquness of ID?
How police?
Trust - duplicate won't be replicated so it won't work - practical disincentive. Need good explanation for data providers Learn what mechanisms are needed in practice e.g. Jan 2004 demo

**DECISION: Support recommendation to adopt IVOA Identifiers V0.2

---------------------------------------------------------------
B RESOURCE METADATA
Bob Hanisch see wiki doc V0.82

See Changes

creator removed from list of required elements,
**address/tel. should be removed

Title - verbose (e.g. Hubble Space Telescope) required ShortName optional

Keywords - ideally one source i.e. IAU Thesaurus
			  meanwhile will be more than one but must have namespace(?)
			  with definitions

Wavelength coverage - mm-wave added, submm may be needed?
				Suggestion - submm instead of mm?
							 e.g. 10 micron to 1 mm ????
				Can't be perfect, can overlap as necessarily

UCDs	  - will there be a problem with getting them two ways?
			  But they aren't compulsory. Having UCDs in registry if
			  required allows quicker searches in some cases. More
			  important, it is allows it to be in model.

Data Quality - agree outline of section, discuss details in light of
			 practice (e.g. distinguish quality of data
			 v. metadata).
			 Data provider assigns A B or C

**DECISION: Support recommendation to adopt IVOA RM v0.82

---------------------------------------------------------------

C
VOResource

Ray Plante v0.8.4

Covers most concepts in RM Working Doc
Extensions will add additional metdata for particular classes of resources as required.

NB See XML Schema style guide provided by Ray Plante

1
Metadata

Minimum required metadata just 7-8 elements

*
Identity
Service metadata - how to invoke/access, virtual address

*
Community

*
Data service - collection
		  - sky service
		  - tabular sky service

etc.

2
*Issues*

Use of STC metadata schema (Rots) ultimate goal
- Use temporary simple version for now

Using/reusing VOTable elements? for table columns, service parameters

>From discussion:
Duplication of services - how resolve

Coverage etc. - may need to introduce upper and lower limits?

Input registration of services, data collections etc.
		- need documentation and templates
		- for VO testers, basic xml templates probably most useful
		- easy web interface for wider input

3
What can we register?

4
Standardisation
- of each schema
- as a data model
- don't need PR version to use a schema (what does PR mean??) *must stabilise schemas*

---------------------------------------------------------------

D WEB SERVICES

Elizabeth Auden

1
How do services talk to each other?
Language not important as long as accessible to web services.

Queries
	Simple query
	Complex Query
	Cone search
	SIA
	+?
Admin

Query language - XQuery? Others also.
			 - VOQL etc

Returns
	AstroGrid uss XML just not
	Next VOTable (already in some use)

Whose problem is taking to back end e.g. data providers' BDs? Not IVOA?

Not everyone needs to use same language behind the scenes

2
Discussion

2a
Does cone search need to be in Registry? Yes - can't assume it is always built in, need to register where to find specialised search/extraction services - many services are not built in to VOs e.g. extraacting spectra from x-ray data...

2b
See Elizabeth's notes
What do registry interfaces need to be able to do?
i)	Check authenticity ID with global registries
ii)  Register authority ID with global registry
iii) Query the registry and get answer (xml-based in
	  VOResource schema format)
iv)  Harvest the registry through an interface (incorporates
	  replication?)  - web service or httpGET? OAI (uses httpGET?
	  - need one method for Jan 2004! ** OAI **
v)	Define each registry interface web service method in WDSL
vi)  Documentation for data entry tools
vii) Administer registry through add, delete, edit

**RECOMMENDATION: NVO, AstroGrid etc. registries share via OAI for
	Jan 2004

---------------------------------------------------------------

E REGISTRY HARVESTING

Keith Noddle

1.
A registry is itself a resource holding metadata about itself.

- What level - global, local etc? - is registry open to query?

Definitions:
_Harvest_ gather metadata
_Replicate_ data

Basic security checks - need to follow path.

Do not want to have to choose which registry to search
- need automatic referal from local to full registry when required
- choice made automatically

2.
Demo

a) Use HTTPGet (OAI)
b) Specify 'since' date ie all changes since....
???

3.
Future consideration:
- replication and harvesting
  - HTTPGet v. Web services or both
- resource attributes
  - suppress updates if local resource doesn't want DoNotReplicate
  - delete DeleteResource - how propagate?
  - etc. ...Deprecated?
Use of wrappers

4.
Summary
AAS Demo Jan2004

**DECISION Get on with it!

---------------------------------------------------------------
F
DEMO ACTION
Tony Linde

1.
Standards
Identifier, RM, Schema doc's to be adopted as above

2.
Future
Web service call interface
Standard query language - SQL and XQuery needed Harvesting/replication

2.
Demos
a) Schema
	Will be based on Schema 0.9 ready ?20 Oct
b) Harvesting
	Use OAI to return all resources since (date) as XML
c) Goals
Not	- passing between registry projects
But	- registries must replicate ID and authority
Maybe	- external services e.g. cone search, data query, ACE

Are present schemas adequate for calling such services _in an automated fashion_

Need one standard way of getting metadata from a service.
*Add to schema* how you get the full metadata for a resource (ie not just what is in registry) Need to investigate further.

d) Organisation of demos
i)	Who will publish registries
	  AstroGrid AVO NVO GAVO CDS? + ?
	  - _Notify via mailing list_

ii)  What sort?


|*META FILEATTACHMENT*|attr="h" comment="" date="1066310134" name="IVOARegistry.ppt" path="IVOA Registry.ppt" size="260096" user="TonyLinde" version="1.1"|
|*META FILEATTACHMENT*|attr="h" comment="" date="1066310371" name="IVOARegistry.pdf" path="IVOA Registry.pdf" size="449628" user="TonyLinde" version="1.1"|
|*META FILEATTACHMENT*|attr="" comment="" date="1066312861" name="IdentifiersOct03.ppt" path="IdentifiersOct03.ppt" size="207360" user="RayPlante" version="1.1"|
|*META FILEATTACHMENT*|attr="" comment="" date="1066312899" name="IdentifiersOct03.pdf" path="IdentifiersOct03.pdf" size="100878" user="RayPlante" version="1.1"|
|*META FILEATTACHMENT*|attr="" comment="" date="1066312936" name="RMXMLOct03.ppt" path="RMXMLOct03.ppt" size="242176" user="RayPlante" version="1.1"|
|*META FILEATTACHMENT*|attr="" comment="" date="1066312973" name="RMXMLOct03.pdf" path="RMXMLOct03.pdf" size="117393" user="RayPlante" version="1.1"|
|*META FILEATTACHMENT*|attr="" comment="Elizabeth Auden's slides: Registry Interface Calls" date="1066313930" name="IVOA_RegInterfaceCalls2.ppt" path="C:\Astrogrid\Presentations\IVOA_RegInterfaceCalls2.ppt" size="70144" user="TWikiGuest" version="/opsw/avo/rcs/bin/rlog  -h /home/web/ivoa/docs/internal/IVOA/InterOpOct2003Registry/IVOA_RegInterfaceCalls2.ppt,v"|
|*META FILEATTACHMENT*|attr="" comment="Resource metadata revisions" date="1066315583" name="RegistriesRM.ppt" path="C:\Documents and Settings\hanisch1.STSCI\My Documents\Talks_and_Papers\adass03\RegistriesRM.ppt" size="184320" user="BobHanisch" version="1.1"|
|*META FILEATTACHMENT*|attr="h" comment="" date="1066640080" name="IVOARegistry2.ppt" path="IVOA Registry 2.ppt" size="218624" user="TonyLinde" version="1.1"|
|*META FILEATTACHMENT*|attr="h" comment="" date="1066640103" name="IVOARegistry2.PDF" path="IVOA Registry2.PDF" size="495909" user="TonyLinde" version="1.1"|
|*META FILEATTACHMENT*|attr="h" comment="" date="1066652003" name="RegHarvest.pdf" path="RegHarvest.pdf" size="132964" user="KeithNoddle" version="1.1"|
|*META FILEATTACHMENT*|attr="h" comment="" date="1066652022" name="RegHarvest.sxi" path="RegHarvest.sxi" size="43002" user="KeithNoddle" version="1.1"|

Revision 112003-10-21 - MarcoLeoni

 
META TOPICPARENT name="IvoaResReg"
This page will hold presentations, discussions and results of the Registry WG meeting at InterOpOct2003.

Presentations

Day 1:

Day 2:

Notes

many thanks to Anita for these notes from the Day 1 meeting

Registry IVOA notes  AMS Richards amsr@jb.man.ac.uk

11:00 2003 October 16

NB two Jan 2004 demos
- AAS
- AVO Munich 27-28 Jan

---------------------------------------------------------------
A IDENTIFIERS
Ray Plante See document on wiki V0.2

Register ID with some global authority to ensure uniqueness etc.

1.
Issues to be resolved through practice:

*
Mirrors/versions have different identifiers - harvesters need to know mirror etc. info too.

*
Permanence - can identifiers be recycled?

*
'ivo' xml format implies resource is _registered_ somewhere ADEC (organisation of NASA astrophysical data centres) adopted IVOA syntax. US data publishing lined to (refereed) journal papers

*
Need standard for logical identifiers - need to automatically point to relevant version. Are Identifiers 'logical' or static? Is there a contradiction here?


2.
FAQs
Who assigns IDs?	Publishing registry (check to avoid
			duplication)

What is ID content?	Conventions to encourage uniqueness but
			decision ultimately up to authority behind ID
			(as long as it is in valid characters).

Where register?		One place (so must have all registries
			accessible via global registry)

3.
Discussion
How to manage global requirements espec uniquness of ID?
How police?
Trust - duplicate won't be replicated so it won't work - practical disincentive. Need good explanation for data providers Learn what mechanisms are needed in practice e.g. Jan 2004 demo

**DECISION: Support recommendation to adopt IVOA Identifiers V0.2

---------------------------------------------------------------
B RESOURCE METADATA
Bob Hanisch see wiki doc V0.82

See Changes

creator removed from list of required elements,
**address/tel. should be removed

Changed:
<
<
Title - verbose (e.g. Hubble Space Telescope) required ShortName optional
>
>
Title - verbose (e.g. Hubble Space Telescope) required ShortName optional
  Keywords - ideally one source i.e. IAU Thesaurus meanwhile will be more than one but must have namespace(?) with definitions

Wavelength coverage - mm-wave added, submm may be needed? Suggestion - submm instead of mm? e.g. 10 micron to 1 mm ???? Can't be perfect, can overlap as necessarily

UCDs - will there be a problem with getting them two ways? But they aren't compulsory. Having UCDs in registry if required allows quicker searches in some cases. More important, it is allows it to be in model.

Data Quality - agree outline of section, discuss details in light of practice (e.g. distinguish quality of data v. metadata). Data provider assigns A B or C

**DECISION: Support recommendation to adopt IVOA RM v0.82


C VOResource

Ray Plante v0.8.4

Covers most concepts in RM Working Doc Extensions will add additional metdata for particular classes of resources as required.

NB See XML Schema style guide provided by Ray Plante

1 Metadata

Minimum required metadata just 7-8 elements

* Identity Service metadata - how to invoke/access, virtual address

* Community

* Data service - collection - sky service - tabular sky service

etc.

2 Issues

Use of STC metadata schema (Rots) ultimate goal - Use temporary simple version for now

Using/reusing VOTable elements? for table columns, service parameters

>From discussion:
Duplication of services - how resolve

Coverage etc. - may need to introduce upper and lower limits?

Input registration of services, data collections etc. - need documentation and templates - for VO testers, basic xml templates probably most useful - easy web interface for wider input

3 What can we register?

4 Standardisation - of each schema - as a data model - don't need PR version to use a schema (what does PR mean??) must stabilise schemas


D WEB SERVICES

Elizabeth Auden

1 How do services talk to each other? Language not important as long as accessible to web services.

Queries Simple query Complex Query Cone search SIA +? Admin

Query language - XQuery? Others also. - VOQL etc

Returns AstroGrid uss XML just not Next VOTable (already in some use)

Whose problem is taking to back end e.g. data providers' BDs? Not IVOA?

Not everyone needs to use same language behind the scenes

2 Discussion

2a Does cone search need to be in Registry? Yes - can't assume it is always built in, need to register where to find specialised search/extraction services - many services are not built in to VOs e.g. extraacting spectra from x-ray data...

2b See Elizabeth's notes What do registry interfaces need to be able to do? i) Check authenticity ID with global registries ii) Register authority ID with global registry iii) Query the registry and get answer (xml-based in VOResource schema format) iv) Harvest the registry through an interface (incorporates replication?) - web service or httpGET? OAI (uses httpGET? - need one method for Jan 2004! * OAI * v) Define each registry interface web service method in WDSL vi) Documentation for data entry tools vii) Administer registry through add, delete, edit

**RECOMMENDATION: NVO, AstroGrid etc. registries share via OAI for Jan 2004


E REGISTRY HARVESTING

Keith Noddle

1. A registry is itself a resource holding metadata about itself.

- What level - global, local etc? - is registry open to query?

Definitions: Harvest gather metadata Replicate data

Basic security checks - need to follow path.

Do not want to have to choose which registry to search - need automatic referal from local to full registry when required - choice made automatically

2. Demo

a) Use HTTPGet (OAI) b) Specify 'since' date ie all changes since.... ???

3. Future consideration: - replication and harvesting - HTTPGet v. Web services or both - resource attributes

Changed:
<
<
- suppress updates if local resource doesn't want DoNotReplicate - delete DeleteResource - how propagate?
>
>
- suppress updates if local resource doesn't want DoNotReplicate - delete DeleteResource - how propagate?
  - etc. ...Deprecated? Use of wrappers

4. Summary AAS Demo Jan2004

**DECISION Get on with it!


F DEMO ACTION Tony Linde

1. Standards Identifier, RM, Schema doc's to be adopted as above

2. Future Web service call interface Standard query language - SQL and XQuery needed Harvesting/replication

2. Demos a) Schema Will be based on Schema 0.9 ready ?20 Oct b) Harvesting Use OAI to return all resources since (date) as XML c) Goals Not - passing between registry projects But - registries must replicate ID and authority Maybe - external services e.g. cone search, data query, ACE

Are present schemas adequate for calling such services in an automated fashion

Need one standard way of getting metadata from a service. Add to schema how you get the full metadata for a resource (ie not just what is in registry) Need to investigate further.

d) Organisation of demos i) Who will publish registries AstroGrid AVO NVO GAVO CDS? + ? - Notify via mailing list

ii) What sort?

META FILEATTACHMENT attr="h" comment="" date="1066310134" name="IVOARegistry.ppt" path="IVOA Registry.ppt" size="260096" user="TonyLinde" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066310371" name="IVOARegistry.pdf" path="IVOA Registry.pdf" size="449628" user="TonyLinde" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312861" name="IdentifiersOct03.ppt" path="IdentifiersOct03.ppt" size="207360" user="RayPlante" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312899" name="IdentifiersOct03.pdf" path="IdentifiersOct03.pdf" size="100878" user="RayPlante" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312936" name="RMXMLOct03.ppt" path="RMXMLOct03.ppt" size="242176" user="RayPlante" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312973" name="RMXMLOct03.pdf" path="RMXMLOct03.pdf" size="117393" user="RayPlante" version="1.1"
META FILEATTACHMENT attr="" comment="Elizabeth Auden's slides: Registry Interface Calls" date="1066313930" name="IVOA_RegInterfaceCalls2.ppt" path="C:\Astrogrid\Presentations\IVOA_RegInterfaceCalls2.ppt" size="70144" user="TWikiGuest" version="/opsw/avo/rcs/bin/rlog -h /home/web/ivoa/docs/internal/IVOA/InterOpOct2003Registry/IVOA_RegInterfaceCalls2.ppt,v"
META FILEATTACHMENT attr="" comment="Resource metadata revisions" date="1066315583" name="RegistriesRM.ppt" path="C:\Documents and Settings\hanisch1.STSCI\My Documents\Talks_and_Papers\adass03\RegistriesRM.ppt" size="184320" user="BobHanisch" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066640080" name="IVOARegistry2.ppt" path="IVOA Registry 2.ppt" size="218624" user="TonyLinde" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066640103" name="IVOARegistry2.PDF" path="IVOA Registry2.PDF" size="495909" user="TonyLinde" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066652003" name="RegHarvest.pdf" path="RegHarvest.pdf" size="132964" user="KeithNoddle" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066652022" name="RegHarvest.sxi" path="RegHarvest.sxi" size="43002" user="KeithNoddle" version="1.1"

Revision 102003-10-20 - KeithNoddle

 
META TOPICPARENT name="IvoaResReg"
This page will hold presentations, discussions and results of the Registry WG meeting at InterOpOct2003.

Presentations

Day 1:

Changed:
<
<
>
>
  Day 2:

Notes

many thanks to Anita for these notes from the Day 1 meeting

Registry IVOA notes  AMS Richards amsr@jb.man.ac.uk

11:00 2003 October 16

NB two Jan 2004 demos
- AAS
- AVO Munich 27-28 Jan

---------------------------------------------------------------
A IDENTIFIERS
Ray Plante See document on wiki V0.2

Register ID with some global authority to ensure uniqueness etc.

1.
Issues to be resolved through practice:

*
Mirrors/versions have different identifiers - harvesters need to know mirror etc. info too.

*
Permanence - can identifiers be recycled?

*
'ivo' xml format implies resource is _registered_ somewhere ADEC (organisation of NASA astrophysical data centres) adopted IVOA syntax. US data publishing lined to (refereed) journal papers

*
Need standard for logical identifiers - need to automatically point to relevant version. Are Identifiers 'logical' or static? Is there a contradiction here?


2.
FAQs
Who assigns IDs?	Publishing registry (check to avoid
			duplication)

What is ID content?	Conventions to encourage uniqueness but
			decision ultimately up to authority behind ID
			(as long as it is in valid characters).

Where register?		One place (so must have all registries
			accessible via global registry)

3.
Discussion
How to manage global requirements espec uniquness of ID?
How police?
Trust - duplicate won't be replicated so it won't work - practical disincentive. Need good explanation for data providers Learn what mechanisms are needed in practice e.g. Jan 2004 demo

**DECISION: Support recommendation to adopt IVOA Identifiers V0.2

---------------------------------------------------------------
B RESOURCE METADATA
Bob Hanisch see wiki doc V0.82

See Changes

creator removed from list of required elements,
**address/tel. should be removed

Title - verbose (e.g. Hubble Space Telescope) required ShortName optional

Keywords - ideally one source i.e. IAU Thesaurus
			  meanwhile will be more than one but must have namespace(?)
			  with definitions

Wavelength coverage - mm-wave added, submm may be needed?
				Suggestion - submm instead of mm?
							 e.g. 10 micron to 1 mm ????
				Can't be perfect, can overlap as necessarily

UCDs	  - will there be a problem with getting them two ways?
			  But they aren't compulsory. Having UCDs in registry if
			  required allows quicker searches in some cases. More
			  important, it is allows it to be in model.

Data Quality - agree outline of section, discuss details in light of
			 practice (e.g. distinguish quality of data
			 v. metadata).
			 Data provider assigns A B or C

**DECISION: Support recommendation to adopt IVOA RM v0.82

---------------------------------------------------------------

C
VOResource

Ray Plante v0.8.4

Covers most concepts in RM Working Doc
Extensions will add additional metdata for particular classes of resources as required.

NB See XML Schema style guide provided by Ray Plante

1
Metadata

Minimum required metadata just 7-8 elements

*
Identity
Service metadata - how to invoke/access, virtual address

*
Community

*
Data service - collection
		  - sky service
		  - tabular sky service

etc.

2
*Issues*

Use of STC metadata schema (Rots) ultimate goal
- Use temporary simple version for now

Using/reusing VOTable elements? for table columns, service parameters

>From discussion:
Duplication of services - how resolve

Coverage etc. - may need to introduce upper and lower limits?

Input registration of services, data collections etc.
		- need documentation and templates
		- for VO testers, basic xml templates probably most useful
		- easy web interface for wider input

3
What can we register?

4
Standardisation
- of each schema
- as a data model
- don't need PR version to use a schema (what does PR mean??) *must stabilise schemas*

---------------------------------------------------------------

D WEB SERVICES

Elizabeth Auden

1
How do services talk to each other?
Language not important as long as accessible to web services.

Queries
	Simple query
	Complex Query
	Cone search
	SIA
	+?
Admin

Query language - XQuery? Others also.
			 - VOQL etc

Returns
	AstroGrid uss XML just not
	Next VOTable (already in some use)

Whose problem is taking to back end e.g. data providers' BDs? Not IVOA?

Not everyone needs to use same language behind the scenes

2
Discussion

2a
Does cone search need to be in Registry? Yes - can't assume it is always built in, need to register where to find specialised search/extraction services - many services are not built in to VOs e.g. extraacting spectra from x-ray data...

2b
See Elizabeth's notes
What do registry interfaces need to be able to do?
i)	Check authenticity ID with global registries
ii)  Register authority ID with global registry
iii) Query the registry and get answer (xml-based in
	  VOResource schema format)
iv)  Harvest the registry through an interface (incorporates
	  replication?)  - web service or httpGET? OAI (uses httpGET?
	  - need one method for Jan 2004! ** OAI **
v)	Define each registry interface web service method in WDSL
vi)  Documentation for data entry tools
vii) Administer registry through add, delete, edit

**RECOMMENDATION: NVO, AstroGrid etc. registries share via OAI for
	Jan 2004

---------------------------------------------------------------

E REGISTRY HARVESTING

Keith Noddle

1.
A registry is itself a resource holding metadata about itself.

- What level - global, local etc? - is registry open to query?

Definitions:
_Harvest_ gather metadata
_Replicate_ data

Basic security checks - need to follow path.

Do not want to have to choose which registry to search
- need automatic referal from local to full registry when required
- choice made automatically

2.
Demo

a) Use HTTPGet (OAI)
b) Specify 'since' date ie all changes since....
???

3.
Future consideration:
- replication and harvesting
  - HTTPGet v. Web services or both
- resource attributes
  - suppress updates if local resource doesn't want DoNotReplicate
  - delete DeleteResource - how propagate?
  - etc. ...Deprecated?
Use of wrappers

4.
Summary
AAS Demo Jan2004

**DECISION Get on with it!

---------------------------------------------------------------
F
DEMO ACTION
Tony Linde

1.
Standards
Identifier, RM, Schema doc's to be adopted as above

2.
Future
Web service call interface
Standard query language - SQL and XQuery needed Harvesting/replication

2.
Demos
a) Schema
	Will be based on Schema 0.9 ready ?20 Oct
b) Harvesting
	Use OAI to return all resources since (date) as XML
c) Goals
Not	- passing between registry projects
But	- registries must replicate ID and authority
Maybe	- external services e.g. cone search, data query, ACE

Are present schemas adequate for calling such services _in an automated fashion_

Need one standard way of getting metadata from a service.
*Add to schema* how you get the full metadata for a resource (ie not just what is in registry) Need to investigate further.

d) Organisation of demos
i)	Who will publish registries
	  AstroGrid AVO NVO GAVO CDS? + ?
	  - _Notify via mailing list_

ii)  What sort?

Added:
>
>
 
META FILEATTACHMENT attr="h" comment="" date="1066310134" name="IVOARegistry.ppt" path="IVOA Registry.ppt" size="260096" user="TonyLinde" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066310371" name="IVOARegistry.pdf" path="IVOA Registry.pdf" size="449628" user="TonyLinde" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312861" name="IdentifiersOct03.ppt" path="IdentifiersOct03.ppt" size="207360" user="RayPlante" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312899" name="IdentifiersOct03.pdf" path="IdentifiersOct03.pdf" size="100878" user="RayPlante" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312936" name="RMXMLOct03.ppt" path="RMXMLOct03.ppt" size="242176" user="RayPlante" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312973" name="RMXMLOct03.pdf" path="RMXMLOct03.pdf" size="117393" user="RayPlante" version="1.1"
META FILEATTACHMENT attr="" comment="Elizabeth Auden's slides: Registry Interface Calls" date="1066313930" name="IVOA_RegInterfaceCalls2.ppt" path="C:\Astrogrid\Presentations\IVOA_RegInterfaceCalls2.ppt" size="70144" user="TWikiGuest" version="/opsw/avo/rcs/bin/rlog -h /home/web/ivoa/docs/internal/IVOA/InterOpOct2003Registry/IVOA_RegInterfaceCalls2.ppt,v"
META FILEATTACHMENT attr="" comment="Resource metadata revisions" date="1066315583" name="RegistriesRM.ppt" path="C:\Documents and Settings\hanisch1.STSCI\My Documents\Talks_and_Papers\adass03\RegistriesRM.ppt" size="184320" user="BobHanisch" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066640080" name="IVOARegistry2.ppt" path="IVOA Registry 2.ppt" size="218624" user="TonyLinde" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066640103" name="IVOARegistry2.PDF" path="IVOA Registry2.PDF" size="495909" user="TonyLinde" version="1.1"
Added:
>
>
META FILEATTACHMENT attr="h" comment="" date="1066652003" name="RegHarvest.pdf" path="RegHarvest.pdf" size="132964" user="KeithNoddle" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066652022" name="RegHarvest.sxi" path="RegHarvest.sxi" size="43002" user="KeithNoddle" version="1.1"
 

Revision 92003-10-20 - TonyLinde

 
META TOPICPARENT name="IvoaResReg"
Changed:
<
<
This page will hold presentations, discussions and results of the Registry WG meeting.
>
>
This page will hold presentations, discussions and results of the Registry WG meeting at InterOpOct2003.
 

Presentations

Day 1:

Deleted:
<
<
 
Added:
>
>
 
Added:
>
>
 
Added:
>
>
Day 2:
 
Added:
>
>
 

Notes

Changed:
<
<
many thanks to Anita for these notes from the meeting
>
>
many thanks to Anita for these notes from the Day 1 meeting
 
Registry IVOA notes  AMS Richards amsr@jb.man.ac.uk

11:00 2003 October 16

NB two Jan 2004 demos
- AAS
- AVO Munich 27-28 Jan

---------------------------------------------------------------
A IDENTIFIERS
Ray Plante See document on wiki V0.2

Register ID with some global authority to ensure uniqueness etc.

1.
Issues to be resolved through practice:

*
Mirrors/versions have different identifiers - harvesters need to know mirror etc. info too.

*
Permanence - can identifiers be recycled?

*
'ivo' xml format implies resource is _registered_ somewhere ADEC (organisation of NASA astrophysical data centres) adopted IVOA syntax. US data publishing lined to (refereed) journal papers

*
Need standard for logical identifiers - need to automatically point to relevant version. Are Identifiers 'logical' or static? Is there a contradiction here?


2.
FAQs
Who assigns IDs?	Publishing registry (check to avoid
			duplication)

What is ID content?	Conventions to encourage uniqueness but
			decision ultimately up to authority behind ID
			(as long as it is in valid characters).

Where register?		One place (so must have all registries
			accessible via global registry)

3.
Discussion
How to manage global requirements espec uniquness of ID?
How police?
Trust - duplicate won't be replicated so it won't work - practical disincentive. Need good explanation for data providers Learn what mechanisms are needed in practice e.g. Jan 2004 demo

**DECISION: Support recommendation to adopt IVOA Identifiers V0.2

---------------------------------------------------------------
B RESOURCE METADATA
Bob Hanisch see wiki doc V0.82

See Changes

creator removed from list of required elements,
**address/tel. should be removed

Title - verbose (e.g. Hubble Space Telescope) required ShortName optional

Keywords - ideally one source i.e. IAU Thesaurus
			  meanwhile will be more than one but must have namespace(?)
			  with definitions

Wavelength coverage - mm-wave added, submm may be needed?
				Suggestion - submm instead of mm?
							 e.g. 10 micron to 1 mm ????
				Can't be perfect, can overlap as necessarily

UCDs	  - will there be a problem with getting them two ways?
			  But they aren't compulsory. Having UCDs in registry if
			  required allows quicker searches in some cases. More
			  important, it is allows it to be in model.

Data Quality - agree outline of section, discuss details in light of
			 practice (e.g. distinguish quality of data
			 v. metadata).
			 Data provider assigns A B or C

**DECISION: Support recommendation to adopt IVOA RM v0.82

---------------------------------------------------------------

C
VOResource

Ray Plante v0.8.4

Covers most concepts in RM Working Doc
Extensions will add additional metdata for particular classes of resources as required.

NB See XML Schema style guide provided by Ray Plante

1
Metadata

Minimum required metadata just 7-8 elements

*
Identity
Service metadata - how to invoke/access, virtual address

*
Community

*
Data service - collection
		  - sky service
		  - tabular sky service

etc.

2
*Issues*

Use of STC metadata schema (Rots) ultimate goal
- Use temporary simple version for now

Using/reusing VOTable elements? for table columns, service parameters

>From discussion:
Duplication of services - how resolve

Coverage etc. - may need to introduce upper and lower limits?

Input registration of services, data collections etc.
		- need documentation and templates
		- for VO testers, basic xml templates probably most useful
		- easy web interface for wider input

3
What can we register?

4
Standardisation
- of each schema
- as a data model
- don't need PR version to use a schema (what does PR mean??) *must stabilise schemas*

---------------------------------------------------------------

D WEB SERVICES

Elizabeth Auden

1
How do services talk to each other?
Language not important as long as accessible to web services.

Queries
	Simple query
	Complex Query
	Cone search
	SIA
	+?
Admin

Query language - XQuery? Others also.
			 - VOQL etc

Returns
	AstroGrid uss XML just not
	Next VOTable (already in some use)

Whose problem is taking to back end e.g. data providers' BDs? Not IVOA?

Not everyone needs to use same language behind the scenes

2
Discussion

2a
Does cone search need to be in Registry? Yes - can't assume it is always built in, need to register where to find specialised search/extraction services - many services are not built in to VOs e.g. extraacting spectra from x-ray data...

2b
See Elizabeth's notes
What do registry interfaces need to be able to do?
i)	Check authenticity ID with global registries
ii)  Register authority ID with global registry
iii) Query the registry and get answer (xml-based in
	  VOResource schema format)
iv)  Harvest the registry through an interface (incorporates
	  replication?)  - web service or httpGET? OAI (uses httpGET?
	  - need one method for Jan 2004! ** OAI **
v)	Define each registry interface web service method in WDSL
vi)  Documentation for data entry tools
vii) Administer registry through add, delete, edit

**RECOMMENDATION: NVO, AstroGrid etc. registries share via OAI for
	Jan 2004

---------------------------------------------------------------

E REGISTRY HARVESTING

Keith Noddle

1.
A registry is itself a resource holding metadata about itself.

- What level - global, local etc? - is registry open to query?

Definitions:
_Harvest_ gather metadata
_Replicate_ data

Basic security checks - need to follow path.

Do not want to have to choose which registry to search
- need automatic referal from local to full registry when required
- choice made automatically

2.
Demo

a) Use HTTPGet (OAI)
b) Specify 'since' date ie all changes since....
???

3.
Future consideration:
- replication and harvesting
  - HTTPGet v. Web services or both
- resource attributes
  - suppress updates if local resource doesn't want DoNotReplicate
  - delete DeleteResource - how propagate?
  - etc. ...Deprecated?
Use of wrappers

4.
Summary
AAS Demo Jan2004

**DECISION Get on with it!

---------------------------------------------------------------
F
DEMO ACTION
Tony Linde

1.
Standards
Identifier, RM, Schema doc's to be adopted as above

2.
Future
Web service call interface
Standard query language - SQL and XQuery needed Harvesting/replication

2.
Demos
a) Schema
	Will be based on Schema 0.9 ready ?20 Oct
b) Harvesting
	Use OAI to return all resources since (date) as XML
c) Goals
Not	- passing between registry projects
But	- registries must replicate ID and authority
Maybe	- external services e.g. cone search, data query, ACE

Are present schemas adequate for calling such services _in an automated fashion_

Need one standard way of getting metadata from a service.
*Add to schema* how you get the full metadata for a resource (ie not just what is in registry) Need to investigate further.

d) Organisation of demos
i)	Who will publish registries
	  AstroGrid AVO NVO GAVO CDS? + ?
	  - _Notify via mailing list_

ii)  What sort?
|*META FILEATTACHMENT*|attr="h" comment="" date="1066310134" name="IVOARegistry.ppt" path="IVOA Registry.ppt" size="260096" user="TonyLinde" version="1.1"|
|*META FILEATTACHMENT*|attr="h" comment="" date="1066310371" name="IVOARegistry.pdf" path="IVOA Registry.pdf" size="449628" user="TonyLinde" version="1.1"|
|*META FILEATTACHMENT*|attr="" comment="" date="1066312861" name="IdentifiersOct03.ppt" path="IdentifiersOct03.ppt" size="207360" user="RayPlante" version="1.1"|
|*META FILEATTACHMENT*|attr="" comment="" date="1066312899" name="IdentifiersOct03.pdf" path="IdentifiersOct03.pdf" size="100878" user="RayPlante" version="1.1"|
|*META FILEATTACHMENT*|attr="" comment="" date="1066312936" name="RMXMLOct03.ppt" path="RMXMLOct03.ppt" size="242176" user="RayPlante" version="1.1"|
|*META FILEATTACHMENT*|attr="" comment="" date="1066312973" name="RMXMLOct03.pdf" path="RMXMLOct03.pdf" size="117393" user="RayPlante" version="1.1"|
|*META FILEATTACHMENT*|attr="" comment="Elizabeth Auden's slides: Registry Interface Calls" date="1066313930" name="IVOA_RegInterfaceCalls2.ppt" path="C:\Astrogrid\Presentations\IVOA_RegInterfaceCalls2.ppt" size="70144" user="TWikiGuest" version="/opsw/avo/rcs/bin/rlog  -h /home/web/ivoa/docs/internal/IVOA/InterOpOct2003Registry/IVOA_RegInterfaceCalls2.ppt,v"|
|*META FILEATTACHMENT*|attr="" comment="Resource metadata revisions" date="1066315583" name="RegistriesRM.ppt" path="C:\Documents and Settings\hanisch1.STSCI\My Documents\Talks_and_Papers\adass03\RegistriesRM.ppt" size="184320" user="BobHanisch" version="1.1"|

Added:
>
>
META FILEATTACHMENT attr="h" comment="" date="1066640080" name="IVOARegistry2.ppt" path="IVOA Registry 2.ppt" size="218624" user="TonyLinde" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066640103" name="IVOARegistry2.PDF" path="IVOA Registry2.PDF" size="495909" user="TonyLinde" version="1.1"
 

Revision 82003-10-17 - TonyLinde

 
META TOPICPARENT name="IvoaResReg"
This page will hold presentations, discussions and results of the Registry WG meeting.

Presentations

Day 1:

Changed:
<
<
>
>
 

Notes

many thanks to Anita for these notes from the meeting

Registry IVOA notes  AMS Richards amsr@jb.man.ac.uk

11:00 2003 October 16

NB two Jan 2004 demos
- AAS
- AVO Munich 27-28 Jan

---------------------------------------------------------------
A IDENTIFIERS
Ray Plante See document on wiki V0.2

Register ID with some global authority to ensure uniqueness etc.

1.
Issues to be resolved through practice:

*
Mirrors/versions have different identifiers - harvesters need to know mirror etc. info too.

*
Permanence - can identifiers be recycled?

*
'ivo' xml format implies resource is _registered_ somewhere ADEC (organisation of NASA astrophysical data centres) adopted IVOA syntax. US data publishing lined to (refereed) journal papers

*
Need standard for logical identifiers - need to automatically point to relevant version. Are Identifiers 'logical' or static? Is there a contradiction here?


2.
FAQs
Who assigns IDs?	Publishing registry (check to avoid
			duplication)

What is ID content?	Conventions to encourage uniqueness but
			decision ultimately up to authority behind ID
			(as long as it is in valid characters).

Where register?		One place (so must have all registries
			accessible via global registry)

3.
Discussion
How to manage global requirements espec uniquness of ID?
How police?
Trust - duplicate won't be replicated so it won't work - practical disincentive. Need good explanation for data providers Learn what mechanisms are needed in practice e.g. Jan 2004 demo

**DECISION: Support recommendation to adopt IVOA Identifiers V0.2

---------------------------------------------------------------
B RESOURCE METADATA
Bob Hanisch see wiki doc V0.82

See Changes

creator removed from list of required elements,
**address/tel. should be removed

Title - verbose (e.g. Hubble Space Telescope) required ShortName optional

Keywords - ideally one source i.e. IAU Thesaurus
			  meanwhile will be more than one but must have namespace(?)
			  with definitions

Wavelength coverage - mm-wave added, submm may be needed?
				Suggestion - submm instead of mm?
							 e.g. 10 micron to 1 mm ????
				Can't be perfect, can overlap as necessarily

UCDs	  - will there be a problem with getting them two ways?
			  But they aren't compulsory. Having UCDs in registry if
			  required allows quicker searches in some cases. More
			  important, it is allows it to be in model.

Data Quality - agree outline of section, discuss details in light of
			 practice (e.g. distinguish quality of data
			 v. metadata).
			 Data provider assigns A B or C

**DECISION: Support recommendation to adopt IVOA RM v0.82

---------------------------------------------------------------

C
VOResource

Ray Plante v0.8.4

Covers most concepts in RM Working Doc
Extensions will add additional metdata for particular classes of resources as required.

NB See XML Schema style guide provided by Ray Plante

1
Metadata

Minimum required metadata just 7-8 elements

*
Identity
Service metadata - how to invoke/access, virtual address

*
Community

*
Data service - collection
		  - sky service
		  - tabular sky service

etc.

2
*Issues*

Use of STC metadata schema (Rots) ultimate goal
- Use temporary simple version for now

Using/reusing VOTable elements? for table columns, service parameters

>From discussion:
Duplication of services - how resolve

Coverage etc. - may need to introduce upper and lower limits?

Input registration of services, data collections etc.
		- need documentation and templates
		- for VO testers, basic xml templates probably most useful
		- easy web interface for wider input

3
What can we register?

4
Standardisation
- of each schema
- as a data model
- don't need PR version to use a schema (what does PR mean??) *must stabilise schemas*

---------------------------------------------------------------

D WEB SERVICES

Elizabeth Auden

1
How do services talk to each other?
Language not important as long as accessible to web services.

Queries
	Simple query
	Complex Query
	Cone search
	SIA
	+?
Admin

Query language - XQuery? Others also.
			 - VOQL etc

Returns
	AstroGrid uss XML just not
	Next VOTable (already in some use)

Whose problem is taking to back end e.g. data providers' BDs? Not IVOA?

Not everyone needs to use same language behind the scenes

2
Discussion

2a
Does cone search need to be in Registry? Yes - can't assume it is always built in, need to register where to find specialised search/extraction services - many services are not built in to VOs e.g. extraacting spectra from x-ray data...

2b
See Elizabeth's notes
What do registry interfaces need to be able to do?
i)	Check authenticity ID with global registries
ii)  Register authority ID with global registry
iii) Query the registry and get answer (xml-based in
	  VOResource schema format)
iv)  Harvest the registry through an interface (incorporates
	  replication?)  - web service or httpGET? OAI (uses httpGET?
	  - need one method for Jan 2004! ** OAI **
v)	Define each registry interface web service method in WDSL
vi)  Documentation for data entry tools
vii) Administer registry through add, delete, edit

**RECOMMENDATION: NVO, AstroGrid etc. registries share via OAI for
	Jan 2004

---------------------------------------------------------------

E REGISTRY HARVESTING

Keith Noddle

1.
A registry is itself a resource holding metadata about itself.

- What level - global, local etc? - is registry open to query?

Definitions:
_Harvest_ gather metadata
_Replicate_ data

Basic security checks - need to follow path.

Do not want to have to choose which registry to search
- need automatic referal from local to full registry when required
- choice made automatically

2.
Demo

a) Use HTTPGet (OAI)
b) Specify 'since' date ie all changes since....
???

3.
Future consideration:
- replication and harvesting
  - HTTPGet v. Web services or both
- resource attributes
  - suppress updates if local resource doesn't want DoNotReplicate
  - delete DeleteResource - how propagate?
  - etc. ...Deprecated?
Use of wrappers

4.
Summary
AAS Demo Jan2004

**DECISION Get on with it!

---------------------------------------------------------------
F
DEMO ACTION
Tony Linde

1.
Standards
Identifier, RM, Schema doc's to be adopted as above

2.
Future
Web service call interface
Standard query language - SQL and XQuery needed Harvesting/replication

2.
Demos
a) Schema
	Will be based on Schema 0.9 ready ?20 Oct
b) Harvesting
	Use OAI to return all resources since (date) as XML
c) Goals
Not	- passing between registry projects
But	- registries must replicate ID and authority
Maybe	- external services e.g. cone search, data query, ACE

Are present schemas adequate for calling such services _in an automated fashion_

Need one standard way of getting metadata from a service.
*Add to schema* how you get the full metadata for a resource (ie not just what is in registry) Need to investigate further.

d) Organisation of demos
i)	Who will publish registries
	  AstroGrid AVO NVO GAVO CDS? + ?
	  - _Notify via mailing list_

ii)  What sort?
|*META FILEATTACHMENT*|attr="h" comment="" date="1066310134" name="IVOARegistry.ppt" path="IVOA Registry.ppt" size="260096" user="TonyLinde" version="1.1"|
|*META FILEATTACHMENT*|attr="h" comment="" date="1066310371" name="IVOARegistry.pdf" path="IVOA Registry.pdf" size="449628" user="TonyLinde" version="1.1"|
|*META FILEATTACHMENT*|attr="" comment="" date="1066312861" name="IdentifiersOct03.ppt" path="IdentifiersOct03.ppt" size="207360" user="RayPlante" version="1.1"|
|*META FILEATTACHMENT*|attr="" comment="" date="1066312899" name="IdentifiersOct03.pdf" path="IdentifiersOct03.pdf" size="100878" user="RayPlante" version="1.1"|
|*META FILEATTACHMENT*|attr="" comment="" date="1066312936" name="RMXMLOct03.ppt" path="RMXMLOct03.ppt" size="242176" user="RayPlante" version="1.1"|
|*META FILEATTACHMENT*|attr="" comment="" date="1066312973" name="RMXMLOct03.pdf" path="RMXMLOct03.pdf" size="117393" user="RayPlante" version="1.1"|
|*META FILEATTACHMENT*|attr="" comment="Elizabeth Auden's slides: Registry Interface Calls" date="1066313930" name="IVOA_RegInterfaceCalls2.ppt" path="C:\Astrogrid\Presentations\IVOA_RegInterfaceCalls2.ppt" size="70144" user="TWikiGuest" version="/opsw/avo/rcs/bin/rlog  -h /home/web/ivoa/docs/internal/IVOA/InterOpOct2003Registry/IVOA_RegInterfaceCalls2.ppt,v"|
|*META FILEATTACHMENT*|attr="" comment="Resource metadata revisions" date="1066315583" name="RegistriesRM.ppt" path="C:\Documents and Settings\hanisch1.STSCI\My Documents\Talks_and_Papers\adass03\RegistriesRM.ppt" size="184320" user="BobHanisch" version="1.1"|

Revision 72003-10-16 - BobHanisch

 
META TOPICPARENT name="IvoaResReg"
This page will hold presentations, discussions and results of the Registry WG meeting.

Presentations

Day 1:

Added:
>
>
 
Added:
>
>
 

Notes

many thanks to Anita for these notes from the meeting

Registry IVOA notes  AMS Richards amsr@jb.man.ac.uk

11:00 2003 October 16

NB two Jan 2004 demos
- AAS
- AVO Munich 27-28 Jan

---------------------------------------------------------------
A IDENTIFIERS
Ray Plante See document on wiki V0.2

Register ID with some global authority to ensure uniqueness etc.

1.
Issues to be resolved through practice:

*
Mirrors/versions have different identifiers - harvesters need to know mirror etc. info too.

*
Permanence - can identifiers be recycled?

*
'ivo' xml format implies resource is _registered_ somewhere ADEC (organisation of NASA astrophysical data centres) adopted IVOA syntax. US data publishing lined to (refereed) journal papers

*
Need standard for logical identifiers - need to automatically point to relevant version. Are Identifiers 'logical' or static? Is there a contradiction here?


2.
FAQs
Who assigns IDs?	Publishing registry (check to avoid
			duplication)

What is ID content?	Conventions to encourage uniqueness but
			decision ultimately up to authority behind ID
			(as long as it is in valid characters).

Where register?		One place (so must have all registries
			accessible via global registry)

3.
Discussion
How to manage global requirements espec uniquness of ID?
How police?
Trust - duplicate won't be replicated so it won't work - practical disincentive. Need good explanation for data providers Learn what mechanisms are needed in practice e.g. Jan 2004 demo

**DECISION: Support recommendation to adopt IVOA Identifiers V0.2

---------------------------------------------------------------
B RESOURCE METADATA
Bob Hanisch see wiki doc V0.82

See Changes

Changed:
<
<
creator removed,
>
>
creator removed from list of required elements,
 **address/tel. should be removed

Title - verbose (e.g. Hubble Space Telescope) required ShortName optional

Keywords - ideally one source i.e. IAU Thesaurus meanwhile will be more than one but must have namespace(?) with definitions

Wavelength coverage - mm-wave added, submm may be needed? Suggestion - submm instead of mm? e.g. 10 micron to 1 mm ???? Can't be perfect, can overlap as necessarily

UCDs - will there be a problem with getting them two ways? But they aren't compulsory. Having UCDs in registry if required allows quicker searches in some cases. More important, it is allows it to be in model.

Data Quality - agree outline of section, discuss details in light of practice (e.g. distinguish quality of data v. metadata). Data provider assigns A B or C

**DECISION: Support recommendation to adopt IVOA RM v0.82


C VOResource

Ray Plante v0.8.4

Covers most concepts in RM Working Doc Extensions will add additional metdata for particular classes of resources as required.

NB See XML Schema style guide provided by Ray Plante

1 Metadata

Minimum required metadata just 7-8 elements

* Identity Service metadata - how to invoke/access, virtual address

* Community

* Data service - collection - sky service - tabular sky service

etc.

2 Issues

Use of STC metadata schema (Rots) ultimate goal - Use temporary simple version for now

Using/reusing VOTable elements? for table columns, service parameters

>From discussion:
Duplication of services - how resolve

Coverage etc. - may need to introduce upper and lower limits?

Input registration of services, data collections etc. - need documentation and templates - for VO testers, basic xml templates probably most useful - easy web interface for wider input

3 What can we register?

4 Standardisation - of each schema - as a data model - don't need PR version to use a schema (what does PR mean??) must stabilise schemas


D WEB SERVICES

Elizabeth Auden

1 How do services talk to each other? Language not important as long as accessible to web services.

Queries Simple query Complex Query Cone search SIA +? Admin

Query language - XQuery? Others also. - VOQL etc

Returns AstroGrid uss XML just not Next VOTable (already in some use)

Whose problem is taking to back end e.g. data providers' BDs? Not IVOA?

Not everyone needs to use same language behind the scenes

2 Discussion

2a Does cone search need to be in Registry? Yes - can't assume it is always built in, need to register where to find specialised search/extraction services - many services are not built in to VOs e.g. extraacting spectra from x-ray data...

2b See Elizabeth's notes What do registry interfaces need to be able to do? i) Check authenticity ID with global registries ii) Register authority ID with global registry iii) Query the registry and get answer (xml-based in VOResource schema format) iv) Harvest the registry through an interface (incorporates replication?) - web service or httpGET? OAI (uses httpGET? - need one method for Jan 2004! * OAI * v) Define each registry interface web service method in WDSL vi) Documentation for data entry tools vii) Administer registry through add, delete, edit

**RECOMMENDATION: NVO, AstroGrid etc. registries share via OAI for Jan 2004


E REGISTRY HARVESTING

Keith Noddle

1. A registry is itself a resource holding metadata about itself.

- What level - global, local etc? - is registry open to query?

Definitions: Harvest gather metadata Replicate data

Basic security checks - need to follow path.

Do not want to have to choose which registry to search - need automatic referal from local to full registry when required - choice made automatically

2. Demo

a) Use HTTPGet (OAI) b) Specify 'since' date ie all changes since.... ???

3. Future consideration: - replication and harvesting - HTTPGet v. Web services or both - resource attributes - suppress updates if local resource doesn't want DoNotReplicate - delete DeleteResource - how propagate? - etc. ...Deprecated? Use of wrappers

4. Summary AAS Demo Jan2004

**DECISION Get on with it!


F DEMO ACTION Tony Linde

1. Standards Identifier, RM, Schema doc's to be adopted as above

2. Future Web service call interface Standard query language - SQL and XQuery needed Harvesting/replication

2. Demos a) Schema Will be based on Schema 0.9 ready ?20 Oct b) Harvesting Use OAI to return all resources since (date) as XML c) Goals Not - passing between registry projects But - registries must replicate ID and authority Maybe - external services e.g. cone search, data query, ACE

Are present schemas adequate for calling such services in an automated fashion

Need one standard way of getting metadata from a service. Add to schema how you get the full metadata for a resource (ie not just what is in registry) Need to investigate further.

d) Organisation of demos i) Who will publish registries AstroGrid AVO NVO GAVO CDS? + ? - Notify via mailing list

ii) What sort?

Deleted:
<
<
 
META FILEATTACHMENT attr="h" comment="" date="1066310134" name="IVOARegistry.ppt" path="IVOA Registry.ppt" size="260096" user="TonyLinde" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066310371" name="IVOARegistry.pdf" path="IVOA Registry.pdf" size="449628" user="TonyLinde" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312861" name="IdentifiersOct03.ppt" path="IdentifiersOct03.ppt" size="207360" user="RayPlante" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312899" name="IdentifiersOct03.pdf" path="IdentifiersOct03.pdf" size="100878" user="RayPlante" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312936" name="RMXMLOct03.ppt" path="RMXMLOct03.ppt" size="242176" user="RayPlante" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312973" name="RMXMLOct03.pdf" path="RMXMLOct03.pdf" size="117393" user="RayPlante" version="1.1"
META FILEATTACHMENT attr="" comment="Elizabeth Auden's slides: Registry Interface Calls" date="1066313930" name="IVOA_RegInterfaceCalls2.ppt" path="C:\Astrogrid\Presentations\IVOA_RegInterfaceCalls2.ppt" size="70144" user="TWikiGuest" version="/opsw/avo/rcs/bin/rlog -h /home/web/ivoa/docs/internal/IVOA/InterOpOct2003Registry/IVOA_RegInterfaceCalls2.ppt,v"
Added:
>
>
META FILEATTACHMENT attr="" comment="Resource metadata revisions" date="1066315583" name="RegistriesRM.ppt" path="C:\Documents and Settings\hanisch1.STSCI\My Documents\Talks_and_Papers\adass03\RegistriesRM.ppt" size="184320" user="BobHanisch" version="1.1"
 

Revision 62003-10-16 - TWikiGuest

 
META TOPICPARENT name="IvoaResReg"
This page will hold presentations, discussions and results of the Registry WG meeting.

Presentations

Day 1:

Notes

many thanks to Anita for these notes from the meeting

Registry IVOA notes  AMS Richards amsr@jb.man.ac.uk

11:00 2003 October 16

NB two Jan 2004 demos
- AAS
- AVO Munich 27-28 Jan

---------------------------------------------------------------
A IDENTIFIERS
Ray Plante See document on wiki V0.2

Register ID with some global authority to ensure uniqueness etc.

1.
Issues to be resolved through practice:

*
Mirrors/versions have different identifiers - harvesters need to know mirror etc. info too.

*
Permanence - can identifiers be recycled?

*
'ivo' xml format implies resource is _registered_ somewhere ADEC (organisation of NASA astrophysical data centres) adopted IVOA syntax. US data publishing lined to (refereed) journal papers

*
Need standard for logical identifiers - need to automatically point to relevant version. Are Identifiers 'logical' or static? Is there a contradiction here?


2.
FAQs
Who assigns IDs?	Publishing registry (check to avoid
			duplication)

What is ID content?	Conventions to encourage uniqueness but
			decision ultimately up to authority behind ID
			(as long as it is in valid characters).

Where register?		One place (so must have all registries
			accessible via global registry)

3.
Discussion
How to manage global requirements espec uniquness of ID?
How police?
Trust - duplicate won't be replicated so it won't work - practical disincentive. Need good explanation for data providers Learn what mechanisms are needed in practice e.g. Jan 2004 demo

**DECISION: Support recommendation to adopt IVOA Identifiers V0.2

---------------------------------------------------------------
B RESOURCE METADATA
Bob Hanisch see wiki doc V0.82

See Changes

creator removed,
**address/tel. should be removed

Title - verbose (e.g. Hubble Space Telescope) required ShortName optional

Keywords - ideally one source i.e. IAU Thesaurus
			  meanwhile will be more than one but must have namespace(?)
			  with definitions

Wavelength coverage - mm-wave added, submm may be needed?
				Suggestion - submm instead of mm?
							 e.g. 10 micron to 1 mm ????
				Can't be perfect, can overlap as necessarily

UCDs	  - will there be a problem with getting them two ways?
			  But they aren't compulsory. Having UCDs in registry if
			  required allows quicker searches in some cases. More
			  important, it is allows it to be in model.

Data Quality - agree outline of section, discuss details in light of
			 practice (e.g. distinguish quality of data
			 v. metadata).
			 Data provider assigns A B or C

**DECISION: Support recommendation to adopt IVOA RM v0.82

---------------------------------------------------------------

C
VOResource

Ray Plante v0.8.4

Covers most concepts in RM Working Doc
Extensions will add additional metdata for particular classes of resources as required.

NB See XML Schema style guide provided by Ray Plante

1
Metadata

Minimum required metadata just 7-8 elements

*
Identity
Service metadata - how to invoke/access, virtual address

*
Community

*
Data service - collection
		  - sky service
		  - tabular sky service

etc.

2
*Issues*

Use of STC metadata schema (Rots) ultimate goal
- Use temporary simple version for now

Using/reusing VOTable elements? for table columns, service parameters

>From discussion:
Duplication of services - how resolve

Coverage etc. - may need to introduce upper and lower limits?

Input registration of services, data collections etc.
		- need documentation and templates
		- for VO testers, basic xml templates probably most useful
		- easy web interface for wider input

3
What can we register?

4
Standardisation
- of each schema
- as a data model
- don't need PR version to use a schema (what does PR mean??) *must stabilise schemas*

---------------------------------------------------------------

D WEB SERVICES

Elizabeth Auden

1
How do services talk to each other?
Language not important as long as accessible to web services.

Queries
	Simple query
	Complex Query
	Cone search
	SIA
	+?
Admin

Query language - XQuery? Others also.
			 - VOQL etc

Returns
	AstroGrid uss XML just not
	Next VOTable (already in some use)

Whose problem is taking to back end e.g. data providers' BDs? Not IVOA?

Not everyone needs to use same language behind the scenes

2
Discussion

2a
Does cone search need to be in Registry? Yes - can't assume it is always built in, need to register where to find specialised search/extraction services - many services are not built in to VOs e.g. extraacting spectra from x-ray data...

2b
See Elizabeth's notes
What do registry interfaces need to be able to do?
i)	Check authenticity ID with global registries
ii)  Register authority ID with global registry
iii) Query the registry and get answer (xml-based in
	  VOResource schema format)
iv)  Harvest the registry through an interface (incorporates
	  replication?)  - web service or httpGET? OAI (uses httpGET?
	  - need one method for Jan 2004! ** OAI **
v)	Define each registry interface web service method in WDSL
vi)  Documentation for data entry tools
vii) Administer registry through add, delete, edit

**RECOMMENDATION: NVO, AstroGrid etc. registries share via OAI for
	Jan 2004

---------------------------------------------------------------

E REGISTRY HARVESTING

Keith Noddle

1.
A registry is itself a resource holding metadata about itself.

- What level - global, local etc? - is registry open to query?

Definitions:
_Harvest_ gather metadata
_Replicate_ data

Basic security checks - need to follow path.

Do not want to have to choose which registry to search
- need automatic referal from local to full registry when required
- choice made automatically

2.
Demo

a) Use HTTPGet (OAI)
b) Specify 'since' date ie all changes since....
???

3.
Future consideration:
- replication and harvesting
  - HTTPGet v. Web services or both
- resource attributes
  - suppress updates if local resource doesn't want DoNotReplicate
  - delete DeleteResource - how propagate?
  - etc. ...Deprecated?
Use of wrappers

4.
Summary
AAS Demo Jan2004

**DECISION Get on with it!

---------------------------------------------------------------
F
DEMO ACTION
Tony Linde

1.
Standards
Identifier, RM, Schema doc's to be adopted as above

2.
Future
Web service call interface
Standard query language - SQL and XQuery needed Harvesting/replication

2.
Demos
a) Schema
	Will be based on Schema 0.9 ready ?20 Oct
b) Harvesting
	Use OAI to return all resources since (date) as XML
c) Goals
Not	- passing between registry projects
But	- registries must replicate ID and authority
Maybe	- external services e.g. cone search, data query, ACE

Are present schemas adequate for calling such services _in an automated fashion_

Need one standard way of getting metadata from a service.
*Add to schema* how you get the full metadata for a resource (ie not just what is in registry) Need to investigate further.

d) Organisation of demos
i)	Who will publish registries
	  AstroGrid AVO NVO GAVO CDS? + ?
	  - _Notify via mailing list_

ii)  What sort?
META FILEATTACHMENT attr="h" comment="" date="1066310134" name="IVOARegistry.ppt" path="IVOA Registry.ppt" size="260096" user="TonyLinde" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066310371" name="IVOARegistry.pdf" path="IVOA Registry.pdf" size="449628" user="TonyLinde" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312861" name="IdentifiersOct03.ppt" path="IdentifiersOct03.ppt" size="207360" user="RayPlante" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312899" name="IdentifiersOct03.pdf" path="IdentifiersOct03.pdf" size="100878" user="RayPlante" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312936" name="RMXMLOct03.ppt" path="RMXMLOct03.ppt" size="242176" user="RayPlante" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312973" name="RMXMLOct03.pdf" path="RMXMLOct03.pdf" size="117393" user="RayPlante" version="1.1"
Added:
>
>
META FILEATTACHMENT attr="" comment="Elizabeth Auden's slides: Registry Interface Calls" date="1066313930" name="IVOA_RegInterfaceCalls2.ppt" path="C:\Astrogrid\Presentations\IVOA_RegInterfaceCalls2.ppt" size="70144" user="TWikiGuest" version="/opsw/avo/rcs/bin/rlog -h /home/web/ivoa/docs/internal/IVOA/InterOpOct2003Registry/IVOA_RegInterfaceCalls2.ppt,v"
 

Revision 52003-10-16 - RayPlante

 
META TOPICPARENT name="IvoaResReg"
This page will hold presentations, discussions and results of the Registry WG meeting.

Presentations

Day 1:

Added:
>
>
 

Notes

many thanks to Anita for these notes from the meeting

Registry IVOA notes  AMS Richards amsr@jb.man.ac.uk

11:00 2003 October 16

NB two Jan 2004 demos
- AAS
- AVO Munich 27-28 Jan

---------------------------------------------------------------
A IDENTIFIERS
Ray Plante See document on wiki V0.2

Register ID with some global authority to ensure uniqueness etc.

1.
Issues to be resolved through practice:

*
Mirrors/versions have different identifiers - harvesters need to know mirror etc. info too.

*
Permanence - can identifiers be recycled?

*
'ivo' xml format implies resource is _registered_ somewhere ADEC (organisation of NASA astrophysical data centres) adopted IVOA syntax. US data publishing lined to (refereed) journal papers

*
Need standard for logical identifiers - need to automatically point to relevant version. Are Identifiers 'logical' or static? Is there a contradiction here?


2.
FAQs
Who assigns IDs?	Publishing registry (check to avoid
			duplication)

What is ID content?	Conventions to encourage uniqueness but
			decision ultimately up to authority behind ID
			(as long as it is in valid characters).

Where register?		One place (so must have all registries
			accessible via global registry)

3.
Discussion
How to manage global requirements espec uniquness of ID?
How police?
Trust - duplicate won't be replicated so it won't work - practical disincentive. Need good explanation for data providers Learn what mechanisms are needed in practice e.g. Jan 2004 demo

**DECISION: Support recommendation to adopt IVOA Identifiers V0.2

---------------------------------------------------------------
B RESOURCE METADATA
Bob Hanisch see wiki doc V0.82

See Changes

creator removed,
**address/tel. should be removed

Title - verbose (e.g. Hubble Space Telescope) required ShortName optional

Keywords - ideally one source i.e. IAU Thesaurus
			  meanwhile will be more than one but must have namespace(?)
			  with definitions

Wavelength coverage - mm-wave added, submm may be needed?
				Suggestion - submm instead of mm?
							 e.g. 10 micron to 1 mm ????
				Can't be perfect, can overlap as necessarily

UCDs	  - will there be a problem with getting them two ways?
			  But they aren't compulsory. Having UCDs in registry if
			  required allows quicker searches in some cases. More
			  important, it is allows it to be in model.

Data Quality - agree outline of section, discuss details in light of
			 practice (e.g. distinguish quality of data
			 v. metadata).
			 Data provider assigns A B or C

**DECISION: Support recommendation to adopt IVOA RM v0.82

---------------------------------------------------------------

C
VOResource

Ray Plante v0.8.4

Covers most concepts in RM Working Doc
Extensions will add additional metdata for particular classes of resources as required.

NB See XML Schema style guide provided by Ray Plante

1
Metadata

Minimum required metadata just 7-8 elements

*
Identity
Service metadata - how to invoke/access, virtual address

*
Community

*
Data service - collection
		  - sky service
		  - tabular sky service

etc.

2
*Issues*

Use of STC metadata schema (Rots) ultimate goal
- Use temporary simple version for now

Using/reusing VOTable elements? for table columns, service parameters

>From discussion:
Duplication of services - how resolve

Coverage etc. - may need to introduce upper and lower limits?

Input registration of services, data collections etc.
		- need documentation and templates
		- for VO testers, basic xml templates probably most useful
		- easy web interface for wider input

3
What can we register?

4
Standardisation
- of each schema
- as a data model
- don't need PR version to use a schema (what does PR mean??) *must stabilise schemas*

---------------------------------------------------------------

D WEB SERVICES

Elizabeth Auden

1
How do services talk to each other?
Language not important as long as accessible to web services.

Queries
	Simple query
	Complex Query
	Cone search
	SIA
	+?
Admin

Query language - XQuery? Others also.
			 - VOQL etc

Returns
	AstroGrid uss XML just not
	Next VOTable (already in some use)

Whose problem is taking to back end e.g. data providers' BDs? Not IVOA?

Not everyone needs to use same language behind the scenes

2
Discussion

2a
Does cone search need to be in Registry? Yes - can't assume it is always built in, need to register where to find specialised search/extraction services - many services are not built in to VOs e.g. extraacting spectra from x-ray data...

2b
See Elizabeth's notes
What do registry interfaces need to be able to do?
i)	Check authenticity ID with global registries
ii)  Register authority ID with global registry
iii) Query the registry and get answer (xml-based in
	  VOResource schema format)
iv)  Harvest the registry through an interface (incorporates
	  replication?)  - web service or httpGET? OAI (uses httpGET?
	  - need one method for Jan 2004! ** OAI **
v)	Define each registry interface web service method in WDSL
vi)  Documentation for data entry tools
vii) Administer registry through add, delete, edit

**RECOMMENDATION: NVO, AstroGrid etc. registries share via OAI for
	Jan 2004

---------------------------------------------------------------

E REGISTRY HARVESTING

Keith Noddle

1.
A registry is itself a resource holding metadata about itself.

- What level - global, local etc? - is registry open to query?

Definitions:
_Harvest_ gather metadata
_Replicate_ data

Basic security checks - need to follow path.

Do not want to have to choose which registry to search
- need automatic referal from local to full registry when required
- choice made automatically

2.
Demo

a) Use HTTPGet (OAI)
b) Specify 'since' date ie all changes since....
???

3.
Future consideration:
- replication and harvesting
  - HTTPGet v. Web services or both
- resource attributes
  - suppress updates if local resource doesn't want DoNotReplicate
  - delete DeleteResource - how propagate?
  - etc. ...Deprecated?
Use of wrappers

4.
Summary
AAS Demo Jan2004

**DECISION Get on with it!

---------------------------------------------------------------
F
DEMO ACTION
Tony Linde

1.
Standards
Identifier, RM, Schema doc's to be adopted as above

2.
Future
Web service call interface
Standard query language - SQL and XQuery needed Harvesting/replication

2.
Demos
a) Schema
	Will be based on Schema 0.9 ready ?20 Oct
b) Harvesting
	Use OAI to return all resources since (date) as XML
c) Goals
Not	- passing between registry projects
But	- registries must replicate ID and authority
Maybe	- external services e.g. cone search, data query, ACE

Are present schemas adequate for calling such services _in an automated fashion_

Need one standard way of getting metadata from a service.
*Add to schema* how you get the full metadata for a resource (ie not just what is in registry) Need to investigate further.

d) Organisation of demos
i)	Who will publish registries
	  AstroGrid AVO NVO GAVO CDS? + ?
	  - _Notify via mailing list_

ii)  What sort?
META FILEATTACHMENT attr="h" comment="" date="1066310134" name="IVOARegistry.ppt" path="IVOA Registry.ppt" size="260096" user="TonyLinde" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066310371" name="IVOARegistry.pdf" path="IVOA Registry.pdf" size="449628" user="TonyLinde" version="1.1"
Added:
>
>
META FILEATTACHMENT attr="" comment="" date="1066312861" name="IdentifiersOct03.ppt" path="IdentifiersOct03.ppt" size="207360" user="RayPlante" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312899" name="IdentifiersOct03.pdf" path="IdentifiersOct03.pdf" size="100878" user="RayPlante" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312936" name="RMXMLOct03.ppt" path="RMXMLOct03.ppt" size="242176" user="RayPlante" version="1.1"
META FILEATTACHMENT attr="" comment="" date="1066312973" name="RMXMLOct03.pdf" path="RMXMLOct03.pdf" size="117393" user="RayPlante" version="1.1"
 

Revision 42003-10-16 - TonyLinde

 
META TOPICPARENT name="IvoaResReg"
This page will hold presentations, discussions and results of the Registry WG meeting.

Presentations

Added:
>
>
Day 1:
 

Notes

many thanks to Anita for these notes from the meeting
Changed:
<
<

>
>

 Registry IVOA notes AMS Richards amsr@jb.man.ac.uk

11:00 2003 October 16

NB two Jan 2004 demos - AAS - AVO Munich 27-28 Jan


A IDENTIFIERS Ray Plante See document on wiki V0.2

Register ID with some global authority to ensure uniqueness etc.

1. Issues to be resolved through practice:

* Mirrors/versions have different identifiers - harvesters need to know mirror etc. info too.

* Permanence - can identifiers be recycled?

* 'ivo' xml format implies resource is registered somewhere ADEC (organisation of NASA astrophysical data centres) adopted IVOA syntax. US data publishing lined to (refereed) journal papers

* Need standard for logical identifiers - need to automatically point to relevant version. Are Identifiers 'logical' or static? Is there a contradiction here?

2. FAQs Who assigns IDs? Publishing registry (check to avoid duplication)

What is ID content? Conventions to encourage uniqueness but decision ultimately up to authority behind ID (as long as it is in valid characters).

Where register? One place (so must have all registries accessible via global registry)

3. Discussion How to manage global requirements espec uniquness of ID? How police? Trust - duplicate won't be replicated so it won't work - practical disincentive. Need good explanation for data providers Learn what mechanisms are needed in practice e.g. Jan 2004 demo

**DECISION: Support recommendation to adopt IVOA Identifiers V0.2


B RESOURCE METADATA Bob Hanisch see wiki doc V0.82

See Changes

creator removed, **address/tel. should be removed

Title - verbose (e.g. Hubble Space Telescope) required ShortName optional

Keywords - ideally one source i.e. IAU Thesaurus meanwhile will be more than one but must have namespace(?) with definitions

Wavelength coverage - mm-wave added, submm may be needed? Suggestion - submm instead of mm? e.g. 10 micron to 1 mm ???? Can't be perfect, can overlap as necessarily

UCDs - will there be a problem with getting them two ways? But they aren't compulsory. Having UCDs in registry if required allows quicker searches in some cases. More important, it is allows it to be in model.

Data Quality - agree outline of section, discuss details in light of practice (e.g. distinguish quality of data v. metadata). Data provider assigns A B or C

**DECISION: Support recommendation to adopt IVOA RM v0.82


C VOResource

Ray Plante v0.8.4

Covers most concepts in RM Working Doc Extensions will add additional metdata for particular classes of resources as required.

NB See XML Schema style guide provided by Ray Plante

1 Metadata

Minimum required metadata just 7-8 elements

* Identity Service metadata - how to invoke/access, virtual address

* Community

* Data service - collection - sky service - tabular sky service

etc.

2 Issues

Use of STC metadata schema (Rots) ultimate goal - Use temporary simple version for now

Using/reusing VOTable elements? for table columns, service parameters

>From discussion:
Duplication of services - how resolve

Coverage etc. - may need to introduce upper and lower limits?

Input registration of services, data collections etc. - need documentation and templates - for VO testers, basic xml templates probably most useful - easy web interface for wider input

3 What can we register?

4 Standardisation - of each schema - as a data model - don't need PR version to use a schema (what does PR mean??) must stabilise schemas


D WEB SERVICES

Elizabeth Auden

1 How do services talk to each other? Language not important as long as accessible to web services.

Queries Simple query Complex Query Cone search SIA +? Admin

Query language - XQuery? Others also. - VOQL etc

Returns AstroGrid uss XML just not Next VOTable (already in some use)

Whose problem is taking to back end e.g. data providers' BDs? Not IVOA?

Not everyone needs to use same language behind the scenes

2 Discussion

2a Does cone search need to be in Registry? Yes - can't assume it is always built in, need to register where to find specialised search/extraction services - many services are not built in to VOs e.g. extraacting spectra from x-ray data...

2b See Elizabeth's notes What do registry interfaces need to be able to do? i) Check authenticity ID with global registries ii) Register authority ID with global registry iii) Query the registry and get answer (xml-based in VOResource schema format) iv) Harvest the registry through an interface (incorporates replication?) - web service or httpGET? OAI (uses httpGET? - need one method for Jan 2004! * OAI * v) Define each registry interface web service method in WDSL vi) Documentation for data entry tools vii) Administer registry through add, delete, edit

**RECOMMENDATION: NVO, AstroGrid etc. registries share via OAI for Jan 2004


E REGISTRY HARVESTING

Keith Noddle

1. A registry is itself a resource holding metadata about itself.

- What level - global, local etc? - is registry open to query?

Definitions: Harvest gather metadata Replicate data

Basic security checks - need to follow path.

Do not want to have to choose which registry to search - need automatic referal from local to full registry when required - choice made automatically

2. Demo

a) Use HTTPGet (OAI) b) Specify 'since' date ie all changes since.... ???

3. Future consideration: - replication and harvesting - HTTPGet v. Web services or both - resource attributes - suppress updates if local resource doesn't want DoNotReplicate - delete DeleteResource - how propagate? - etc. ...Deprecated? Use of wrappers

4. Summary AAS Demo Jan2004

**DECISION Get on with it!


F DEMO ACTION Tony Linde

1. Standards Identifier, RM, Schema doc's to be adopted as above

2. Future Web service call interface Standard query language - SQL and XQuery needed Harvesting/replication

2. Demos a) Schema Will be based on Schema 0.9 ready ?20 Oct b) Harvesting Use OAI to return all resources since (date) as XML c) Goals Not - passing between registry projects But - registries must replicate ID and authority Maybe - external services e.g. cone search, data query, ACE

Are present schemas adequate for calling such services in an automated fashion

Need one standard way of getting metadata from a service. Add to schema how you get the full metadata for a resource (ie not just what is in registry) Need to investigate further.

d) Organisation of demos i) Who will publish registries AstroGrid AVO NVO GAVO CDS? + ? - Notify via mailing list

ii) What sort?

META FILEATTACHMENT attr="h" comment="" date="1066310134" name="IVOARegistry.ppt" path="IVOA Registry.ppt" size="260096" user="TonyLinde" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066310371" name="IVOARegistry.pdf" path="IVOA Registry.pdf" size="449628" user="TonyLinde" version="1.1"

Revision 32003-10-16 - MarcoLeoni

 
META TOPICPARENT name="IvoaResReg"
This page will hold presentations, discussions and results of the Registry WG meeting.

Presentations

Notes

many thanks to Anita for these notes from the meeting
Changed:
<
<

>
>

 Registry IVOA notes AMS Richards amsr@jb.man.ac.uk

11:00 2003 October 16

NB two Jan 2004 demos - AAS - AVO Munich 27-28 Jan


A IDENTIFIERS Ray Plante See document on wiki V0.2

Register ID with some global authority to ensure uniqueness etc.

1. Issues to be resolved through practice:

* Mirrors/versions have different identifiers - harvesters need to know mirror etc. info too.

* Permanence - can identifiers be recycled?

* 'ivo' xml format implies resource is registered somewhere ADEC (organisation of NASA astrophysical data centres) adopted IVOA syntax. US data publishing lined to (refereed) journal papers

* Need standard for logical identifiers - need to automatically point to relevant version. Are Identifiers 'logical' or static? Is there a contradiction here?

2. FAQs Who assigns IDs? Publishing registry (check to avoid duplication)

What is ID content? Conventions to encourage uniqueness but decision ultimately up to authority behind ID (as long as it is in valid characters).

Where register? One place (so must have all registries accessible via global registry)

3. Discussion How to manage global requirements espec uniquness of ID? How police? Trust - duplicate won't be replicated so it won't work - practical disincentive. Need good explanation for data providers Learn what mechanisms are needed in practice e.g. Jan 2004 demo

**DECISION: Support recommendation to adopt IVOA Identifiers V0.2


B RESOURCE METADATA Bob Hanisch see wiki doc V0.82

See Changes

creator removed, **address/tel. should be removed

Title - verbose (e.g. Hubble Space Telescope) required ShortName optional

Keywords - ideally one source i.e. IAU Thesaurus meanwhile will be more than one but must have namespace(?) with definitions

Wavelength coverage - mm-wave added, submm may be needed? Suggestion - submm instead of mm? e.g. 10 micron to 1 mm ???? Can't be perfect, can overlap as necessarily

UCDs - will there be a problem with getting them two ways? But they aren't compulsory. Having UCDs in registry if required allows quicker searches in some cases. More important, it is allows it to be in model.

Data Quality - agree outline of section, discuss details in light of practice (e.g. distinguish quality of data v. metadata). Data provider assigns A B or C

**DECISION: Support recommendation to adopt IVOA RM v0.82


C VOResource

Ray Plante v0.8.4

Covers most concepts in RM Working Doc Extensions will add additional metdata for particular classes of resources as required.

NB See XML Schema style guide provided by Ray Plante

1 Metadata

Minimum required metadata just 7-8 elements

* Identity Service metadata - how to invoke/access, virtual address

* Community

* Data service - collection - sky service - tabular sky service

etc.

2 Issues

Use of STC metadata schema (Rots) ultimate goal - Use temporary simple version for now

Using/reusing VOTable elements? for table columns, service parameters

>From discussion:
Duplication of services - how resolve

Coverage etc. - may need to introduce upper and lower limits?

Input registration of services, data collections etc. - need documentation and templates - for VO testers, basic xml templates probably most useful - easy web interface for wider input

3 What can we register?

4 Standardisation - of each schema - as a data model - don't need PR version to use a schema (what does PR mean??) must stabilise schemas


D WEB SERVICES

Elizabeth Auden

1 How do services talk to each other? Language not important as long as accessible to web services.

Queries Simple query Complex Query Cone search SIA +? Admin

Query language - XQuery? Others also. - VOQL etc

Returns AstroGrid uss XML just not Next VOTable (already in some use)

Whose problem is taking to back end e.g. data providers' BDs? Not IVOA?

Not everyone needs to use same language behind the scenes

2 Discussion

2a Does cone search need to be in Registry? Yes - can't assume it is always built in, need to register where to find specialised search/extraction services - many services are not built in to VOs e.g. extraacting spectra from x-ray data...

2b See Elizabeth's notes What do registry interfaces need to be able to do? i) Check authenticity ID with global registries ii) Register authority ID with global registry iii) Query the registry and get answer (xml-based in VOResource schema format) iv) Harvest the registry through an interface (incorporates replication?) - web service or httpGET? OAI (uses httpGET? - need one method for Jan 2004! * OAI * v) Define each registry interface web service method in WDSL vi) Documentation for data entry tools vii) Administer registry through add, delete, edit

**RECOMMENDATION: NVO, AstroGrid etc. registries share via OAI for Jan 2004


E REGISTRY HARVESTING

Keith Noddle

1. A registry is itself a resource holding metadata about itself.

- What level - global, local etc? - is registry open to query?

Definitions: Harvest gather metadata Replicate data

Basic security checks - need to follow path.

Do not want to have to choose which registry to search - need automatic referal from local to full registry when required - choice made automatically

2. Demo

a) Use HTTPGet (OAI) b) Specify 'since' date ie all changes since.... ???

3. Future consideration: - replication and harvesting - HTTPGet v. Web services or both - resource attributes - suppress updates if local resource doesn't want DoNotReplicate - delete DeleteResource - how propagate? - etc. ...Deprecated? Use of wrappers

4. Summary AAS Demo Jan2004

**DECISION Get on with it!


F DEMO ACTION Tony Linde

1. Standards Identifier, RM, Schema doc's to be adopted as above

2. Future Web service call interface Standard query language - SQL and XQuery needed Harvesting/replication

2. Demos a) Schema Will be based on Schema 0.9 ready ?20 Oct b) Harvesting Use OAI to return all resources since (date) as XML c) Goals Not - passing between registry projects But - registries must replicate ID and authority Maybe - external services e.g. cone search, data query, ACE

Are present schemas adequate for calling such services in an automated fashion

Need one standard way of getting metadata from a service. Add to schema how you get the full metadata for a resource (ie not just what is in registry) Need to investigate further.

d) Organisation of demos i) Who will publish registries AstroGrid AVO NVO GAVO CDS? + ? - Notify via mailing list

ii) What sort?

META FILEATTACHMENT attr="h" comment="" date="1066310134" name="IVOARegistry.ppt" path="IVOA Registry.ppt" size="260096" user="TonyLinde" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066310371" name="IVOARegistry.pdf" path="IVOA Registry.pdf" size="449628" user="TonyLinde" version="1.1"

Revision 22003-10-16 - TWikiGuest

 
META TOPICPARENT name="IvoaResReg"
This page will hold presentations, discussions and results of the Registry WG meeting.

Presentations

Notes

many thanks to Anita for these notes from the meeting
Changed:
<
<
>
>

 Registry IVOA notes AMS Richards amsr@jb.man.ac.uk

11:00 2003 October 16

NB two Jan 2004 demos - AAS - AVO Munich 27-28 Jan


A IDENTIFIERS Ray Plante See document on wiki V0.2

Register ID with some global authority to ensure uniqueness etc.

1. Issues to be resolved through practice:

* Mirrors/versions have different identifiers - harvesters need to know mirror etc. info too.

* Permanence - can identifiers be recycled?

* 'ivo' xml format implies resource is registered somewhere ADEC (organisation of NASA astrophysical data centres) adopted IVOA syntax. US data publishing lined to (refereed) journal papers

* Need standard for logical identifiers - need to automatically point to relevant version. Are Identifiers 'logical' or static? Is there a contradiction here?

2. FAQs Who assigns IDs? Publishing registry (check to avoid duplication)

What is ID content? Conventions to encourage uniqueness but decision ultimately up to authority behind ID (as long as it is in valid characters).

Where register? One place (so must have all registries accessible via global registry)

3. Discussion How to manage global requirements espec uniquness of ID? How police? Trust - duplicate won't be replicated so it won't work - practical disincentive. Need good explanation for data providers Learn what mechanisms are needed in practice e.g. Jan 2004 demo

**DECISION: Support recommendation to adopt IVOA Identifiers V0.2


B RESOURCE METADATA Bob Hanisch see wiki doc V0.82

See Changes

creator removed, **address/tel. should be removed

Title - verbose (e.g. Hubble Space Telescope) required ShortName optional

Keywords - ideally one source i.e. IAU Thesaurus meanwhile will be more than one but must have namespace(?) with definitions

Wavelength coverage - mm-wave added, submm may be needed? Suggestion - submm instead of mm? e.g. 10 micron to 1 mm ???? Can't be perfect, can overlap as necessarily

UCDs - will there be a problem with getting them two ways? But they aren't compulsory. Having UCDs in registry if required allows quicker searches in some cases. More important, it is allows it to be in model.

Data Quality - agree outline of section, discuss details in light of practice (e.g. distinguish quality of data v. metadata). Data provider assigns A B or C

**DECISION: Support recommendation to adopt IVOA RM v0.82


C VOResource

Ray Plante v0.8.4

Covers most concepts in RM Working Doc Extensions will add additional metdata for particular classes of resources as required.

NB See XML Schema style guide provided by Ray Plante

1 Metadata

Minimum required metadata just 7-8 elements

* Identity Service metadata - how to invoke/access, virtual address

* Community

* Data service - collection - sky service - tabular sky service

etc.

2 Issues

Use of STC metadata schema (Rots) ultimate goal - Use temporary simple version for now

Using/reusing VOTable elements? for table columns, service parameters

>From discussion:
Duplication of services - how resolve

Coverage etc. - may need to introduce upper and lower limits?

Input registration of services, data collections etc. - need documentation and templates - for VO testers, basic xml templates probably most useful - easy web interface for wider input

3 What can we register?

4 Standardisation - of each schema - as a data model - don't need PR version to use a schema (what does PR mean??) must stabilise schemas


D WEB SERVICES

Elizabeth Auden

1 How do services talk to each other? Language not important as long as accessible to web services.

Queries Simple query Complex Query Cone search SIA +? Admin

Query language - XQuery? Others also. - VOQL etc

Returns AstroGrid uss XML just not Next VOTable (already in some use)

Whose problem is taking to back end e.g. data providers' BDs? Not IVOA?

Not everyone needs to use same language behind the scenes

2 Discussion

2a Does cone search need to be in Registry? Yes - can't assume it is always built in, need to register where to find specialised search/extraction services - many services are not built in to VOs e.g. extraacting spectra from x-ray data...

2b See Elizabeth's notes What do registry interfaces need to be able to do? i) Check authenticity ID with global registries ii) Register authority ID with global registry iii) Query the registry and get answer (xml-based in VOResource schema format) iv) Harvest the registry through an interface (incorporates replication?) - web service or httpGET? OAI (uses httpGET? - need one method for Jan 2004! * OAI * v) Define each registry interface web service method in WDSL vi) Documentation for data entry tools vii) Administer registry through add, delete, edit

**RECOMMENDATION: NVO, AstroGrid etc. registries share via OAI for Jan 2004


E REGISTRY HARVESTING

Keith Noddle

1. A registry is itself a resource holding metadata about itself.

- What level - global, local etc? - is registry open to query?

Definitions: Harvest gather metadata Replicate data

Basic security checks - need to follow path.

Do not want to have to choose which registry to search - need automatic referal from local to full registry when required - choice made automatically

2. Demo

a) Use HTTPGet (OAI) b) Specify 'since' date ie all changes since.... ???

3. Future consideration: - replication and harvesting - HTTPGet v. Web services or both - resource attributes - suppress updates if local resource doesn't want DoNotReplicate - delete DeleteResource - how propagate? - etc. ...Deprecated? Use of wrappers

4. Summary AAS Demo Jan2004

**DECISION Get on with it!


F DEMO ACTION Tony Linde

1. Standards Identifier, RM, Schema doc's to be adopted as above

2. Future Web service call interface Standard query language - SQL and XQuery needed Harvesting/replication

2. Demos a) Schema Will be based on Schema 0.9 ready ?20 Oct b) Harvesting Use OAI to return all resources since (date) as XML c) Goals Not - passing between registry projects But - registries must replicate ID and authority Maybe - external services e.g. cone search, data query, ACE

Are present schemas adequate for calling such services in an automated fashion

Need one standard way of getting metadata from a service. Add to schema how you get the full metadata for a resource (ie not just what is in registry) Need to investigate further.

d) Organisation of demos i) Who will publish registries AstroGrid AVO NVO GAVO CDS? + ? - Notify via mailing list

ii) What sort?

Changed:
<
<
>
>
 
META FILEATTACHMENT attr="h" comment="" date="1066310134" name="IVOARegistry.ppt" path="IVOA Registry.ppt" size="260096" user="TonyLinde" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066310371" name="IVOARegistry.pdf" path="IVOA Registry.pdf" size="449628" user="TonyLinde" version="1.1"

Revision 12003-10-16 - TonyLinde

 
META TOPICPARENT name="IvoaResReg"
This page will hold presentations, discussions and results of the Registry WG meeting.

Presentations

Notes

many thanks to Anita for these notes from the meeting

Registry IVOA notes  AMS Richards amsr@jb.man.ac.uk

11:00 2003 October 16

NB two Jan 2004 demos
- AAS
- AVO Munich 27-28 Jan

---------------------------------------------------------------
A IDENTIFIERS
Ray Plante See document on wiki V0.2

Register ID with some global authority to ensure uniqueness etc.

1.
Issues to be resolved through practice:

*
Mirrors/versions have different identifiers - harvesters need to know mirror etc. info too.

*
Permanence - can identifiers be recycled?

*
'ivo' xml format implies resource is _registered_ somewhere ADEC (organisation of NASA astrophysical data centres) adopted IVOA syntax. US data publishing lined to (refereed) journal papers

*
Need standard for logical identifiers - need to automatically point to relevant version. Are Identifiers 'logical' or static? Is there a contradiction here?


2.
FAQs
Who assigns IDs?   Publishing registry (check to avoid
         duplication)

What is ID content?   Conventions to encourage uniqueness but
         decision ultimately up to authority behind ID
         (as long as it is in valid characters).

Where register?      One place (so must have all registries
         accessible via global registry)

3.
Discussion
How to manage global requirements espec uniquness of ID?
How police?
Trust - duplicate won't be replicated so it won't work - practical disincentive. Need good explanation for data providers Learn what mechanisms are needed in practice e.g. Jan 2004 demo

**DECISION: Support recommendation to adopt IVOA Identifiers V0.2

---------------------------------------------------------------
B RESOURCE METADATA
Bob Hanisch see wiki doc V0.82

See Changes

creator removed,
**address/tel. should be removed

Title - verbose (e.g. Hubble Space Telescope) required ShortName optional

Keywords - ideally one source i.e. IAU Thesaurus
           meanwhile will be more than one but must have namespace(?)
           with definitions

Wavelength coverage - mm-wave added, submm may be needed?
            Suggestion - submm instead of mm?
                      e.g. 10 micron to 1 mm ????
            Can't be perfect, can overlap as necessarily

UCDs     - will there be a problem with getting them two ways?
           But they aren't compulsory. Having UCDs in registry if
           required allows quicker searches in some cases. More
           important, it is allows it to be in model.

Data Quality - agree outline of section, discuss details in light of
          practice (e.g. distinguish quality of data
          v. metadata).
          Data provider assigns A B or C

**DECISION: Support recommendation to adopt IVOA RM v0.82

---------------------------------------------------------------

C
VOResource

Ray Plante v0.8.4

Covers most concepts in RM Working Doc
Extensions will add additional metdata for particular classes of resources as required.

NB See XML Schema style guide provided by Ray Plante

1
Metadata

Minimum required metadata just 7-8 elements

*
Identity
Service metadata - how to invoke/access, virtual address

*
Community

*
Data service - collection
        - sky service
        - tabular sky service

etc.

2
*Issues*

Use of STC metadata schema (Rots) ultimate goal
- Use temporary simple version for now

Using/reusing VOTable elements? for table columns, service parameters

>From discussion:
Duplication of services - how resolve

Coverage etc. - may need to introduce upper and lower limits?

Input registration of services, data collections etc.
      - need documentation and templates
      - for VO testers, basic xml templates probably most useful
      - easy web interface for wider input

3
What can we register?

4
Standardisation
- of each schema
- as a data model
- don't need PR version to use a schema (what does PR mean??) *must stabilise schemas*

---------------------------------------------------------------

D WEB SERVICES

Elizabeth Auden

1
How do services talk to each other?
Language not important as long as accessible to web services.

Queries
   Simple query
   Complex Query
   Cone search
   SIA
   +?
Admin

Query language - XQuery? Others also.
          - VOQL etc

Returns
   AstroGrid uss XML just not
   Next VOTable (already in some use)

Whose problem is taking to back end e.g. data providers' BDs? Not IVOA?

Not everyone needs to use same language behind the scenes

2
Discussion

2a
Does cone search need to be in Registry? Yes - can't assume it is always built in, need to register where to find specialised search/extraction services - many services are not built in to VOs e.g. extraacting spectra from x-ray data...

2b
See Elizabeth's notes
What do registry interfaces need to be able to do?
i)   Check authenticity ID with global registries
ii)  Register authority ID with global registry
iii) Query the registry and get answer (xml-based in
     VOResource schema format)
iv)  Harvest the registry through an interface (incorporates
     replication?)  - web service or httpGET? OAI (uses httpGET?
     - need one method for Jan 2004! ** OAI **
v)   Define each registry interface web service method in WDSL
vi)  Documentation for data entry tools
vii) Administer registry through add, delete, edit

**RECOMMENDATION: NVO, AstroGrid etc. registries share via OAI for
   Jan 2004

---------------------------------------------------------------

E REGISTRY HARVESTING

Keith Noddle

1.
A registry is itself a resource holding metadata about itself.

- What level - global, local etc? - is registry open to query?

Definitions:
_Harvest_ gather metadata
_Replicate_ data

Basic security checks - need to follow path.

Do not want to have to choose which registry to search
- need automatic referal from local to full registry when required
- choice made automatically

2.
Demo

a) Use HTTPGet (OAI)
b) Specify 'since' date ie all changes since....
???

3.
Future consideration:
- replication and harvesting
  - HTTPGet v. Web services or both
- resource attributes
  - suppress updates if local resource doesn't want DoNotReplicate
  - delete DeleteResource - how propagate?
  - etc. ...Deprecated?
Use of wrappers

4.
Summary
AAS Demo Jan2004

**DECISION Get on with it!

---------------------------------------------------------------
F
DEMO ACTION
Tony Linde

1.
Standards
Identifier, RM, Schema doc's to be adopted as above

2.
Future
Web service call interface
Standard query language - SQL and XQuery needed Harvesting/replication

2.
Demos
a) Schema
   Will be based on Schema 0.9 ready ?20 Oct
b) Harvesting
   Use OAI to return all resources since (date) as XML
c) Goals
Not   - passing between registry projects
But   - registries must replicate ID and authority
Maybe   - external services e.g. cone search, data query, ACE

Are present schemas adequate for calling such services _in an automated fashion_

Need one standard way of getting metadata from a service.
*Add to schema* how you get the full metadata for a resource (ie not just what is in registry) Need to investigate further.

d) Organisation of demos
i)   Who will publish registries
     AstroGrid AVO NVO GAVO CDS? + ?
     - _Notify via mailing list_

ii)  What sort?
META FILEATTACHMENT attr="h" comment="" date="1066310134" name="IVOARegistry.ppt" path="IVOA Registry.ppt" size="260096" user="TonyLinde" version="1.1"
META FILEATTACHMENT attr="h" comment="" date="1066310371" name="IVOARegistry.pdf" path="IVOA Registry.pdf" size="449628" user="TonyLinde" version="1.1"
 
This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback