Requirements and Use Cases for New Registry Search Interface

This page aims to capture requirements and use cases for a new (post-RI v1.0) registry search interface.

The RI v1.0 spec provides two interfaces for searching: one is a required ADQL-based interface and the other is an optional XQuery-based query. Both use SOAP. Since a searchable registry is only required to support the ADQL interface, this is the one that applications would prefer to be interoperable across registries. These interfaces, and particularly the ADQL interface, have following disadvantages:

• They are the only remaining recommended IVOA service interface based on SOAP. Because of the complexities introduced by SOAP and WSDLs for both providers and clients, our community has evolved to prefer RESTful interfaces.
• It relies on an old version of ADQL (v1.0) that itself never made it as an IVOA Recommendation.
• The ADQL interface returns either full VOResource records (some of which can be memory-bustingly large) or just their identifiers; most applications only want a few pieces of information from the VOResource records

This effort is motivated to address these problems by providing a searchable interface that is simpler and scriptable.

Use Cases

Consider the following as potentially useful queries and ways to express queries. not all of which may be included explicitly in a standard interface. In addition to adding more use cases, feel free to add comments to those that already appear that would indicate their relative importance or role for an application. For the purposes of discussion, annotate with your name (or linked initials).

-- Ray Plante (RP)

Interface clients

Clients that would interact with a programmable web search interface:

• General purpose discovery portals (like the NVO Directory)
• Desktop applications that work with specific types of services (e.g. Topcat)
• Workflow environments
• Ad hoc scripts looking for specific types of services.

Sample Queries

• find all TAP services; return their accessURLs
• find all SIA services that might have spiral galaxies
  • this is a common type of query desired by interactive users: they want services of a particular type related to a subject --RP
• find all SIA services that provides infrared images.
• find all searchable catalogs that provide a column containing redshift
• find all the resources that registered by me (or another publisher)
• find all searchable registries; return the accessURLs to their search interfaces
  • desktop apps can use this query to cache a list of alternative registries should the primary one be down.
• find all resources of a particular type (e.g. Authority)
• return the resource having a particular identifier

Query Interactions

This section looks at possible service interaction patterns.

Single Resource resolution

1. Input: identifier
   Output: a full VOResource record
   • This might be most useful for browsing purposes --RP
2. GetCapabilities
   Input: identifier
   Output: a list of supported capabilities
3. GetTables
   Input: identifier
   Output: a VOResource TableSet description
4. Get the accessURL
   Input: identifier, capability type (by standardID?)
   Output: access URL
   • Considered most important to desktop apps and scripts --RP

Search Queries

Search queries provide return a list of records that match a set of input constraints.

We note that a resource record can, in general, have zero or more capabilities associated with it, and each capability can have multiple interfaces (e.g. supporting different versions of a capability), each with its primary own accessURL. Thus, the result records return from search queries could represent ...

• a list of resources
• a list of capabilities
  • depending on the output details, this is probably the most convenient record type for clients looking to access services --RP
• a list of interfaces

In principle, these could come back in a variety of formats, including table formats. Desktop applications, portal apps, and workflow builders will usually want to get a few pieces of information (most notably, a service's accessURL; see opening motivation above). It would be both easier for these clients and more efficient if the client could specify exactly what information they want back.

Possible output forms

1. an XML list of VOResource records
   • This is provided by the current RI search interface; however, there are no known cases where this output is the most desired. As long as VOResurce records are available individually via single identifier resolution, we can probably do without supporting this type of query. --RP
2. VOTable format
   • This format would be convenient for both portal clients that might want to display (and allow the user to manipulate) the results and scripters (with access to an easy to use VOTable parser) --RP
3. CSV table format
   • This format, depending on the specific clients, might be most conenient for scripters and specialized desktop tools (Topcat?) --RP
4. a list of space-delimited identifiers

Questions:

1. Would it be useful to allow a user to select a non-standard format through the standard search interface?

Ways to constrain search results

1. keywords: find resources related to a particular subject
2. keywords + capability type: find SIA services relatedd to a particular subject
   • not currently support by the current RI, a number of developers have since requested this. --RP
3. ADQL/SQL-like constraints
   • Given current experience with the RI search interface, it is unlikely that clients will need to place constraints on any arbitrary data available in a VOResource record. That is, not all VOResource metadata need be searchable. --RP
   • Despite the above statement, the query constraint must be extensible to the different specialized capabilities (e.g. TAP, SIA, etc.) including future ones. In other words, we should not have to update the new RI spec everytime a new registry exetension is added. --RP
4. ADQL/SQL-like constraints + keywords: find resources related to a particular subject where publisher like %NASA%
   • I have found queries of this type particularly helpful for exploring the contents of the registry for administrative purposes. Publishers could use this capability to explore how their own resources respond to different keyword queries. --RP

Requirements

Here we list any functional and technical requirements (some deriving from the above use cases).

• the interface should be REST-like based primarily on HTTP Get.

Implementation Considerations

Here we collect considerations for implementations that do not necessarily derive from requirements (but may feed into them).

XML vs. Relational DB backends

The current RI v1.0 search interface reflects an attempt to provide a search interface that could be implemented against either an XML database backend or a relational one. Can this be achieved with a new, RESTful design? Can at least part of the interface be database agnostic? Is it worth trying?

TAP Interface

A TAP interface that supports a "Registry Data Model" has an advantage to developers (in particular, scripters) because it is potentially a query interface they already know.

• A simplified view of the VOResource model could probably be mapped to three tables: resources, capabilities, and table. Some DB Views that dynamically join these tables could make things more convenient for TAP users. For example, one view could combine capability information (including the accessURLs) with resource level metadata (like identifier, shortName, and title). --RP