gather.town registry workshop, Nov 3, 2021.

pyvo and data discovery in the registry, working session on changes.

Notes: Theresa Dower

Main presentation: Tess Jaffe / Markus Demleitner

Attendance: 14

Background:

Tess Jaffe & NAVO workshop authors/runners for AAS have a series of use cases for registry search. The session is a mix of discussion and collaborative test on a jupiter hub. We are covering differences between existing registry search behavior and proposed data discovery, with the goal of hiding some registry complexity without removing existing functionality.

Issue that spawned this: https://github.com/astropy/pyvo/issues/238

Markus' branch with changes: https://github.com/msdemlei/pyvo/tree/add-discoverdata

NAVO's notebooks: https://github.com/nasa-navo/navo-workshop

Major changes with the discover data branch:

Returned columns are entirely changed; returns resources vs services.

Change in approach/workflow: we're not looking for 'services' we're looking for resource info and can drill down to the same interface level.

Challenges and obstacles :

• Tess: Most changes can be rewritten around in the notebooks. 1 issue - if we're not setting #aux inclusion as the default (a good idea, because jargon), we need a way to include these capabilities: in some new function? without jargon?

• Tom & Adrian: current search requires a lot of understanding of registry internals.... is this an opportunity to create a higher-level interface?

• We are moving from plain programmatic access to interactive access, with the notebooks.

• Compatibility: do old queries from old notebooks return the same data?

• We could register the notebooks! http://dc.g-vo.org/VOTT per https://ivoa.net/documents/Notes/EDU/index.html

• Hendrik: The type for this may have a last-checked timestamp to curate for stale resources.

Markus reviews suggested new workflow:

"Searching for services was a mistake" (but much of the VO still relies on it). Change to search pattern to start searching for resources, then get_service, which will constrain the type (and allow service-specific functionality?)

Tess: This doesn't help much for use case where we know we're looking for TAP for this data. (But we could construct a query for that in either of these workflows?)

Takeaways:

• #includeaux version already in pyvo regsearch from first stab at this should be removed when we have a better solution, no matter what it is

• This could be an opportunity for higher-level data discovery workflow and functionality?

• Having parallel functionality is probably necessary, will need documentation/communication about the different modes.

• Action on ???: Test backward compatibility: not just unit tests but existing queries and workflows

• Action on NAVO: Translate notebooks to Markus' workflow and see what works well, see what works best

• Action on Theresa: Attempt registering notebooks http://dc.g-vo.org/VOTT per https://ivoa.net/documents/Notes/EDU/index.html

• Action on Theresa: follow up on Note, other interest in this recently. Endorsed note or doc?