CSP telecon 9 November 2018
College Park Interop

At the meeting: Ciau XXx, Mark Lacy, Raffaele D’Abrusco, 
Christophe Arviset, Peppi Fabbiano, Janet Evans, Gregory Dubbois-Felsmann, 
Debbie Baines, Kai Polsterer and Ada Nebot

Agenda:

Discussion on the portal
Plans for a Focus session for Paris interop
 - portal or science science platforms

Summary from CSP discussion:
 
- Discussion on the portal might be slightly premature
- IVOA portal was driven more towards first re-designing the
website

PF: in principle, it was already agreed that the website and
the portal would be hosted in Trieste thanks to Fabio.

MA: It was not completely clear whether there were funds 
available from Trieste for this or not.

GDF: rebranding the website might be more a task for the Media
group, while the CSP should be more involved in making the data
available. The CSP should limit itself to try to simplify how 
users will discover data providers that use IVOA or are part of
it

PF: the CSP could help testing the new IVOA portal design.

KP: A portal for IVOA data might not be what users need, but 
rather something like astropy, which is a community-driven project,
even if VO implementations in astropy so far have been creating
inefficient access to VO resources.

KP: what happens today is that the performance to access VO data
via astropy is not good enough and we finally had to dump data
from Edimburg directly to our system to be able to do the analysis
that we need to do. Alternatively we could improve python 
implementations.

GDF: another high priority is to give access to large data surveys, 
which requires a redefinition of certain interfaces to avoid many
small transactions in favor of more sensible technical means to 
facilitate consumption of massive amounts of data.

GDF: the second priority is the client side of the story: the IVOA
itself should not only do python code, but probably it should bridge
the link between the data providers and the users via hackathons, 
new software, language binding and other actions to facilitate the
access from users to the data.

GDF: the final thing we could do is to require reference implementations
for clients (while traditionally we have focussed more on the server
implementations). Since the community is using python, it would make
a lot of sense to enforce python implementations.

MA/PF: a hackathon would make sense a lot of sense for the Paris interop
than maybe a Focus Session with presentations etc..

CA: the IVOA probably needs to start thinking outside of the box to
recognize that outside community is doing work. So if the CSP would 
request e.g. astropy implementations or any other thing that the
community needs, then it is the TCG role to define which standards could
be amenable to fulfill the requests from the users. 

CA: An idea might be to evolve into the creation of a set of s/w libraries
that data providers 

AN: If we organize hackathons, it is essential that we involve the external
community. 

GDF: there are astropy coordination meetings, the next one in a few months,
and maybe the IVOA could send some representative to coordinate the 
hackathons or potential developments.

PF: NAVO is doing a hackathon in the AAS but more geared towards the external
users.

CA: standard developers might actually be quite happy to dedicate one day
once in a while to just develop software for the community..

MA: we need the CSP to guide the scope of any such hackathon involving
external participants.

RD: one of the things that we would like to see is a better mapping between
the data models and data structures defined in the astropy library that 
users are using. An example is the skycoord data structure, which happens
to have been provided by Omar and presented at this very same interop! 

GDF: another example is if you download a table with RA and Dec, a new
table should generate a python object that recognizes that those two
columns are coordinates and automatically allows them to be plotted e.g.
on top of an image. This is the type of things that we should be doing.
This is one of the things that Erik Tollerund mentioned when he presented
to us in the Victoria interop. 

MA: maybe we could think about a astropy - IVOA hackathon ?

KP: we also need some feedback system to deal with things not working,
this would not work in the context of the astropy unless there is a clear
why to identify the IVOA as provider of this service. This would help
in case e.g a data centre starts receiving lots of calls from some python
interface that is related to the IVOA.

CA: maybe the CSP could develop a set of use cases and then use it to
define the scope of a hackathon to be run together with the astropy
community and users.

GDF: an example is a standard that would allow to make thousands of 
image cut-outs per second against any data service. This should be made
available by the IVOA.

MA: this idea was already proposed some years ago and the response to 
that was back then was to use TAP.

AN: in the schools that we organize, we could also have python examples
with notebooks with external tutors from astropy.

CA: we could think about liaising with .Astronomy, schools, etc, 

Action on the CSP: define a theme for a hackathon to be organized for
Paris (maybe via an email discussion plus a telecon) to be presented to
the exec well before the Paris interop.

JE: I see two parts to this: the standards need to be there that people
can use to start with and then maybe people in Paris can elaborate the
ideas and then at the next fall interop, together with the ADASS and
potentially more people, potentially not related to the IVOA.

RD: It is crucial that we get participation by non-IVOA people to get
real feedback from users not familiar with our standards.

CA: probably we should give this idea a try in May first and then if it
works well, then we can bring it for the ADASS meeting, which is certainly
broader.

GDF: maybe we can start linking to the NAVO python hackathon in the AAS.

ML: getting young people to see how easy it is to work with VO-supported
infrastructure would be great.

JE: community has already reported that having astroquery implementations
of our services would win the IVOA a lot of enthusiastic users.

MA: there are queries to SIMBAD and/or Vizier developed by the astroquery
community. It would make sense that all the work invested in the TAP
development would have a visible face and interfaces in astropy, but who
owns that development then? Probably no one at the astropy have the 
community has the resources 

CL: I can report on the difficulties that I have experienced myself retrieving
data from large surveys into my harddisk to be able to work with it, e.g.
with Panstarrs data. Enabling that for real users would impose pretty serious
requirements on the IVOA standards.

PF: would you have in China the resources to help developing the type
of software you mention?

CL: all this kind of functionality could be developed by large surveys.

KP: having clean interfaces to the users and a powerful data synchronization
system that sync data with certain frequencies such that it is transparent
to the users without they having to know how data is synchronized is what 
people need.

GDF: we are implementing the TAP interface on our data sytems, plus a front
end based on the CADC software stack.

GDF: We will need to review the old priorities and understand the old priorities
and see where they are (time-series and multi-dimensional).

MA: the CSP could make discuss with the Time-Domain interest group the 
status. The multi-D work can be considered mostly done.

GDF: we can mention the plan to meet with the TDIG at the closing.

GDF: I am not too enthusiastic about the idea of a portal because it is 
against the idea of the registry, which would allow anyone to develop a
portal. The issue to investigate is the data discoverability which should
be investigated, in particular in the context of python. 

KP: it would be great to use “natural language” processing to be able to
help users identifying which protocol and/or service might be appropriate
for the kind of data they are looking for.

CA: I see three priorities: 
	- use cases for python / astropy
 	- time-domain
	- the portal or data discoverability

AN: for the hackathon, wouldn’t it be useful to contact the astropy person
involved in the time-domain development?

GDF: definitively we should investigate the possibility of inviting astropy
people for the Paris interop. 

GDF: in general, we need to develop language bindings for our developments.

GDF: I want to propose another priority, which is handling of large datasets.