BoF2019 < IVOA

IVOA Web>IvoaEvents>InterOpOct2019>InterOpOct2019Gws>BoF2019 (2019-10-12, GiulianoTaffoni) (raw view)

---+ Science Platform: Towards Data Science
---++ ADASS 2019 - Gronigen - 9-10-2019

Room 1-2

Participants: ~50
---+++ Discussion Topics

<span style="background-color: transparent;">&bull; What is your science platform?</span>

&bull; How can our platforms work together or interoperate?

&bull; How are we approaching HPC or cloud resources (public or provate)

&bull; The role of IVOA standards.

&bull; Let make our platform sustainable.
---+++ Participats Presentations

   * JJ Kavelaars (CADC) [[%ATTACHURL%/01-ADASS2019_SciencePlatform_BoF_CANFAR.pptx][Slides]]
   * Gerard Lemson(JHU - IDIES) [[%ATTACHURL%/02-science platforms bof - gerard.pptx][Slides]]
   * Steven Crawford (STScl) [[%ATTACHURL%/03-adass_bof.pptx][Slides]]
   * Andre Schaaff (CDS) [[%ATTACHURL%/04-ADASS-2019-BoF-SP-CDS.pdf][Slides]]
   * Christine Banek (LSST)
   * Petr Skoda (VO-CLOUD) [[%ATTACHURL%/06-BoF-VOCLOUD-Skoda.pdf][Slides]]
   * Zheng Mayer(Astron ESCAPE Wp5) [[%ATTACHURL%/07-2019-10-09_ESCAPE_ESAP.pdf][Slides]]
   * Sonia Zorba (IA2) [[%ATTACHURL%/08-OAuth2 _ OIDC on Science Platforms_V2.pdf][Slides]]
   * Dave Morris+Marco Molinaro (Virtual Observatory) [[%ATTACHURL%/09-MolinaroMorris_ADASSXXIX_BoF8.pdf][Slides]]
[[%ATTACHURL%/ADASS2019-BoF-Summary.pptx][ADASS 2019 BoF Summary Session Slides]]
---+++ Discussion

*CADC* introduces the CANFAR platform and the ARCADE science platform on top of it. ARCADE is a Desktop oriented Science Platform for data reduction. The design has been done for ALMA users.

CADC platform is "entirely" based on IVOA standards. It offers DOI to assign to VOSpace nodes.

Q: how can we implement an efficent data transfer? Can we introduce a off line data transfer service?

---

<strong>SciServer </strong>is a platform user oriented that implements resource sharing,<span style="background-color: transparent;"> agnostic storage to </span><span style="background-color: transparent;"> host and share datasets, </span><span style="background-color: transparent;">query and analysis tools. Is can be used as </span><span style="background-color: transparent;">collaborative workspace. It can be installed with K8S and HELM. It can be used to explore CPU and GPU computing or to access </span><span style="background-color: transparent;">DASK or </span><span style="background-color: transparent;">SQL Server 2019 Big Data Studio.</span>

<span style="background-color: transparent;">Q: how can I make my SP interoperate? in practice how can a user execute a contaner based on my image on another science platform?</span>

<span style="background-color: transparent;">Q: sustainbility problem, I have a finite resourcepool, can I make my SP able to deploy on commercial clouds so that users pay for it? Is it a functional sustainability approach?</span>

---

<strong>STScl </strong>open source oriented science platform that profit of github and AWS clouds. It uses K8S HELM and Docker and provide Jupyterh Labs to users.

Q: _Security_ How do we secure the environment from external and internal sources? How do we control exclusive access data in the environment?

Q: <em>Collaboration </em>How do we provide an environment that enables collaboration while being secure?

Q: <em>Observability </em>What are the important metrics to monitor? How do we have real time insight into the system?

Q: <em>Cost </em>How do we control costs for different users and different use cases? How do we provide access for different users?

---

<strong>CDS Science platform concepts </strong>A science platform could provide an access to Simbad and VizieR through APIs, visualisation through Aladin Lite (ipyaladin) and python tools for HiPS and MOC, computational facilities to X-Match catalogues.

Comment: It must be well framed and sized, complementary to the services and developed with scientists for scientists =&gt; *%BLUE%scientific sustainability%ENDCOLOR%*

Q: storage space is necessary and must be allocated

Q: Can we accept external Containers?

---

<strong>LSST </strong>science platform allows accessing to data and processing for LSST. There are three interactive ways of exploring the data portal (discovery, structured work flow between datasets, querries), notebook environment (ipython, contemporary hub technologies) and public API/VO and REST.

---

<strong>VO-CLOUD </strong>is an example of science paltform that implements batch processing based on UWS VO standard and it is focusing on interactive Machine learning: "active learning".

---

<span style="background-color: transparent;">The European funded project </span> *ESCAPE* <span style="background-color: transparent;"> is developing a science platform on top of European Open Science Cloud. It will include VO data access standards, AARC compliant authentication and authorization, containers from the ESCAPE software market place and EOSC market-place.</span>

Q: what can IVOA provides to simplify the design and implementation?
---

<span style="background-color: transparent;">The </span> *Virtual Observatory* <span style="background-color: transparent;"> point of view (not IVOA) identify some important items to build real interoperability and in particular what is missing and what is already available:</span>

   * <span style="background-color: transparent;">data discovery and access, open VO standards</span>
   * interoperable AAI solutions (IVOA GWS working on it ... )
   * metadata to characterize network proximity
   * IVOA metadata EXIST for describing data, NEED to have them for code
   * there probably already are some standards for this
   * we should find/evaluate and adopt them rather than invent new ones
---

<span style="background-color: transparent;">The use of </span><strong style="background-color: transparent;">OAuth tokens </strong><span style="background-color: transparent;">for authentication has been discussed by IA2 Sonia Zorba that introduce the problems related to the use of tokens outside the browser and web enviroment. We must support also CLI and terminals so this is crucial for Astronomers.</span>

Q: should we build complex clients or use separate AA for CLI?

Q: what about using certificates? PKI is still very supported and secure.

<span style="background-color: transparent; color: #630000; font-size: 17.29px; font-weight: bold;">Conclusions</span>

We identify three different science platforms types:

   * Applications oriented: platform that provides access to tools and visualization
   * Project oriented: access to data and sw and pipelins and contaners for a specific Project.
   * user oriented: the user can also provide his own software and data.
We identify some common concepts:
   * Interoperability among science platforms aims at enabling direct re-use of data/service/code/workflow resources among standalone platforms
   * "ecosystem is better than monoculture" so we should build iteroperability from the ecosystem idea
   * Interoperability means: 
      * interoperable data resources
      * interoperable code resources
      * moving them around in a realistic and reasonable way
   * The use of FAIR principles

<span style="background-color: transparent;">Actions: </span>

   * <span style="background-color: transparent;">Common Resource access control?</span>
   * Run my platform on your k8s cluster?

<span style="background-color: transparent;">We identify research area for the future:</span>

   * Interopoerable Authentication and Authorisation
   * Data Access and proximity standards: how far are may data, shoudl I move data or software
   * Software regisitry: how can I find contaiiners
We need to indentify starndards procedures to allow users to move from one platform to another or use at the same time two platforms.

IVOA has a central role but we do not want to "reinvent the weel" so let see also what is the activity of large commercial data providers.

Continue the discussion on IVOA framework and also in ADASS in a couple of years

<!--
      * Set ALLOWTOPICRENAME = IVOA.TWikiAdminGroup
-->

Attachments

Topic attachments
I	Attachment	History	Action	Size	Date	Who
pptx	00-ADASS-SP-BoF.pptx	r1	manage	467.2 K	2019-10-10 - 09:27	GiulianoTaffoni
pptx	01-ADASS2019_SciencePlatform_BoF_CANFAR.pptx	r1	manage	9356.9 K	2019-10-10 - 09:28	GiulianoTaffoni
pptx	02-science_platforms_bof_-_gerard.pptx	r1	manage	6403.9 K	2019-10-10 - 09:29	GiulianoTaffoni
pptx	03-adass_bof.pptx	r1	manage	1896.4 K	2019-10-10 - 09:29	GiulianoTaffoni
pdf	04-ADASS-2019-BoF-SP-CDS.pdf	r1	manage	13715.9 K	2019-10-10 - 09:29	GiulianoTaffoni
pdf	06-BoF-VOCLOUD-Skoda.pdf	r1	manage	1259.5 K	2019-10-10 - 09:29	GiulianoTaffoni
pdf	07-2019-10-09_ESCAPE_ESAP.pdf	r1	manage	3276.8 K	2019-10-10 - 09:29	GiulianoTaffoni
pdf	08-OAuth2___OIDC_on_Science_Platforms_V2.pdf	r1	manage	650.2 K	2019-10-10 - 09:29	GiulianoTaffoni
pdf	09-MolinaroMorris_ADASSXXIX_BoF8.pdf	r1	manage	549.2 K	2019-10-10 - 09:29	GiulianoTaffoni
pptx	ADASS2019-BoF-Summary.pptx	r1	manage	1011.8 K	2019-10-10 - 09:41	GiulianoTaffoni

Topic revision: r2 - 2019-10-12 - GiulianoTaffoni

IVOA

Log in or Register

IVOA.net
Wiki Home
WebChanges
WebTopicList
WebStatistics

Twiki Meta & Help
IVOA
Know
Main
Sandbox
TWiki

TWiki intro
TWiki tutorial
User registration
Notify me

Working Groups

Interest Groups

Committees

Stds&Procs

www.ivoa.net
Documents
Events
Members
XML Schema