*IVOA May 2021 Interoperability Meeting - Apps/DAL session *Time: Wednesday May 27 05:00 UTC participants: ~55 *schedule Brent Miszalski - Data Central's Data Aggregation Service Mathieu Servillat - Provenance information management Volodymyr Savchenko - Workflow interoperability for telescope operations and time domain astronomy Valentin Lefranc - Astro Colibri *Notes *Brent Miszalski - Data Central's Data Aggregation Service FRB followup - see what data are near the candidate position, need it fast and frequently Existing aggregation portals didn't match requirements Leverage existing DAL services and combine so it just works Built around Aladin Lite with python, django services Components: Catalogues, images, aladin lite, name resolver Catalogues * Query TAP/SSA directly for catalogues * Catalogue table and aladin lite linked * Redshift aggregator uses vizier query using redshift UCDs * SAMP export enabled IMages * Uses SIA services * fits2hips * Produce mosaics on demand - new DC endpoints * Image table and Aladin lite linked Planning * Authentication for cusotmisation and data access * MOC usage to filter services * Extend AAO endpoints Consider IVOA services as building blocks Intermediary services to reduce complexity improve accessibility and widen audience What other services should AAO DC be building? Ada Nebot: Have you had a lok at photmetric viewer? WOuld this be useful to the service BM: Haven;t looked at photometry yet, but consider plotting table data Catherine Boisson: @ada photometric viewer is great / usfull. Would it be possible to add some Galactic absortion correction ? AN: Here is the link to the photometric viewer I just mentioned: http://vizier.unistra.fr/vizier/sed/ BM: We pull the Schlegel E(B-V) from IRSA directly, so in theory it could be applied to any photometric tables AN: you can download the data as a VOTable and then add your fav. absorption correction to it BM: Sounds good Ada. If you can think of some science use cases for SED explorer it would be interesting to explore this further MA: I wonder whether someone is developing a generic spectra viewer that could be used is a similar to way to Aladin Lite is being used for images. BM: It’s a great idea Mark! I believe my colleagues are working on that. Simon may give some more details Xiuqin Wu: IPAC Firefly team is developing a special viewer for spectra and SED. Vandana will give a presentation at 6:30 DAL/DM session after current session. Baptiste: the CASSIS spectral tool might be something to look at, also Simon O'Toole: @Mark yes, we are working on such a spectrum viewer as @Brent mentioned. It is structured to allow plugins for different analysis tools There is some work still to integrate a few different pieces together, but we’re well on the way Laurent Michel: CASSIS cannot be run within a Web page AFAK MA: @Laurent - perhaps a CASSIS Lite in analogy to Aladin and Aladin Lite ? LM: Great idea MA: Very interested in all the ideas mentioned!! Thomas BochL I’m aware of this spectrum viewer running in the browser: https://github.com/ANU-RSAA/MarzV2 SO'T:Our spectrum viewer is a generalised version of that one @Thomas :) TB: oh, I see, thanks @Simon MA - One aspect that is important for Aladin Lite is its incarnation as ipyaladin for use in notebooks. I wonder if the spectral/SED tool developers are also thinking along these lines? There are probably already good spectral tools available for use in notebooks - so is there something to leverage there? MM: What do you use for mosaicing BM: Montage py package *Mathieu Servillat - Provenance information management Context of FAIR - Reusable Important to record provenance in a structured format Allows rerunning of processing, and finding those affected by problematic versions Recording should be detailed and naive Provenance DM based on W3C Provenance is represented as a graph Can be serialised using W3C standards, e,g. XML, json Proposed YAML serialisation solution to be human readable (closer to FITS keywords) than either json or XML Choose granularity and level of detail (specific to each project) Last-step embedded provenance is what scientists think of and have an approach for that but it won;t be the complete graph -> if standardized with relevant information from ProvDM, could help to rebuild the full graph, step by step Advanced manager would capture, store and visualise the graph (e.g. using a database) CTA have a prototype of such a manager incorporating ProvDM, ProvTAP and ProvSAP Visualise with web based apps, or topcat , aladin etc voprov - based on W3C prov, adapated to IVOA ProvDM logprov - python based provTAP - tap schema to allow querying provenance to use to generate last step provenance - Can use a view to allow simpler presentation and querying *Volodymyr Savchenko - Workflow interoperability for telescope operations and time domain astronomy Integral data centre - multiple intruments a challenge Also important to cobine data from integral with other telescopes e.g. Fermi Now have more of a focus on providing services than just data Traditional Integral transient analysis - experts work on data for each use case but now too difficult Now standard - experts contribute tools for each data type, then pipeline produces products Combine variety of software from diffierent sources to build services Use cloud-native technologies with code to service workflows - and allow people to replciate the services Attach provenance when use other's services Hard to share workflows - harder than just software Describe all services in a workflow using RDF Create knowledge graph using IVOA vocabs , DC etc Can use graphs to run computational experiments as they contain the records of workflows Publishing federates services in collaboration with other agencies Marco Molinaro: Work ongoing in IVOA regards describing workflows. VS: Interested in contacts MM: Dave Morris gave a presentation early in the interop *Valentin Lefranc - Astro Colibri Web app for multi-messenger follow-up Aim to get quick overeview of sources and transient events Real-time access to data Arch: Built around API taking in notifications, store alerts in Mongo DB, front end - cross platform - iOS, android, web - google database and dev tool Users can filter data to see and be notified by - including mobile notifications Can produce cone search at custom radius around events Includes observatory access (eg. HESS) Includes visibility plots for observatories and filters for sources Can use wavelength filters to filter message types and observatories Some observatory specfic access (e.g. HESS) Cone search is interactive, adjustable size and show results by position Can then provide visibility plots for desired observatories Plans: * First release soon * Distribute via mobile stores so poeple ban get notifications on the go. * Link to ObsVisSAP and ObsLocTAP [BM} Valentin: Have you considered using Aladin Lite for the plot? VL: Haven't looked at it before, will do so BM: Cool! Great application! :-) Fabian Schussler: We link directly to Aladin for easy access. But we aimed at building an app and didn’t find a way to make an Aladin light based plot work on all systems… AN:Are you considering plotting images VL: Not currently, have to consider delivery via mobile so can't be too crowded. Perhaps only on web view BM: I guess creating finder charts (e.g. 10 arcmin FOV with target identified) for those observers following up candidates would be useful? (If not done already) Fabian Schussler: we show a list of all events/sources within the cone search region and information (coordinates, links to external pages, etc.) for all of them. BM: Cool. I suspect some of the external pages may already have the finder charts. In the case of quick followup it may be helpful to have it quickly at hand - depends on who is using it of course. FS: Indeed! Also, we need to work on dealing with information at small angular scales. For the moment we focus on VHE observations and the angular scales are at the deg level. Going to smaller scales (e.g. for radio/optical observations) we will need to improve the filtering, etc. or restrict the total size of the plot in order not to overload the dislays. BM: Aladin Lite's indexed approach with HiPS may help here with these different scales, but yes, only for the web side if you decide to go that route. Jesus Salgado: Really nice application! Waiting to interact with you for the ObjVisSAP and ObsLocTAP interaction. FS: Lets hope we'll be find the time/manpower to work on this soon. We'll be in touch.