Summary from Shanghai DAL/DM/TDIG session:

NDCube Time domain application (M. Cresitello-Dittmar)

• Models:
  • Coordinates: coo, frame, systems
  • Measurements: measure values + errors
  • DatasetMetadata: provides high level data description and supports access & discovery
  • Cube
• Example Serialisation:
  • real-world datasets
  • example files
  • validated against schema
  • validation against model spec in progress
• Time Series in Cube
  • Note delivered to DM (Feb 2017): high level of correlation with cube model, productive discussion
  • Concerns fit into 2 categories:
    • modeling (concepts not properly related)
    • annotation (not meeting user expectations)
  • Model changes:
    • added concept of dependent to DataAxis
    • strict relation of Dataset/Cube loosened
    • TimeSeries of spectra —> abstracting DataAxis tree
    • Requirements added: “model agnostic clients should be able to find basic element with minimal specialised knowledge”
    • Moved coords/meas from pattern model to realised base model

Time Series use cases at INAF (M. Molinaro)

• Two science cases
  • GAPS Exoplanets: radial velocity series, plus other type of data
    • Dataset and content description
    • Simple use case:
      • datasets discovery: date, rv & delta_rv, CCF param, stellar activity indexes.
      • access/retrieve time series: including all instrumental details and host star characteristics.
      • link points to original / reduced spectra
  • TSRS Space Weather: polarimetric fluxes
    • Dataset metadata and content description
    • Simple use case:
      • Search by observation date/time
      • Retrieve/cut available data
• Probably not issue at model / serialisation level.
• Discovery and access to check.

Time Series and the VO (M. Alacid & E. Solano):

• Data Access:
  • SSAP: SSA is capable of describing most tabular spectrophotometric data, including time series and spectral energy distributions (SEDs) as well as 1-D spectra.
  • According to the SSAP Protocol, the Dataset.Type value must be “Spectrum”
• Data Model:
  • Spectral DM: Metadata information: position, time frame, flux, spectral frame, DataID and Curation, data serialisation as FITS. CoRoT light curves described with the Spectral Data Model can be managed with VO tools like SPLAT.
  • Time Series Cube Data Model: For our case (simple LCs) this new data model does not bring any improvement.
• Data Discovery
  • Time Series cannot be discovered at Registry level.
  • Time Series could be discovered using ObsCore / TAP.
  • The discovery of SSAP Time Series services has to be a must. DataType = TimeSeries
  • TOPCAT: But Datatype is not included in the Match Fields options of TOPCAT.
  • Splat-VO does not have Time Series option
• Conclusions: it is necessary to fix the problems related to
  • data discovery (registry)
  • data representation (data model).

SSIG inputs on Time Domain (B. Cecconi, M. Louys, S. Erard)

• Solar System science cases involve photometry, spectra, images, polarisation.
• Axes:
  • Time scales/formats: SCLK, SCET, UTC,…
  • In addition to temporal axes: rotation, period,
• Data discovery or catalogue mining: EpnCore temporal queries
  • Use of temporal parameters for data selection / discovery
    • a specific epoch or interval
    • a specific sampling step condition
    • a specific exposure time (int. time) condition
    • select a time scale and/or time origin
    • …
• EpnCore / ObsCore comparison of keyword, description, Unit, UCD, utype
  • Temporal time interval
  • Temporal sampling step
  • Observation exposure time
  • Observation time resolution
• Time data products with time information:
  • Time-series (tables)
  • Spectrograms
  • Cubes (images/spectra)
  • Events
• Data product content:
  • Often sparse multiD data with time as main axis
  • Publishing time-series as: VOTable, FITS, CDF, …
  • Mandatory File + search metadata:
    • Time scale (+starting point/epoch), time origin (location)
    • Time coverage + sampling/resolution/exposure
• Time-series:
  • Plotting tools: AMDA, TOPCAT (Time plot option), Autoplot (with SAMP)
  • Analysis: wide variety of methods (FFT, periodograms, lomb scargle,…). Analyse with IDL, python?
  • Data service? NASA/Heliophysics is working on an API for time series (HAPI).
  • Complexity?
    • Astronomy & Solar System: observational parameters can change over time
    • Use same time axis / time scale references

Simple (?) Time Series in VizieR (S. Derriere)

• Content of VizieR: > 10% contain "time series” flag
• SED-viewer like tool for time series is difficult:
  • Heterogeneous formats
  • Missing characterisation / metadata or only in human-readable form
• Catalogues:
  • Big missions: HIPPARCOS & Tycho light curves, Kepler, CoRoT, OGLE,MACHO, EROS
  • Variability surveys
  • Tables dedicated to individual object
  • Solar data
• Kind of time series:
  • 70% are light curves
  • 23% are radial velocities
  • …
• Several examples reflecting how heterogeneous data in VizieR is
  • Example 1:
    • One table for one target, JD versus RV (no coordinates)
    • Easily exported to VOTable, SAMP,…
  • Example 2: CoRoT
    • One catalogue row per target
    • Thumbnails
    • Link to FITS file
  • Example 3:
    • 8 targets in same table (no coordinates)
    • JD-offser & Phase (T_0 + Period)
  • Example 4:
    • Coordinates and target
    • Link to plot + ascii file
    • Missing metadata
• Parameters: Param = f(time)
  • Time (JD, MJD, HJD, JD-xxxxx, phase)
  • Param = Y-axis (flux, magnitude, differential magnitude, color, counts, radial velocity…)
• Extracting time series data:
  • cone search not sufficient (some cats. only have target name…)
  • add standardised parameters (TIME) to output, but nee to keep all original params & description (provenance)
• Data Provider:
  • Some metadata and mapping will have to be added to VizieR
  • Only deal with large missions? but dedicated surveys to one source are very important. 90/10 rule (use popularity).
• A time series standard should:
  • not mandate too many metadata (or it won’t be characterised properly)
  • allow for dataset-specific parameters: flags, S/N,…

Asterics hackathon report (F. Bonnarel)

• Metadata needed to describe and discover TimeSeries:
  • Spatial coordinate system
  • Time coordinate system: scale, ref. position, representation
    • Time, spectral, space and polarisation characterisation and statistics (raw or mean positions, raw bounding limits, std deviation)
  • Time sampling characterisation and statistics:
    • Mean sampling step
    • Sampling step limits
    • Sampling step standard deviation
    • Total exposure time
  • Exposure time characterisation and statistics:
    • Mean total exposure time
    • Mean exposure time per step
    • Min, max and std deviation of exposure time per step
  • Characterisation on the time frequency axis:
    • Periodograms, period (s), Fourier coefficients and frequencies, phase,…
  • What are the dependant and indépendant quantities and what is their nature
  • Which mode are the data? Transient or periodic? Seen by periodogram or by Target class
  • Target name, class, subclass (SN, SB,…)
• Type of data we call TimeSeries:
  • Temporal sequence of measurement points containing
    • A time coordinate + flux (or similar) / RV / position / spectra / images
• Representation of these data: time + spectra / images are better represented as a regular cube with only one sparse axis or as an event list
• Recommend a time representation for standard output? MJD?
• Relative time for theoretical data?
• DAL chair view for a TS discovery & access:
  • Extend ObsCore with a new TimeSeriesCore table
  • ObsTAP-TS can query both tables together
  • Extend “SIAV2” query interface to new TimeSeries specific query parameters
  • Archived TimeSeries retrieval or DataLink
  • Virtual data discovery (=TimeSeries produced on the fly) in SIAV2 = DsDisc. Access.reference is a SODA url
  • SODA expansions to TS
    • Cutout or time selection
    • Selection on time frequencies
    • Selection in exposure times
    • Time binning
    • Frequency or phase output

TimeSeries Cube DM (J. Nadvornik & P. Skoda)

• Introduction on OLAP cubes
• Definition of Sparse Cube DM:
  • Can describe any time series axes
  • Is flexible
  • Is extensible
• Time Series Cube DM: Separating Data & Information:
  • Describing meaning (Information layer)
  • Cube DM describes meaning on axes (data layer) without knowing all physical domain model
  • Changes to physical domain models (STC, Phot DM, provenance) wont require Cube DM to change.
• Time Series Cube UML:
  • Cube DM:
    • SparseCube
    • TimeSeriesCube — data
  • ColumnRef
    • Quantity (holds stats for ColumnRef)
    • Axis Domain DMs (holds context for ColumnRef)
      • STC DM
      • Photometry DM
      • Characterisation DM
      • Gravitational Wave DM
      • Probability Distribution DM
• Science use Cases (from E. Solano)
• Discovery & Access
  • ObsCore
  • SSA
  • SPLAT-VO / TOPCAT / Aladin
    • Light-curve
    • Datalink
    • Cutouts
• Open questions:
  • Add datalink to ObsCore
  • What to put into Quantity DM
  • Two kinds of models (real world model & application of data model for publishing the data)
  • What do I need to discover about the data cube
  • Datalink for cutouts of cubes seems like best option
  • Use cases…

Time Series, VO-DML, DaCHS (M. Demleitner)

• DaCHS Annotation:
  • DaCHS is a general VO publishing framework.
  • Each resource is described using a resource descriptor (RD) with (among others) table metadata
  • Before VO-DML, DaCHS understood one DM: STC. Annotation used a variant of STC-S.
  • Plan is to move into the VO-DML age:
    • Independent task-specific annotations
    • Ad-hoc annotation language specific to DaCHS rather than XML on input
    • Not backed up by VO-DM…
• Proposed NDCube Processing:
  • Client parses STC annotations
  • Client looks for ndcube:Cube-typed annotation.
  • Client inspects existing STC annotations on hjd
  • Client pulls the set of dependent axes from ndcube: Cube annotations. Let the user which to plot?
  • Plotting component looks or ivoa:Measurement-typed annotation of df to work out what to use as error in the plot.

Status and plans for the TS and spectra distribution for Gaia DR2 & DR3 (J. Gonzalez)

• Gaia DR2 content:
  • Main catalogue
  • X-matches with “main” catalogues
  • SSOs
  • Epoch photometry
• Gaia DR3 content: also adds spectra
• SSOs integration
  • TAP service adaptation for SSO handling:
    • UDFs def. for SSOs
    • Based on IMCCE Eproc integration performed for ESASky SSO search service
    • Possible ADQL extension with datatypes and functions to EPN-TAP?
  • SSO Data Model:
    • EPNCore V2 draft: Orbital params —> ephemerides table —> TAP interface
• Time Series & Spectra:
  • DAL side:
    • Extension of DataLink through Custom Access Data Service
      • No protocol extension needed
      • Archive data distribution has mission specific features
      • DataLink recognition
    • On the fly data serialisation to requested formats
    • SSAP, ObsCore TS vs SVO TS-SSAP ?
  • DM side:
    • OK for spectra, pending Time Series
    • DR ~ April 2018
    • Need to base in and extend current WIP (Jiri’s TS draft)
  • Applications:
    • Spectra (VOSpec, SPLAT-VO)
    • Time Series