Difference: DAL_DM_TDIG_ShanghaiSession2017 ( vs. 1)

Revision 12017-06-09 - AdaNebot

 
META TOPICPARENT name="IvoaVOEvent"

Summary from Shanghai DAL/DM/TDIG session:




NDCube Time domain application (M. Cresitello-Dittmar)

  • Models:
    • Coordinates: coo, frame, systems
    • Measurements: measure values + errors
    • DatasetMetadata: provides high level data description and supports access & discovery
    • Cube
  • Example Serialisation:
    • real-world datasets
    • example files
    • validated against schema
    • validation against model spec in progress
  • Time Series in Cube
    • Note delivered to DM (Feb 2017): high level of correlation with cube model, productive discussion
    • Concerns fit into 2 categories:
      • modeling (concepts not properly related)
      • annotation (not meeting user expectations)
    • Model changes:
      • added concept of dependent to DataAxis
      • strict relation of Dataset/Cube loosened
      • TimeSeries of spectra —> abstracting DataAxis tree
      • Requirements added: “model agnostic clients should be able to find basic element with minimal specialised knowledge”
      • Moved coords/meas from pattern model to realised base model

Time Series use cases at INAF (M. Molinaro)

  • Two science cases
    • GAPS Exoplanets: radial velocity series, plus other type of data
      • Dataset and content description
      • Simple use case:
        • datasets discovery: date, rv & delta_rv, CCF param, stellar activity indexes.
        • access/retrieve time series: including all instrumental details and host star characteristics.
        • link points to original / reduced spectra
    • TSRS Space Weather: polarimetric fluxes
      • Dataset metadata and content description
      • Simple use case:
        • Search by observation date/time
        • Retrieve/cut available data
  • Probably not issue at model / serialisation level.
  • Discovery and access to check.

Time Series and the VO (M. Alacid & E. Solano):

  • Data Access:
    • SSAP: SSA is capable of describing most tabular spectrophotometric data, including time series and spectral energy distributions (SEDs) as well as 1-D spectra.
    • According to the SSAP Protocol, the Dataset.Type value must be “Spectrum”
  • Data Model:
    • Spectral DM: Metadata information: position, time frame, flux, spectral frame, DataID and Curation, data serialisation as FITS. CoRoT light curves described with the Spectral Data Model can be managed with VO tools like SPLAT.
    • Time Series Cube Data Model: For our case (simple LCs) this new data model does not bring any improvement.
  • Data Discovery
    • Time Series cannot be discovered at Registry level.
    • Time Series could be discovered using ObsCore / TAP.
    • The discovery of SSAP Time Series services has to be a must. DataType = TimeSeries
    • TOPCAT: But Datatype is not included in the Match Fields options of TOPCAT.
    • Splat-VO does not have Time Series option
  • Conclusions: it is necessary to fix the problems related to
    • data discovery (registry)
    • data representation (data model).

SSIG inputs on Time Domain (B. Cecconi, M. Louys, S. Erard)

  • Solar System science cases involve photometry, spectra, images, polarisation.
  • Axes:
    • Time scales/formats: SCLK, SCET, UTC,…
    • In addition to temporal axes: rotation, period,
  • Data discovery or catalogue mining: EpnCore temporal queries
    • Use of temporal parameters for data selection / discovery
      • a specific epoch or interval
      • a specific sampling step condition
      • a specific exposure time (int. time) condition
      • select a time scale and/or time origin
  • EpnCore / ObsCore comparison of keyword, description, Unit, UCD, utype
    • Temporal time interval
    • Temporal sampling step
    • Observation exposure time
    • Observation time resolution
  • Time data products with time information:
    • Time-series (tables)
    • Spectrograms
    • Cubes (images/spectra)
    • Events
  • Data product content:
    • Often sparse multiD data with time as main axis
    • Publishing time-series as: VOTable, FITS, CDF, …
    • Mandatory File + search metadata:
      • Time scale (+starting point/epoch), time origin (location)
      • Time coverage + sampling/resolution/exposure
  • Time-series:
    • Plotting tools: AMDA, TOPCAT (Time plot option), Autoplot (with SAMP)
    • Analysis: wide variety of methods (FFT, periodograms, lomb scargle,…). Analyse with IDL, python?
    • Data service? NASA/Heliophysics is working on an API for time series (HAPI).
    • Complexity?
      • Astronomy & Solar System: observational parameters can change over time
      • Use same time axis / time scale references

Simple (?) Time Series in VizieR (S. Derriere)

  • Content of VizieR: > 10% contain "time series” flag
  • SED-viewer like tool for time series is difficult:
    • Heterogeneous formats
    • Missing characterisation / metadata or only in human-readable form
  • Catalogues:
    • Big missions: HIPPARCOS & Tycho light curves, Kepler, CoRoT, OGLE,MACHO, EROS
    • Variability surveys
    • Tables dedicated to individual object
    • Solar data
  • Kind of time series:
    • 70% are light curves
    • 23% are radial velocities
  • Several examples reflecting how heterogeneous data in VizieR is
    • Example 1:
      • One table for one target, JD versus RV (no coordinates)
      • Easily exported to VOTable, SAMP,…
    • Example 2: CoRoT
      • One catalogue row per target
      • Thumbnails
      • Link to FITS file
    • Example 3:
      • 8 targets in same table (no coordinates)
      • JD-offser & Phase (T_0 + Period)
    • Example 4:
      • Coordinates and target
      • Link to plot + ascii file
      • Missing metadata
  • Parameters: Param = f(time)
    • Time (JD, MJD, HJD, JD-xxxxx, phase)
    • Param = Y-axis (flux, magnitude, differential magnitude, color, counts, radial velocity…)
  • Extracting time series data:
    • cone search not sufficient (some cats. only have target name…)
    • add standardised parameters (TIME) to output, but nee to keep all original params & description (provenance)
  • Data Provider:
    • Some metadata and mapping will have to be added to VizieR
    • Only deal with large missions? but dedicated surveys to one source are very important. 90/10 rule (use popularity).
  • A time series standard should:
    • not mandate too many metadata (or it won’t be characterised properly)
    • allow for dataset-specific parameters: flags, S/N,…

Asterics hackathon report (F. Bonnarel)


  • Metadata needed to describe and discover TimeSeries:
    • Spatial coordinate system
    • Time coordinate system: scale, ref. position, representation
      • Time, spectral, space and polarisation characterisation and statistics (raw or mean positions, raw bounding limits, std deviation)
    • Time sampling characterisation and statistics:
      • Mean sampling step
      • Sampling step limits
      • Sampling step standard deviation
      • Total exposure time
    • Exposure time characterisation and statistics:
      • Mean total exposure time
      • Mean exposure time per step
      • Min, max and std deviation of exposure time per step
    • Characterisation on the time frequency axis:
      • Periodograms, period (s), Fourier coefficients and frequencies, phase,…
    • What are the dependant and indépendant quantities and what is their nature
    • Which mode are the data? Transient or periodic? Seen by periodogram or by Target class
    • Target name, class, subclass (SN, SB,…)
  • Type of data we call TimeSeries:
    • Temporal sequence of measurement points containing
      • A time coordinate + flux (or similar) / RV / position / spectra / images
  • Representation of these data: time + spectra / images are better represented as a regular cube with only one sparse axis or as an event list
  • Recommend a time representation for standard output? MJD?
  • Relative time for theoretical data?
  • DAL chair view for a TS discovery & access:
    • Extend ObsCore with a new TimeSeriesCore table
    • ObsTAP-TS can query both tables together
    • Extend “SIAV2” query interface to new TimeSeries specific query parameters
    • Archived TimeSeries retrieval or DataLink
    • Virtual data discovery (=TimeSeries produced on the fly) in SIAV2 = DsDisc. Access.reference is a SODA url
    • SODA expansions to TS
      • Cutout or time selection
      • Selection on time frequencies
      • Selection in exposure times
      • Time binning
      • Frequency or phase output

TimeSeries Cube DM (J. Nadvornik & P. Skoda)


  • Introduction on OLAP cubes
  • Definition of Sparse Cube DM:
    • Can describe any time series axes
    • Is flexible
    • Is extensible
  • Time Series Cube DM: Separating Data & Information:
    • Describing meaning (Information layer)
    • Cube DM describes meaning on axes (data layer) without knowing all physical domain model
    • Changes to physical domain models (STC, Phot DM, provenance) wont require Cube DM to change.
  • Time Series Cube UML:
  • Science use Cases (from E. Solano)
  • Discovery & Access
    • ObsCore
    • SSA
    • SPLAT-VO / TOPCAT / Aladin
      • Light-curve
      • Datalink
      • Cutouts
  • Open questions:
    • Add datalink to ObsCore
    • What to put into Quantity DM
    • Two kinds of models (real world model & application of data model for publishing the data)
    • What do I need to discover about the data cube
    • Datalink for cutouts of cubes seems like best option
    • Use cases…

Time Series, VO-DML, DaCHS (M. Demleitner)


  • DaCHS Annotation:
    • DaCHS is a general VO publishing framework.
    • Each resource is described using a resource descriptor (RD) with (among others) table metadata
    • Before VO-DML, DaCHS understood one DM: STC. Annotation used a variant of STC-S.
    • Plan is to move into the VO-DML age:
      • Independent task-specific annotations
      • Ad-hoc annotation language specific to DaCHS rather than XML on input
      • Not backed up by VO-DM…
  • Proposed NDCube Processing:
    • Client parses STC annotations
    • Client looks for ndcube:Cube-typed annotation.
    • Client inspects existing STC annotations on hjd
    • Client pulls the set of dependent axes from ndcube: Cube annotations. Let the user which to plot?
    • Plotting component looks or ivoa:Measurement-typed annotation of df to work out what to use as error in the plot.

Status and plans for the TS and spectra distribution for Gaia DR2 & DR3 (J. Gonzalez)

  • Gaia DR2 content:
    • Main catalogue
    • X-matches with “main” catalogues
    • SSOs
    • Epoch photometry
  • Gaia DR3 content: also adds spectra
  • SSOs integration
    • TAP service adaptation for SSO handling:
      • UDFs def. for SSOs
      • Based on IMCCE Eproc integration performed for ESASky SSO search service
      • Possible ADQL extension with datatypes and functions to EPN-TAP?
    • SSO Data Model:
      • EPNCore V2 draft: Orbital params —> ephemerides table —> TAP interface
  • Time Series & Spectra:
    • DAL side:
      • Extension of DataLink through Custom Access Data Service
        • No protocol extension needed
        • Archive data distribution has mission specific features
        • DataLink recognition
      • On the fly data serialisation to requested formats
      • SSAP, ObsCore TS vs SVO TS-SSAP ?
    • DM side:
      • OK for spectra, pending Time Series
      • DR ~ April 2018
      • Need to base in and extend current WIP (Jiri’s TS draft)
    • Applications:
      • Spectra (VOSpec, SPLAT-VO)
      • Time Series
 
This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback