|
META TOPICPARENT |
name="IvoaVOEvent" |
Summary from Shanghai DAL/DM/TDIG session:
- Models:
- Coordinates: coo, frame, systems
- Measurements: measure values + errors
- DatasetMetadata: provides high level data description and supports access & discovery
- Cube
- Example Serialisation:
- real-world datasets
- example files
- validated against schema
- validation against model spec in progress
- Time Series in Cube
- Note delivered to DM (Feb 2017): high level of correlation with cube model, productive discussion
- Concerns fit into 2 categories:
- modeling (concepts not properly related)
- annotation (not meeting user expectations)
- Model changes:
- added concept of dependent to DataAxis
- strict relation of Dataset/Cube loosened
- TimeSeries of spectra —> abstracting DataAxis tree
- Requirements added: “model agnostic clients should be able to find basic element with minimal specialised knowledge”
- Moved coords/meas from pattern model to realised base model
- Two science cases
- GAPS Exoplanets: radial velocity series, plus other type of data
- Dataset and content description
- Simple use case:
- datasets discovery: date, rv & delta_rv, CCF param, stellar activity indexes.
- access/retrieve time series: including all instrumental details and host star characteristics.
- link points to original / reduced spectra
- TSRS Space Weather: polarimetric fluxes
- Dataset metadata and content description
- Simple use case:
- Search by observation date/time
- Retrieve/cut available data
- Probably not issue at model / serialisation level.
- Discovery and access to check.
- Data Access:
- SSAP: SSA is capable of describing most tabular spectrophotometric data, including time series and spectral energy distributions (SEDs) as well as 1-D spectra.
- According to the SSAP Protocol, the Dataset.Type value must be “Spectrum”
- Data Model:
- Spectral DM: Metadata information: position, time frame, flux, spectral frame, DataID and Curation, data serialisation as FITS. CoRoT light curves described with the Spectral Data Model can be managed with VO tools like SPLAT.
- Time Series Cube Data Model: For our case (simple LCs) this new data model does not bring any improvement.
- Data Discovery
- Time Series cannot be discovered at Registry level.
- Time Series could be discovered using ObsCore / TAP.
- The discovery of SSAP Time Series services has to be a must. DataType = TimeSeries
- TOPCAT: But Datatype is not included in the Match Fields options of TOPCAT.
- Splat-VO does not have Time Series option
- Conclusions: it is necessary to fix the problems related to
- data discovery (registry)
- data representation (data model).
- Solar System science cases involve photometry, spectra, images, polarisation.
- Axes:
- Time scales/formats: SCLK, SCET, UTC,…
- In addition to temporal axes: rotation, period,
- Data discovery or catalogue mining: EpnCore temporal queries
- Use of temporal parameters for data selection / discovery
- a specific epoch or interval
- a specific sampling step condition
- a specific exposure time (int. time) condition
- select a time scale and/or time origin
- …
- EpnCore / ObsCore comparison of keyword, description, Unit, UCD, utype
- Temporal time interval
- Temporal sampling step
- Observation exposure time
- Observation time resolution
- Time data products with time information:
- Time-series (tables)
- Spectrograms
- Cubes (images/spectra)
- Events
- Data product content:
- Often sparse multiD data with time as main axis
- Publishing time-series as: VOTable, FITS, CDF, …
- Mandatory File + search metadata:
- Time scale (+starting point/epoch), time origin (location)
- Time coverage + sampling/resolution/exposure
- Time-series:
- Plotting tools: AMDA, TOPCAT (Time plot option), Autoplot (with SAMP)
- Analysis: wide variety of methods (FFT, periodograms, lomb scargle,…). Analyse with IDL, python?
- Data service? NASA/Heliophysics is working on an API for time series (HAPI).
- Complexity?
- Astronomy & Solar System: observational parameters can change over time
- Use same time axis / time scale references
- Content of VizieR: > 10% contain "time series” flag
- SED-viewer like tool for time series is difficult:
- Heterogeneous formats
- Missing characterisation / metadata or only in human-readable form
- Catalogues:
- Big missions: HIPPARCOS & Tycho light curves, Kepler, CoRoT, OGLE,MACHO, EROS
- Variability surveys
- Tables dedicated to individual object
- Solar data
- Kind of time series:
- 70% are light curves
- 23% are radial velocities
- …
- Several examples reflecting how heterogeneous data in VizieR is
- Example 1:
- One table for one target, JD versus RV (no coordinates)
- Easily exported to VOTable, SAMP,…
- Example 2: CoRoT
- One catalogue row per target
- Thumbnails
- Link to FITS file
- Example 3:
- 8 targets in same table (no coordinates)
- JD-offser & Phase (T_0 + Period)
- Example 4:
- Coordinates and target
- Link to plot + ascii file
- Missing metadata
- Parameters: Param = f(time)
- Time (JD, MJD, HJD, JD-xxxxx, phase)
- Param = Y-axis (flux, magnitude, differential magnitude, color, counts, radial velocity…)
- Extracting time series data:
- cone search not sufficient (some cats. only have target name…)
- add standardised parameters (TIME) to output, but nee to keep all original params & description (provenance)
- Data Provider:
- Some metadata and mapping will have to be added to VizieR
- Only deal with large missions? but dedicated surveys to one source are very important. 90/10 rule (use popularity).
- A time series standard should:
- not mandate too many metadata (or it won’t be characterised properly)
- allow for dataset-specific parameters: flags, S/N,…
- Metadata needed to describe and discover TimeSeries:
- Spatial coordinate system
- Time coordinate system: scale, ref. position, representation
- Time, spectral, space and polarisation characterisation and statistics (raw or mean positions, raw bounding limits, std deviation)
- Time sampling characterisation and statistics:
- Mean sampling step
- Sampling step limits
- Sampling step standard deviation
- Total exposure time
- Exposure time characterisation and statistics:
- Mean total exposure time
- Mean exposure time per step
- Min, max and std deviation of exposure time per step
- Characterisation on the time frequency axis:
- Periodograms, period (s), Fourier coefficients and frequencies, phase,…
- What are the dependant and indépendant quantities and what is their nature
- Which mode are the data? Transient or periodic? Seen by periodogram or by Target class
- Target name, class, subclass (SN, SB,…)
- Type of data we call TimeSeries:
- Temporal sequence of measurement points containing
- A time coordinate + flux (or similar) / RV / position / spectra / images
- Representation of these data: time + spectra / images are better represented as a regular cube with only one sparse axis or as an event list
- Recommend a time representation for standard output? MJD?
- Relative time for theoretical data?
- DAL chair view for a TS discovery & access:
- Extend ObsCore with a new TimeSeriesCore table
- ObsTAP-TS can query both tables together
- Extend “SIAV2” query interface to new TimeSeries specific query parameters
- Archived TimeSeries retrieval or DataLink
- Virtual data discovery (=TimeSeries produced on the fly) in SIAV2 = DsDisc. Access.reference is a SODA url
- SODA expansions to TS
- Cutout or time selection
- Selection on time frequencies
- Selection in exposure times
- Time binning
- Frequency or phase output
- Introduction on OLAP cubes
- Definition of Sparse Cube DM:
- Can describe any time series axes
- Is flexible
- Is extensible
- Time Series Cube DM: Separating Data & Information:
- Describing meaning (Information layer)
- Cube DM describes meaning on axes (data layer) without knowing all physical domain model
- Changes to physical domain models (STC, Phot DM, provenance) wont require Cube DM to change.
- Time Series Cube UML:
- Cube DM:
- ColumnRef
- Quantity (holds stats for ColumnRef)
- Axis Domain DMs (holds context for ColumnRef)
- STC DM
- Photometry DM
- Characterisation DM
- Gravitational Wave DM
- Probability Distribution DM
- Science use Cases (from E. Solano)
- Discovery & Access
- ObsCore
- SSA
- SPLAT-VO / TOPCAT / Aladin
- Light-curve
- Datalink
- Cutouts
- Open questions:
- Add datalink to ObsCore
- What to put into Quantity DM
- Two kinds of models (real world model & application of data model for publishing the data)
- What do I need to discover about the data cube
- Datalink for cutouts of cubes seems like best option
- Use cases…
- DaCHS Annotation:
- DaCHS is a general VO publishing framework.
- Each resource is described using a resource descriptor (RD) with (among others) table metadata
- Before VO-DML, DaCHS understood one DM: STC. Annotation used a variant of STC-S.
- Plan is to move into the VO-DML age:
- Independent task-specific annotations
- Ad-hoc annotation language specific to DaCHS rather than XML on input
- Not backed up by VO-DM…
- Proposed NDCube Processing:
- Client parses STC annotations
- Client looks for ndcube:Cube-typed annotation.
- Client inspects existing STC annotations on hjd
- Client pulls the set of dependent axes from ndcube: Cube annotations. Let the user which to plot?
- Plotting component looks or ivoa:Measurement-typed annotation of df to work out what to use as error in the plot.
- Gaia DR2 content:
- Main catalogue
- X-matches with “main” catalogues
- SSOs
- Epoch photometry
- Gaia DR3 content: also adds spectra
- SSOs integration
- TAP service adaptation for SSO handling:
- UDFs def. for SSOs
- Based on IMCCE Eproc integration performed for ESASky SSO search service
- Possible ADQL extension with datatypes and functions to EPN-TAP?
- SSO Data Model:
- EPNCore V2 draft: Orbital params —> ephemerides table —> TAP interface
- Time Series & Spectra:
- DAL side:
- Extension of DataLink through Custom Access Data Service
- No protocol extension needed
- Archive data distribution has mission specific features
- DataLink recognition
- On the fly data serialisation to requested formats
- SSAP, ObsCore TS vs SVO TS-SSAP ?
- DM side:
- OK for spectra, pending Time Series
- DR ~ April 2018
- Need to base in and extend current WIP (Jiri’s TS draft)
- Applications:
- Spectra (VOSpec, SPLAT-VO)
- Time Series
|