% draft for Obs core components data model
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%  For an conversion via cgiprint (HTX):
%  See http://vizier.u-strasbg.fr/local/man/cgiprint.htx
\def\ifhtx{\iffalse}    % Lines used only for the HTML version
\ifhtx
% . . .
% . . . Definitions in HTX context
% . . .
\else
\documentclass[12pt,notitlepage,onecolumn]{ivoa}
% . . .
% . . . Definitions in LaTeX context
% . . .
\fi
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%% Comment/uncomment lines below to follow your LateX distribution...

%%
%% If document is processed with latex, dvips and ps2pdf
%%
\ifx\pdftexversion\undefined
  \usepackage[dvips]{graphicx}
  \DeclareGraphicsExtensions{.eps,.ps}
%% Uncomment following line if you want PDF thumbnails
%  \usepackage[ps2pdf]{thumbpdf}
% for old hyperref, use:
  \usepackage[ps2pdf]{hyperref}
%% for recent hyperref, use:
%  \usepackage[ps2pdf,bookmarks=true,bookmarksnumbered=true,hypertexnames=false,breaklinks=true,%
%  colorlinks,linkcolor=blue,urlcolor=blue]{hyperref}



%%
%% else if document is processed with pdflatex
%%
\else
  \usepackage[pdftex]{graphicx} %% graphics for pdftex (supports .pdf .jpg .png)
  \usepackage{epstopdf}         %% requires epstopdf
%% this is to support .ps files :
  \makeatletter
  \g@addto@macro\Gin@extensions{,.ps}
  \@namedef{Gin@rule@.ps}#1{{pdf}{.pdf}{`ps2pdf #1}}
  \makeatother
%% comment above lines if you have included ps files
%\DeclareGraphicsExtensions{.pdf,.jpg,.png}
%% Uncomment following line if you want PDF thumbnails
%  \usepackage[pdftex]{thumbpdf}
%% for old hyperref, use:
%  \usepackage[ps2pdf]{hyperref}
% for recent hyperref, use:
 \usepackage[pdftex,bookmarks=true,bookmarksnumbered=true,hypertexnames=false,breaklinks=true,%
  colorlinks,linkcolor=blue,urlcolor=blue]{hyperref}
  \pdfadjustspacing=1
\fi
\usepackage[final]{pdfpages}
\usepackage{tabulary}
 %%
%%  Header of the document...
%%
% Provide a title for your document
\title{Observation Data Model Core Components}
%\title{Observation Data Model: \\ \vspace{5px}~ \\ {\Large Core Components and the ObsTAP service}\\}
% Give date and version number
\date{\today}

% Choose one document type from below
%\ivoatype{IVOA Note}
\ivoatype{IVOA Working Draft}
%\ivoatype{IVOA Proposed Recommendation}
%\ivoatype{IVOA Recommendation}

\version{1.0}
% Give author list: separate different authors with \\
% You can add email addresses with links \url{mailto:yourname@ivoa.net}
\author{Mireille Louys, FranÃ§\c{c}ois Bonnarel, Alberto Micol,\\
        David Schade, Anita Richards, Patrick Dowler,\\
        Laurent Michel, Jesus Salgado, Doug Tody,\\
        Igor Chilingarian, Daniel Durand, Bruno Rino}
\editor{Mireille Louys, FranÃ§\c{c}ois Bonnarel, Patrick Dowler, Alberto Micol\\}
\urlthisversion{\url{http://www.ivoa.net/Documents/Notes/WD-ObsCoreDM-1.0-20101207.pdf}}
\urllastversion{\url{http://www.ivoa.net/Documents/Notes/WD-ObsCoreDM-1.0-20100701.pdf}}
\previousversion{1.0-20100701}
%%%%%%%%%%%%%%%%
%mir \documentclass[12pt]{article}
%\usepackage{graphicx}
%\usepackage{hyperref}
%\usepackage{psfig}
%\usepackage{html}
%\usepackage{epsf}
\usepackage{lscape}
%mir \textheight 9.0in \hoffset -0.5in \voffset -0.5in
%\newcommand{\Sensitiv}{Variation}
\newcommand{\Sensitiv}{Sensitivity}

%Mir colors definitions
\definecolor{ipink}{rgb}{1.0,0.937,0.957}
\definecolor{dblue}{rgb}{0.60,0.60,1.0}
\definecolor{ddblue}{rgb}{0.20,0.20,1.0}
\definecolor{iblue}{rgb}{0.9,0.9,1.0}

\newcommand{\bleu}[1]{\textcolor[rgb]{0.00,0.00,1.00}{#1}}
\newcommand{\blue}{\textcolor{blue}}
\newcommand{\violet}{\textcolor[rgb]{0.50,0.00,0.50}}
\newcommand{\green}{\textcolor[rgb]{0.00,1.00,0.00}}
\newcommand{\darkgreen}{\textcolor[rgb]{0.0,0.4,0.0}}
%%%%%%%%%%%%%%%%%
\newcommand{\m}[1]{\mbox{#1}}
%\newcommand{\change}[1]{{\color{red}\it #1}}
\newcommand{\change}[1]{{ #1}}
\newsavebox{\fmbox}
\newenvironment{fmpage}
     {\begin{lrbox}{\fmbox}\begin{footnotesize}\begin{minipage}{15cm}} 
     {\end{minipage}\end{footnotesize}\end{lrbox} \colorbox{iblue}{\fbox{\usebox{\fmbox}}}}
     
\newenvironment{fmpagesmall}
     {\begin{lrbox}{\fmbox} \begin{minipage}{5cm}} 
     {\end{minipage}\end{lrbox}\colorbox{ipink}{\fbox{ \usebox{\fmbox}}}}

\newenvironment{fmppage}
     {\begin{lrbox}{\fmbox}\begin{minipage}{\textwidth}}
     {\end{minipage}\end{lrbox}\colorbox{ipink}{\fbox{\usebox{\fmbox}}}}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{document}

\maketitle % print header in standard form

\section*{Abstract}
This document discusses the definition of the \textbf{core components} of the Observation data model that are necessary to perform data discovery when querying data centers for observations of interest.
It exposes the use-cases to be carried out, explains the model and provides  guide lines for its implementation as a TAP service based on the Table Access Protocol (TAP). Such a simple model is easy to understand and to implement by data providers that wish to publish their data into the Virtual Observatory. This effort which integrates data modeling and data access aspects in a single service is named Obs/TAP and will be referenced as such in the IVOA registries.
This is not covering the full Observation data model that will be developed in another IVOA document, and will include  a description of the Provenance of an Observation.
 
\section*{Status of this document}
This document has been produced by the Data Model Working Group in coordination with partners from DAL and VOQL WG. On one hand it describes the metadata attached to an astronomical observation and develops an abstract view of the Observation data model Core Components part. On the other hand it contains a guide for implementing in the Table Access Protocol framework and covers the data access layer aspects of this effort.
Due to the DM and DAL aspects of this document, this will circulate and be reviewed by both Working Groups.

The Observation Core Data Model (ObsCoreDM) is meant to define the core
components of all queryable metadata that play a role in the discovery of
observations. It is meant to be implemented using the Utypes derived from this
model in a TAP/ADQL implementation at various archives sites. It focuses on
reasonable set of common readily-available metadata descriptors and thus
allows for reasonable cost implementation for data providers. This has often
been referred as the ObsTAP project.
As with most of the VO Data Models, ObsCoreDM makes use of STC, Utypes,
Units and UCDs. ObsCoreDM can be serialized with a VOTable.
ObsCoreDM can make reference to more complete DM, such as ObsProvDM,
CharDM, Spectrum DM or SSLDM. 

\begin{figure}[!ht]
\begin{flushleft}  
\includegraphics[width=\textwidth]{./images/VOArchitecture-ObsTAP.pdf}
\caption{\footnotesize How the Observation data model Core Components fits into the overall IVOA architecture.
Highlighted blocks in red are data models or specifications that are used by within this model.}
\label{fig:obsdmarchi}
\end{flushleft}
\end{figure}

\section*{Acknowledgements}
This work has been partly funded by Euro-VO AIDA project that we acknowledge here. SSC XMM Catalog service supported the implementation of the SAADA version of Obs/TAP at Strasbourg Observatory. 
The Canfar (tb completed , corrected ???) project also contributed for the implementation of Obs/TAP at CADC, Victoria.
\clearpage

\tableofcontents

\clearpage

% ********************************** CHAPTER: INTRODUCTION  ***********************************
\section{Introduction}

Modeling of observational metadata has been an important activity of the IVOA since its creation in 2002.
Various modeling efforts like the Resource Metadata, the Space Time Coordinates (STC), the Spectrum, and the Characterisation data models have become approved IVOA standards and are currently used in IVOA services and applications.

%alberto's: I propose here a new introduction, see later the %amcom commented section for the original version.

Meanwhile, the Table Access Protocol (TAP) was developed and reached recommendation status in March 2010.
TAP defines a service protocol for accessing tabular data, like astronomical catalogues, or, more generally, database tables.
TAP basic idea is to allow a client to (step 1) browse through the various tables and columns (names, units, etc.) to collect 
the information necessary to then (step 2) actually perform a tabular query. 

Building on the two above-mentioned important blocks, data providers have now the ability to expose
data model-based information through the TAP protocol.

The third fundamental building block relevant to this effort is an initiative of the IVOA Take Up Committee that,
in the course of 2009, collected a number of use cases for data discovery (see Appendix \ref{app:use-cases}).
Those use cases aim at providing an astronomer the possibility to pose a world-wide query for scientific data
 of certain characteristics and to eventually retrieve the relevant data products.
The ability of posing scientific queries to several data centres at once is a fundamental case for the Virtual Observatory.

The last step for an homogeneous access across all data centres, which would allow a client
to send one and the same query to all TAP services that implement the same model, is the
stipulation that a data model is exposed (through TAP) using agreed naming conventions,
formats, units, and reference systems. This is what this document is for. 

The purpose of this document is twofold: (1) to define a simple core data model for observational metadata,
and (2) to define the standard way to expose it through the TAP protocol, so to be able to provide a uniform
interface to observational data products. The herein described ObsTAP protocol is the answer to the questions
posed by the Take Up Committee (2009) now changed to Standing Committee for Science Priorities(May 2010).
% end of alberto's

%amcom ---------------------------------------------------------------
%amcom This work now reaches a mature state where we need to homogenise the various description and access ways to discover, retrieve and/or analyse any kind of data products resulting from an astronomical observation.
%amcom However data model implementation among data providers is still poorly represented except for the most common data model fields used to publish a data collection in the Registry (based on the Resource metadata DM) or in protocols like the well supported SSA or SIA.
%amcom %mir
%amcom  This situation is also reinforced by more technical problems: serialisation (pure
%amcom XML or Utypes), and protocols for metadata access were not always available
%amcom for practical implementation of these data models.  The emergence of
%amcom the TAP protocol allows now to use a generic method for all kind of datasets
%amcom and models. Definition of TAP services implementing the Core of Observation data model for various archives will provide a unified discovery interface to a large set of heterogeneous data archives of images, cubes, spectra and catalogs.  
%amcom 
%amcom The goal of the current note is to propose a subset of Utypes of the overall Observation data model currently under construction that enables to support a full set of use-cases collected from the community, and to show how it can be implemented as a TAP service. 
%amcom Such a simple and general service should be easily implemented by any data provider, for any kind of data.
%amcom 
%amcom ---------------------------------------------------------------
The document is composed as following: 

% --- DOCUMENT STRUCTURE ---

\begin{itemize}
\item Section  \ref{sec:usecase} briefly presents an overview on the use cases collected within the astronomical community by the IVOA Uptake committee. 
\item Section \ref{sec:desc_model} describes how the Observation/Characterization data model interact and defines the core components of the Observation data model in UML. The various elements of the data model are summarized in the table at figure \ref{tab:dmsummary}
\item Section \ref{sec:impl} specifies the required data model fields as they are used in the TAP service: table names, column names, column datatype, utype from the Observation Core components data model, and required units. 

\item Section \ref{registering} provides information to register an Obs/TAP service into the Virtual Observatory Registries.
More detailed description are available in the appendices
%\item Section 7 is the conclusion. %amcom: I could not find such section.
% --- APPENDICES ---
\item Appendix \ref{app:use-cases} lists all the use cases as defined by the Take Up Committee
\item Appendix \ref{app:dmdesc} contains a full description of the Observation data model Core Components, that completes the table summary in data model section.
\item Appendix \ref{app:sample-ddl} shows how to build up and fill in the TAP/SCHEMA tables necessary for the implementation of an OBS/TAP service.

\item Appendix \ref{app:examples} shows examples of Obs/TAP queries.
%\item Appendix (TBW)(tracebility matrix) shows which use cases are actually covered by this Obs/TAP standard, and which ones would require additional work by the IVOA working groups.
\item Section \ref{obstap-implementations} describes two implementations realised with the Obs/Tap service 
\item Section \ref{app:docupdate} Updates of the document.

% --- END OF DOCUMENT STRUCTURE ---
\end{itemize}

%\newpage
% mireille ***CHAPTER: USE CASE: introduced shortly here with pointer on the APPENDIX prepared by (A.Micol)***
\section{Use cases}
\label{sec:usecase}
We first focused on data discovery use-cases, aimed at finding observations in the VO domain, by broadcasting the same query to a bunch of data centers or to all VO subscribers.

Ultimately we need to provide data providers with a list of items and features that they could easily map to their database system, in order to answer to the kind of queries listed below.

The goal is to be simple enough to be implementable, and not to be exhaustive on all exotic data sets.
 
The main features of these use-cases are mainly: 
\begin{itemize}
	\item  multi-wavelength search
  \item  all kinds of data products(spectrum, cube, time series, etc.)
  \item  various distribution formats data products(FITS, VOTable, compressed file archive,etc.)
 \end{itemize}
 
Refined or advanced searches may include extra knowledge stemming from astronomical objects classification and would need to extract results from catalogs, possibly by using fine sub-queries.

The detailed list of use cases proposed for data discovery is in the appendix \ref{app:use-cases}.
% ********************************** CHAPTER: USE CASE: ABORTED IN FAVOUR OF AN APPENDIX (A.Micol)  ***********************************
%\newpage
% alberto's: The Use Cases section was providing a couple of examples. I moved such section to an appendix,
% where I have inserted ALL the use cases. The idea is that Use Cases are important to define the project,
% but are not part of the protocol/model itself; an implementer needs a working document, the rationale is
% important but not immediately relevant to an implementer. All the rest is therefore %amcom commented out.

%amcom ---------------------------------------------------------------
%amcom \section{Use cases}
%amcom \label{sec:usecase}
%amcom 
%amcom  We first focused on data discovery use-cases, aimed at finding observations in the VO domain, by broadcasting the same query to a bunch of data centers or to all VO subscribers.
%amcom  
%amcom  Ultimately we need to provide data providers with a list of items and features that they could easily map to their database system, in order to answer to the kind of queries listed below.
%amcom
%amcom The goal is to be simple enough to be implementable, and not to be exhaustive on all exotic data sets.
%amcom 
%amcom  The main features of these use-cases are mainly: 
%amcom  \begin{itemize}
%amcom  	\item  multi-wavelength search
%amcom    \item  all kinds of data products(spectrum, cube, time series, etc.)
%amcom    \item  various distribution formats data products(FITS, VOTable, compressed file archive,etc.) % alberto: this is not in the use cases.
%amcom   \end{itemize}
%amcom   
%amcom Refined or advanced searches may include extra knowledge stemming from astronomical objects classification and would need to extract results from catalogs, possibly by using fine sub-queries.
%amcom 
%amcom Here we list just one example in each use-case category but the full list of data discovery use-cases we considered is available at \\ 
%amcom {\scriptsize \url{http://www.ivoa.net/internal/IVOA/ObsDMCoreComponents/TAPObsDM_ScienceUseCasesv6.doc}}.  \\
%amcom 
%amcom This list is probably not an exhaustive view of all queries astronomers may formulate but most types of searches are present. Fine queries to a specific archive or data collection that a user is very familiar to is always feasible and will be commented below.
%amcom 
%amcom \subsection {Discover imaging data of interest}
%amcom \label{sec:usecaseImages}
%amcom 
%amcom Use-Case 1.2: Let me input a list of RA and DEC coordinates and show me spatially coincident data that satisfies
%amcom \begin{itemize}
%amcom \item [1.2.1] Data type is Imaging or spectroscopy data 
%amcom \item [1.2.2] Includes one or more of the RA,DEC 
%amcom \item [1.2.3] Includes both a wavelength in the range 5000-900 angstroms AND an X-ray image (AND=SERVREQ)
%amcom \end{itemize}
%amcom  
%amcom \subsection{Discover Spectral data of interest }
%amcom \label{sec:u_spect}
%amcom Use-case 2.2: Show me a list of all data that satisfies 
%amcom \begin{itemize}
%amcom \item [2.2.1] DataType=Spectrum
%amcom \item [2.2.2] Wavelength includes 6500 angstroms
%amcom \item [2.2.3] Spectral Resolution better than 15 angstroms
%amcom \item [2.2.4] Spatial Resolution better than 2 arcseconds FWHM
%amcom \item [2.2.5] Exposure Time $>$ 3600 seconds
%amcom \item [2.2.6] Data Quality = Any
%amcom \end{itemize}
%amcom 
%amcom \subsection{Discover Data cubes of interest }
%amcom \label{sec:u_cube}
%amcom 
%amcom Use-Case 3.4 : Show me a list of all data that satisfies
%amcom \begin{itemize}
%amcom \item [3.4.1] DataType=cube
%amcom \item [3.4.2] RA includes 16.00
%amcom \item [3.4.3] Dec includes +41.00
%amcom \item [3.4.4] Wavelength includes 6500 angstroms
%amcom \item [3.4.5] Wavelength includes 4000 angstroms
%amcom \item [3.4.6] Spectral resolution better than 5 angstroms
%amcom \item [3.4.7] Exposure time more than 3600 seconds
%amcom \item [3.4.8] Data Quality= Fully Calibrated
%amcom \end{itemize} 
%amcom 
%amcom \subsection{Discover general data of interest }
%amcom \label{sec:u_alltype}
%amcom 
%amcom Use-Case 5.3 : Show me a list of all data that satisfies
%amcom \begin{itemize}
%amcom \item [5.3.1] DataType=Imaging or Spectroscopy
%amcom \item [5.3.2] RA includes 16.00 hours
%amcom \item [5.3.3] DEC includes +41.00 degrees
%amcom \item [5.3.4] SDSS images and spectra AND CFHTLS images and spectra
%amcom \end{itemize} 
%amcom 
%amcom \subsection*{How to answer these use cases? }
%amcom \label{sec:sol}
%amcom A common denominator to most of the use-cases is the set of constraints put on physical features in terms of spatial, spectral, temporal, and photometric properties.  This is well covered by the Characterisation Data Model and can be re-used within the Observation DM.
%amcom The Characterisation DM as well as the Spectrum DM define lists of Utypes ~\cite{UtypeCharac} that are ready to use for our cases.
%amcom 
%amcom Access and identification metadata, defined in the Spectrum data model and summarized in the Simple Spectral Access Protocol Utype list ~\cite{SSA}, provide the general mechanism necessary to identify and retrieve a data set.
%amcom 
%amcom 	The goal of the Obs/Tap service specification is to build up a VO interface to allow data providers to simply describe their observation metadata stored in DBMS systems and provide a query mechanism for users to discover, then retrieve the data products. The currently developed TAP protocol offers the appropriate implementation layer. Here we specify the data model and the relational model that TAP services must implement. 
%amcom 	
%amcom 	The Observation data model Core Components (also named ObsCoreDM for short) is present as a logical set of Utypes, mapped to columns of the tables exposed in an archive.
%amcom The advantage of this approach is that users will be able to submit an identical query to all TAP services that implement the ObsCoreDM  and get back uniform results. 
%amcom This can be described in a standard way using a \verb!TAP_SCHEMA! . 
%amcom Queries to databases may be done in all cases by ADQL as well as other query languages supported by individual services (for example using PQL when that standard will be finalised and implemented).
%amcom ---------------------------------------------------------------

% ********************************** CHAPTER: OBSCORE DATA MODEL ***************************************************
\newpage
\section{Observation Core Components Data Model}
\label{sec:desc_model}

%alberto's
This section highlights and describes the components of the observation data model necessary to support the
well-identified set of use cases listed in Appendix \ref{app:use-cases}. That is, all parameters used to characterise
the observations should be included in the model. In reality, it is worth mentioning here that this effort is the outcome
of a trade-off between what Astronomers want and what Data Providers are ready to offer. The aim is to quickly buying-in
data providers with a simple and good enough model to cover the majority of the Astronomers'needs.

%amcom ---------------------------------------------------------------
%amcom Here we sketch out which main core components of the overall Observation data model are necessary to support the above use-cases. 

%amcom All metadata useful to formalise search criteria should be included.
%amcom The model should also include the information necessary to the user to sort and select the data set he/she wants to recover from the query response list.  
%amcom ---------------------------------------------------------------

The project of elaborating a general data model for the metadata necessary to describe any astronomical observation was launched at the first Data Model WG meeting held in Cambridge, UK at the IVOA meeting in May 2003. The Observation data model was sketched out relying on some key concepts : Dataset or Observation, Identification, Curation, Physical Characterisation and Provenance (either instrumental or software). A description of the early stages of this development can be found in \cite{obsnote}, (Observation IVOA note). 
Some of these concepts have already been fleshed out in existing data models namely the Spectrum data model \cite{IVOASpectrum} for the general items and the Characterisation data model \cite{IVOAchar} for the description of the physical axes and properties of an observation.
       
\subsection{Characterisation data model}
The Characterisation data model organizes metadata as in a 3D matrix spanning independently the various physical axes of an observation (spatial, spectral, time, flux or whatever observable quantity), expressing 'Properties' such as  coverage, resolution, sampling, accuracy of an observation along these axes and with up to four levels of description if necessary. This scheme allows to support selection of data sets for data discovery as well as data analysis. 

%\subsubsection{ New efforts}
%While the Characterisation data model was setting up a logical framework to describe the properties/features of each observation in the VO, the approach here is more pragmatic and leads to a simple logical data model, and its protocol implementation using the emerging TAP/SCHEMA framework.
%In the mean time, the Generic Data set/Observation data model is currently developed integrating Curation and DataID description borrowed from the Spectrum Data model with a detailed description of the Provenance ( instrumental and computational) for observed datasets.
%The consistency between the two efforts is a major goal, and will be warranted by defining the Core components to be re-usable at the more general level.
%        
\subsection{Observation data model Core Components and attributes definition}
Metadata of interest in the data discovery process are not only the one used to express queries and adjust selection criteria ( type of product, positions, value, range, etc.) but also the returned parameters that allow to identify, in the query response the returned data sets and access to them.
  
From all the metadata covered and described in the full Observation data model, only a small part is needed to support data discovery in a regular and efficient manner. We then concentrate first on the definition of core components of the Observation data model that will be used to support the use-cases described above. The Observation data model Core Components (ObsCore DM) is summarized in a UML class diagram below; this is a \textit{logical} data model that described and binds together the main concepts related to an observation in astronomy. The implementation of such a model is described in Section \ref{sec:impl}.
 
\subsubsection{ UML description of the model}

The Observation data model is organised according to some Object Oriented Programing principles in order to define unique and consistent concepts, as re-usable classes. UML helps to sketch out the class organisation as shown in Fig.
\ref{fig:dmoverview}.
This class diagram covers all classes used in the context of the Observation Core components Model. 

The Characterisation classes, describing how the data span along the main physical axes, are re-used here partly, with  only the necessary attributes shown here.
This is also the case for the DataID and Curation classes extracted from the Spectrum/SSA data model where only a subset of attributes are necessary for data discovery. We consider for now that we use Characterisation classes only down to the level of the Support Class (level 3).

\begin{figure}[!ht]
\begin{flushleft}
%\includegraphics[angle=90]{./images/obscoreOverviewJune3} too large 
\includegraphics[height=0.85 \textheight]{./images/Obscore-dm-overview}
\caption{\footnotesize Here is a class diagram representing the classes used to organise observational metadata. Classes may be linked together via an association or aggregation link. The minimal set of necessary attributes for data discovery is shown in brown.}
	\label{fig:dmoverview}
\end{flushleft}
\end{figure}

For the sake of clarity, the SpatialAxis, SpectralAxis and TimeAxis classes on the diagram, are not expanded on the main class diagram. 

Details for these axes are shown on Fig.\ref{fig:spatdetail} for the Spatial axis, Fig.\ref{fig:specdetail} for the spectral axis.
%mir no such fig useful for now/ Fig.\ref{fig:timedetail} for the temporal axis (or observable) .  


\begin{figure}[!ht]
\begin{flushleft}
\includegraphics[angle=90, height=0.80 \textheight]{./images/spatialAxisDetails}
\caption{\footnotesize Details of the classes linked to the description of the Spatial axis for an Observation. All axes in this model inherits the main structure from the CharacterisationAxis class, but some peculiar attributes are necessary for Space coordinates.}
	\label{fig:spatdetail}
\end{flushleft}
\end{figure}
\clearpage

\begin{figure}[!ht]
\begin{flushleft}
\includegraphics[angle=90, height=0.85 \textheight]{./images/spectralAxisDetails}
\caption{\footnotesize Spectral axis: details of the classes necessary to describe the spectral properties of an Observation. UCD and units are essential to disentangle various possible spectral quantities.}
	\label{fig:specdetail}
\end{flushleft}
\end{figure}
\clearpage

%\begin{figure}[!ht]
%\begin{flushleft}
%\includegraphics[angle=90, height=0.8 \textheight]{./images/TimeAxisDetails}
%\caption{\it Details of the classes necessary to describe the properties of the Time axis for an Observation. Note the Time reference is important. }
%	\label{fig:timedetail}
%\end{flushleft}
%\end{figure}
	
Details on the axes definition are available in the Chararacterisation data model standard document \cite{IVOAchar}. 

\subsection{Data Model Summary}
The table below contains a list of the fields defined in the Observation data model Core Components. 
The mandatory fields used in the an OBS/TAP service are listed in green ans labeled with \textbf{M}.   They should be present in an Ivoa.ObsCore table for any service compliant to the Obs/Tap specification. See Section \ref{sec:impl}.

All other fields are optional and flagged with \textbf{O}; they can be used by data providers to support finer queries. These names are recommended only ; This gives the possibility to the data provider to define her/his own set of additional parameters, when not defined here. However, for the sake of homogeneity between services, using optional fields defined here is preferable.

The hypertext documentation of the model is (will be) available at \\ \footnotesize{\url{http://alinda.u-strasbg.fr/Model/ObsCore/index.html}}.

\normalsize
The Appendix \ref{app:dmdesc} contains an extensive description of the classes and attributes defined to capture all necessary metadata that complements the above list of items. 

%\begin{figure}
%	\centering
%		\includegraphics{images/ObsTAP-DMlistV2-1.pdf}
%	\caption{liste des champs du modèle}
%	\label{fig:ObsTAP-DMlistV2-1}
%\end{figure}

\subsection{New concepts necessary for the Observation data model} 
\label{sec:newconcepts}

Here we introduce new definitions in order to complete a description of a general observation, namely a data product type, for identifying the type of content, and a classification for various levels of calibration.

\subsubsection{Type of Data Product}  

%\textit{EDITOR TEMPORARY COMMENTS: (1) the UML in fig 1/2/3 does not show the product type, nor the calib level. (2) The relationship between an Observation and a Data Product is not clarified anywhere, and now that we introduce product type and calib level this is really needed: any idea on how to describe this (and also in the UML?)? (3) Doug's proposal might change the list of values, for now agreement was expressed for a simple list of values}\\
The model defines a \textit{data product type} attribute for the Observation Class. It is the type of observation the user queries for or selects for retrieval. This is coded as a string that conveys a general idea of the content and organisation of a dataset.
We consider a coarse classification of the types of dataset interesting for science usage, covering: image, spectrum, time series, visibility, event list, and cube.
The \footnotesize{Observation.DataProductType}
 attribute takes its string value in the following set: %amcom, organized according to up to 4 levels of granularity:
\begin{itemize}	
\item \textbf{image} 
\item \textbf{spectrum}
\item \textbf{cube}  
\item \textbf{timeseries}
\item \textbf{visibility} 
\item \textbf{eventlist}
\end{itemize}

\clearpage
\begin{figure}[t]
\begin{flushleft}
\includepdf[pages={1},angle=90]{./images/ObsTAP-DMlistV2-1.pdf}
\label{tab:dmsummary}
\end{flushleft}
\end{figure}

\clearpage
\begin{figure}[!ht]
\begin{flushleft}
\includepdf[pages={2},angle=90]{./images/ObsTAP-DMlistV2-1.pdf}
\end{flushleft}
\end{figure}

\clearpage

\begin{figure}[!ht]
\begin{flushleft}
\includepdf[pages={3},angle=90]{./images/ObsTAP-DMlistV2-1.pdf}
\end{flushleft}
\end{figure}
\clearpage

%amcom ---------------------------------------------------------------
%amcom\begin{itemize}
%amcom  \item Image.2D \\         
%amcom  \footnotesize{any 2D image: weight maps, bad-pixel map, etc: TBD}
%amcom  \item Image.Longslit\\    
%amcom  \footnotesize{2D image for a long slit spectrum  with one axis mapped on wavelength}
%amcom  \item Image.Cube	  \\
%amcom  \footnotesize{2D+ extra dimension(s})
%amcom  \item Image.Cube.Spectral
%amcom  \item Image.Cube.Time
%amcom  \item Image.Cube.Polarization
%amcom\end{itemize}
%amcom ---------------------------------------------------------------
  

%amcom ---------------------------------------------------------------
%amcom\begin{itemize}
%amcom  \item Spectrum.1D
%amcom  \item Spectrum.SED
%amcom  \item Spectrum.Polarisation
%amcom  \item Spectrum.Echelle
%amcom  \item Spectrum.IFU 	
%amcom  \footnotesize{(TBD)}
%amcom\end{itemize}
%amcom ---------------------------------------------------------------
  
%amcom ---------------------------------------------------------------
%amcom\begin{itemize}
%amcom  \item TimeSeries
%amcom  \item TimeSeries.LightCurve \\       
%amcom  \footnotesize{flux variable with Time}
%amcom\end{itemize}
%amcom ---------------------------------------------------------------
  


%amcom ---------------------------------------------------------------
%amcom\begin{itemize}
%amcom%  ? Do we need such subtypes?
%amcom\item Visibility.Image
%amcom\item Visibility.Cube
%amcom\end{itemize}
%amcom ---------------------------------------------------------------
  

%alberto's: I fundamentally disagree with this next paragraph on the grounds that if data providers are allowed
%to come up with any value they want to, the standardisation will break. I therefore comment this out (%amcom)
% until we clarify it.

%amcom ---------------------------------------------------------------
%amcom This provides a hierarchy of possible data product types that is stated here
%amcom but can be extended by data providers in the future, depending on new kinds of data
%amcom and search procedures.
%amcom We give the possibility here for data provider to extend this classification
%amcom and return a more precise string as a data product type. ( see Doug's proposition )
%amcom However, this should start with some element given in the hierarchy above and should
%amcom not be redundant with any already existing string defined above.
%amcom ---------------------------------------------------------------
 
\subsubsection{Calibration level}

The calibration level concept conveys to the users information on how much data reduction/processing has been applied
to the data.  It is up to the data provider to consider how to map his own internal classification to the suggested scale here:

%alberto's: between level 1 (raw) and level 2 (calibrated on all pyshical axes) we miss CALIBRATED ON SOME OF THE AXES: 
%alberto's: how do I describe ESO spectra calibrated only in wavelength and not in flux?
\begin{itemize} 
	\item Level \textbf{0}:
	raw instrumental data, in proprietary or anyway internal provider format, that need specific tools to be handled. 
	\item Level \textbf{1}:
	Instrumental data in a standard format(FITS, VOTable, SDFITS, ASDM, etc). Data sets with not all axes calibrated.  
	\item Level \textbf{2}: Science ready data, with instrument signature removed, and calibration status defined on all physical axes.
	\item Level \textbf{3}: Enhanced data products like mosaics, improved co-added image cubes, resampled or drizzled images, etc., spectra with calibrated velocity axis at a particular line rest frequency. 
%alberto's: this following sentence does not tell an implementer what to do, hence I comment it out
%amcom In such case, the improved calibration procedure is described by the data provider in some way, progenitors of such a data product can be identified into the reduction pipeline.
\end{itemize}

%alberto's: what has a PI to do with this?
%amcom  This classification is simple enough to cover all regimes. From the knowledge of the various steps in the data production pipeline, data providers will adjust the mapping of their various internal levels of calibration to this general frame. This clearly is not automatically mappable from one archive to the other and on the contrary offers every data providers to publish in a general grid of description for interoperability.


Following examples can help to find the most appropriate value for the \textit{calib\_level} attribute.
\paragraph{Examples of data sets and their calibration level}

Here are examples of various data sets, classified according to scheme defined above.

\textit{Editor  MIR comment : please check . Should we add a link to some proper example files?}

\begin{table}[htbp]
\begin{tabular}{|l | l | l | l| l|}
\hline
data product&	data collection 	& Calibration & Comments \\ 
type				&	name							&	Level				& \\ \hline \hline 
spectrum		& XMM-Newton 				& 1  					& \\
 						& EPIC spectra 			&    					& \\ \hline
Image 			&	IRAS /NASA 				& 2     			& science ready \\
						&										&							&	data \\ \hline
Image				& IRIS/IRSA 				&	3 					& recalibrated from \\
						&										&							& infrared IRAS images \\ 
				    &										&							& with removal of the \\
				    &										&							&	sensor memory effect. \\ \hline
Image				& HDFS/ACS 					& 3 					& Image associations \\ 
						&	GOODS data				&							& mosaicing/stacking \\ \hline
EventList		&Rosat/HEASARC 			&	1 & \\ \hline
Visibility 	&Merlin  						&	3 					& recombination wrt diff axes \\ \hline
\end{tabular}
\caption{ \footnotesize Exemples of data sets with calibration level}
\label{tab:caliblevelex}
\end{table}


% ********************************** CHAPTER: IMPLEMENTATION OF OBSCORE IN A TAP SERVICE *******************************************
\newpage
\section{Implementation of ObsCore in a TAP Service}
\label{sec:impl}

The ObsCore model will be implemented within Table Access Protocol (TAP) services such that all
valid queries can be executed unchanged on any service that implements the model, as long as they do not make use of additional columns (see \ref{sec:extracols}).

Here we specify an explicit mapping of the model to relational database tables; in the context of TAP this means we are specifying the logical tables as described in the \verb!TAP_SCHEMA! (the TAP-required database schema where tables and columns included in the service are described). % and VOSI-tables metadata content.
This does not necessarily imply that the underlying database will have the identical structure (what is exposed through TAP could be, for example, a database view of the underlying database tables), but in most cases the relationship between \verb!TAP_SCHEMA! description and the underlying tables is straightforward.

\begin{table}[htbp]
\begin{tabular}{|l | l | l|}
\hline
\verb!schema_name! & \verb!table_name! & description \\
\hline \hline
\verb!ivoa! & \verb!ivoa.ObsCore! & ObsCore 1.0 \\
\hline
\end{tabular}
\caption{ \footnotesize TAP/SCHEMA.tables description of the ObsCore model}
\label{tab:tstables}
\end{table}

\begin{table}[htbp]
\begin{tabular}{|l | l | l | l| l|}
\hline
\verb!table_name! & \verb!column_name! & \verb!datatype! & \verb!units! & \verb!constraint! \\
\hline \hline
ivoa.ObsCore & \verb!dataproduct_type! & adql:VARCHAR & & not null \\ \hline
ivoa.ObsCore & \verb!calib_level! & adql:INTEGER & & not null \\ \hline
ivoa.ObsCore & \verb!obs_collection! & adql:VARCHAR & & not null \\ \hline
ivoa.ObsCore & \verb!obs_id! & adql:VARCHAR & & not null \\ \hline
ivoa.ObsCore & \verb!obs_publisher_did! & adql:CLOB & & not null \\ \hline
ivoa.ObsCore & \verb!access_url! & adql:CLOB & & \\ \hline
ivoa.ObsCore & \verb!access_format! & adql:VARCHAR & & \\ \hline
ivoa.ObsCore & \verb!access_estsize! & adql:INTEGER & KB & \\ \hline
ivoa.ObsCore & \verb!target_name! & adql:VARCHAR & & \\ \hline
ivoa.ObsCore & \verb!s_ra! & adql:DOUBLE & deg & \\ \hline
ivoa.ObsCore & \verb!s_dec! & adql:DOUBLE & deg & \\ \hline
ivoa.ObsCore & \verb!s_fov! & adql:DOUBLE & deg & \\ \hline
ivoa.ObsCore & \verb!s_region! & adql:REGION & deg & \\ \hline
ivoa.ObsCore & \verb!s_resolution! & adql:DOUBLE & arcsec & \\ \hline
ivoa.ObsCore & \verb!t_min! & adql:DOUBLE & d & \\ \hline
ivoa.ObsCore & \verb!t_max! & adql:DOUBLE & d & \\ \hline
ivoa.ObsCore & \verb!t_exptime! & adql:DOUBLE & s & \\ \hline
ivoa.ObsCore & \verb!t_resolution! & adql:DOUBLE & s & \\ \hline
ivoa.ObsCore & \verb!em_min! & adql:DOUBLE & m & \\ \hline
ivoa.ObsCore & \verb!em_max! & adql:DOUBLE & m & \\ \hline
ivoa.ObsCore & \verb!em_res_power! & adql:DOUBLE & & \\ \hline
ivoa.ObsCore & \verb!o_fluxucd! & adql:VARCHAR & & \\ \hline
\end{tabular}
\caption{\footnotesize TAP/SCHEMA.columns description of the ivoa.ObsCore table}
\label{tab:tscolumns}
\end{table}

Tables \ref{tab:tstables} and \ref{tab:tscolumns} provides the primary information needed to describe the ObsCore model in terms of \verb!TAP_SCHEMA! tables and columns. The "constraint" specified in Table \ref{tab:tscolumns} above is not part of the the \verb!TAP_SCHEMA.columns! description, but is required by the ObsCore model and specified here to make this clear to implementors.
Additional standard content for the individual columns is specified below. See Appendix B for a usable SQL script that inserts the standard \verb!TAP_SCHEMA! content. 

\subsection{Data Product Type}

\begin{itemize}	
\item \verb!column_name!: \verb!dataproduct_type!
\item datatype: adql:VARCHAR
\item size: implementor decides
\item units: NULL
\item utype: \darkgreen {\footnotesize{Observation.DataProductType}}
\item UCD: meta.id;class
\item principal: 1
\item indexed: implementor decides
\item std: 1
\end{itemize}

The \verb!dataproduct_type! column contains a simple string value describing the primary nature of the data product. Allowed values are: image, spectrum, timeseries, visibility, eventlist, cube. Values are in lower case. Subtypes, separated by one or more dots from the base categorisation, may be specified in provider-specific columns (see \ref{sec:extracols}).

Values in the \verb!dataproduct_type! column must not be NULL.

TODO: reconsider extending to two-level product type when the data model in previous section is completed. 

\subsection{Calibration Level}

\begin{itemize}	
\item \verb!column_name!: \verb!calib_level!
\item datatype: adql:INTEGER
\item size: NULL
\item units: NULL
\item utype: \darkgreen{\footnotesize{Observation.calibLevel}}
\item UCD: meta.id;class
\item principal: 1
\item indexed: implementor decides
\item std: 1
\end{itemize}

The \verb!calib_level! column tells the user the amount of calibration processing that has been applied to create the data product. Allowed values are: 0 (instrumental/raw data in a non-standard format), 1 (raw data in a standard format), 2 (calibrated data in standard format, instrument signature removed), and 3 (more highly processed data product). Data providers decide which value best describes their data products.

Values in the \verb!calib_level! column must not be NULL.
TODO Examples of calib level for well known data set examples:

\subsection{Collection Name}

\begin{itemize}	
\item \verb!column_name!: \verb!obs_collection!
\item datatype: adql:VARCHAR
\item size: implementor decides
\item units: NULL
\item utype: \darkgreen{\footnotesize{Observation.DataID.collection}}
\item UCD: meta.id
\item principal: 1
\item indexed: implementor decides
\item std: 1
\end{itemize}

The \verb!obs_collection! column identifies the data collection to which the data product belongs.  A data collection can be any collection of datasets which are alike in some fashion.  Typical data collections might be all the data from a particular telescope, instrument, or survey. The value is either the registered shortname for the data collection, the full registered IVOA identifier for the collection, or a data provider defined shortname for the collection. Examples: HST/WFPC2, VLT/FORS2, CHANDRA/ACIS-S, etc.

Values in the \verb!obs_collection! column must not be NULL.

\subsection{Observation Identifier}

\begin{itemize}	
\item \verb!column_name!: \verb!obs_id!
\item datatype: adql:VARCHAR
\item size: implementor decides
\item units: NULL
\item utype: \darkgreen{\footnotesize{Observation.DataID.CreatorDID}}
\item UCD: meta.id
\item principal: 1
\item indexed: implementor decides
\item std: 1
\end{itemize}

The \verb!obs_id! column contains a collection-specific identifier for an observation. In the case where multiple data products are available for an observation (e.g. with different calibration levels), the \verb!obs_id! value will be the same for each product of the observation.

Values in the \verb!obs_id! column must not be NULL.

%TODO: Determine correct utype. 
%used the utype defined in Spectrum/SSA summary./Mireille 
%TOBE discussed
\subsection{Publisher Dataset Identifier}

\begin{itemize}	
\item \verb!column_name!: \verb!obs_publisher_did!
\item datatype: adql:CLOB
\item size: implementor decides
\item units: NULL
\item utype: \darkgreen{\footnotesize{Observation.Curation.PublisherDID}}
\item UCD: meta.ref.url;meta.curation
\item principal: 1
\item indexed: implementor decides
\item std: 1
\end{itemize}

The \verb!obs_publisher_did! column contains the IVOA dataset identifier \cite{IVOAIdent} for the published data product. This value is globally unique since different publishers (data centres) that provide access to a data product will each assign their own value. 

We specify the datatype as CLOB in the TAP service so that users will know they can only use the \verb!obs_publisher_did! column in the select clause of a query. That is, users cannot specify this column as part of a condition in the WHERE clause and implementors are free to generate the URL during output.

Values in the \verb!obs_publisher_did! column must not be NULL.

\subsection{Access URL}

\begin{itemize}	
\item \verb!column_name!: \verb!access_url!
\item datatype: adql:CLOB
\item size: NULL
\item units: NULL
\item utype: \darkgreen{\footnotesize{Observation.Access.Reference}}
\item UCD: meta.ref.url
\item principal: 1
\item indexed: 0
\item std: 1
\end{itemize}

The \verb!access_url! column contains a URL that can be used to download the data product (files).

We specify the datatype as CLOB in the TAP service so that users will know they can only use the \verb!access_url! column in the SELECT clause of a query. That is, users cannot specify this column as part of a condition in the WHERE clause and implementors are free to generate the URL during output.

\subsection{Access Format}

\begin{itemize}	
\item \verb!column_name!: \verb!access_format!
\item datatype: adql:VARCHAR
\item size: NULL
\item units: NULL
\item utype: \darkgreen{\footnotesize{Observation.Access.Format}}
\item UCD: meta.id;class
\item principal: 1
\item indexed: 0
\item std: 1
\end{itemize}

The \verb!access_format! column contains a string indicating the format of downloaded the data product (files). The values
are made up of dot-deparated words that describe (in increasing detail) the structure of the data file(s). Allowed values include any legal mimetype [REF] that describes the format or one of the following short forms: fits, fits.image, fits.image.mef, fits.table, fits.cube, fits.cube.mef, directory, VOTable, csv, tsv, pdf. 
TODO: Doug and Arnold to provide list by 2010-06-15.

%TODO: Determine correct utype.

\subsection{Estimated Download Size}

\begin{itemize}	
\item \verb!column_name!: \verb!access_estsize!
\item datatype: adql:BIGINT
\item size: NULL
\item units: KB
\item utype: \darkgreen{\footnotesize{Observation.Access.Size}}
\item UCD: size??
\item principal: 1
\item indexed: 0
\item std: 1
\end{itemize}

The \verb!access_estsize! column contains an approximate size (in KB) of the file(s) available via the \verb!access_url!.

%TODO: Determine correct utype.

\subsection{Target Name}

\begin{itemize}	
\item \verb!column_name!: \verb!target_name!
\item datatype: adql:VARCHAR
\item size: implementor decides
\item units: NULL
\item utype: \darkgreen{\footnotesize{Observation.Target.Name}}
\item UCD: meta.id;src
\item principal: 1
\item indexed: implementor decides
\item std: 1
\end{itemize}

The \verb!target! column contains the name of the target of the observation. This is typically the proper name of an astronomical object, but could be the name of a survey field.

\subsection{Central Coordinates}
The coordinate system in which coordinates are expressed is by default ICRS. 
\begin{itemize}	
\item \verb!column_name!: \verb!s_ra!
\item datatype: adql:DOUBLE
\item size: NULL
\item units: deg
\item utype: \darkgreen{\footnotesize{Characterisation.SpatialAxis.Coverage.Location.coord.Position2D.Value2.C1}}
\item UCD: pos.eq.ra
\item principal: 1
\item indexed: implementor decides
\item std: 1
\end{itemize}

The \verb!s_ra! column stores the ICRS Right Ascension of the centre of the observation.

\begin{itemize}	
\item \verb!column_name!: \verb!s_dec!
\item datatype: adql:DOUBLE
\item size: NULL
\item units: deg
\item utype: \darkgreen{\footnotesize{Characterisation.SpatialAxis.Coverage.Location.coord.Position2D.Value2.C2}}
\item UCD: pos.eq.dec
\item principal: 1
\item indexed: implementor decides
\item std: 1
\end{itemize}

The \verb!s_dec! column stores the ICRS Declination of the centre of the observation.

\subsection{Spatial Extent}

\begin{itemize}	
\item \verb!column_name!: \verb!s_fov!
\item datatype: adql:DOUBLE
\item size: NULL
\item units: deg
\item utype: \darkgreen{\footnotesize{Characterisation.SpatialAxis.Coverage.Bounds.Extent}}
\item UCD: phys.angSize;instr.fov
\item principal: 1
\item indexed: implementor decides
\item std: 1
\end{itemize}

The \verb!s_fov! column contains the approximate size of the region covered by the data product. For a circular region, this is the diameter (not the radius). It is an estimate of the Aperture angular size , as used in the Spectrum DM.

\subsection{Spatial Coverage}

\begin{itemize}	
\item \verb!column_name!: \verb!s_region!
\item datatype: adql:REGION
\item size: NULL
\item units: deg
\item utype: \darkgreen{\footnotesize{Characterisation.SpatialAxis.Coverage.Support.Area}}
\item UCD: ?
\item principal: 1
\item indexed: implementor decides
\item std: 1
\end{itemize}

We specify the datatype as the logical type REGION so that users can specify spatial queries using a single column and in a limited number of ways. If included in the select list of the query, the output is always an STC-S string as described in \cite{TAP}[section 6]. In the WHERE clause, the \verb!s_region! column can be used with the ADQL geometry functions (INTERSECTS, CONTAINS) to specify conditions; the service will generally have to translate these into native SQL that enforces the same conditions or a suitable approximation. Implementors may approximate the spatial query conditions by translating the INTERSECTS and CONTAINS function calls in the query. 

In addition, ADQL specifies several functions which may take the \verb!s_region! column as an argument: AREA, CENTROID, and COORDSYS. The AREA function returns the area (in sq. deg.) of the observed the region. In cases where the \verb!s_region! itself is an approximation (a bounding box, for example), this function should still return the actual value. This may be implemented by computing and storing the area in a separate column and converting the  AREA(\verb!s_region!) function call into a column reference in the query. The CENTROID function returns an ADQL POINT value; if used in the select list the output is always an STC-S string as described in \cite{TAP}[section 6]. The coordinates must be the same as those found in the \verb!s_ra! and \verb!s_dec! columns, which are provided for convenience. The COORDSYS function returns the coordinate system used for the \verb!s_region!; in the ObsCore model implementation here this is restricted to ICRS, so this can be implemented by converting the COORDSYS(\verb!s_region!) function call to a  constant in the query. 

\subsection{Spatial Resolution}

\begin{itemize}	
\item \verb!column_name!: \verb!s_resolution!
\item datatype: adql:DOUBLE
\item size: NULL
\item units: arcsec
\item utype: \darkgreen{\footnotesize{Characterisation.SpatialAxis.Resolution.refVal.CResolution}}
\item UCD: pos.angResolution
\item principal: 1
\item indexed: implementor decides
\item std: 1
\end{itemize}

TODO: define here...

\subsection{Time Bounds}

\begin{itemize}	
\item \verb!column_name!: \verb!t_min!
\item datatype: adql:DOUBLE
\item size: NULL
\item units: d
\item utype: \darkgreen{ \footnotesize{Characterisation.TimeAxis.Coverage.Bounds.limits.Interval.StartTime}}
\item UCD: time.start;obs.exposure
\item principal: 1
\item indexed: implementor decides
\item std: 1
\end{itemize}

The \verb!t_min! column contains the start time of the observation in Modified Julian Day(s).

\begin{itemize}	
\item \verb!column_name!: \verb!t_max!
\item datatype: adql:DOUBLE
\item size: NULL
\item units: d
\item utype: \darkgreen{\footnotesize{Characterisation.TimeAxis.Coverage.Bounds.limits.Interval.StopTime}}
\item UCD: time.stop;obs.exposure
\item principal: 1
\item indexed: implementor decides
\item std: 1
\end{itemize}

The \verb!t_max! column contains the stop time of the observation in Modified Julian Day(s).

\subsection{Exposure Time}

\begin{itemize}	
\item \verb!column_name!: \verb!t_exptime!
\item datatype: adql:DOUBLE
\item size: NULL
\item units: sec
\item utype: \darkgreen{\footnotesize{Characterisation.TimeAxis.Coverage.Support.Extent}}
\item UCD: time.duration;obs.exposure
\item principal: 1
\item indexed: implementor decides
\item std: 1
\end{itemize}

The \verb!t_exptime! column contains the exposure time. For simple exposures, this is just \verb!t_max - t_min! expressed in seconds. For data where the detector is not active at all times (e.g. products made by combining exposures taken at different times), the \verb!t_exptime! will be smaller than \verb!t_max - t_min!. For data where the \verb!t_exptime! is not constant over
the entire data product, the median exposure time per pixel is a good way to characterise the typical value. In all cases,
\verb!t_exptime! is generally used as an indicator or relative sensitivity (depth) within a single collection (e.g. \verb!obs_collection!); providers should supply a suitable relative value when it is not feasible to define or compute the true 
exposure time.

\subsection{Time Resolution}

\begin{itemize}	
\item \verb!column_name!: \verb!t_resolution!
\item datatype: adql:DOUBLE
\item size: NULL
\item units: sec
\item utype: \darkgreen{\footnotesize{Characterisation.TimeAxis.Resolution.refVal}}
\item UCD: time.resolution
\item principal: 1
\item indexed: implementor decides
\item std: 1
\end{itemize}

The minimal interpretable interval between two points along the time axis. Can be an averaged or typical value. 

\subsection{Spectral Bounds}
As mentioned in the data model appendix, \ref{par:specaxis}, at least 3 physical quantities can be mapped on the spectral axis: energy, wavelength and frequency.
In this simple model we restrict the spectral bounds to wavelength expressed in meter units.
Conversion to other quantities could be performed either at the client site , for an application encapsulating queries,
or at the server side , for a data provider to expose its data from other regimes to OBS/TAP queries.

\begin{itemize}	
\item \verb!column_name!: \verb!em_min!
\item datatype: adql:DOUBLE
\item size: NULL
\item units: m
\item utype: \darkgreen{\footnotesize{Characterisation.SpectralAxis.Coverage.Bounds.limits.Interval.LoLim}}
\item UCD: em.wl;stat.min
\item principal: 1
\item indexed: implementor decides
\item std: 1
\end{itemize}

The \verb!em_min! column contains the minimum energy observed, expressed as (Barycentric? vacuum?) wavelength.

\begin{itemize}	
\item \verb!column_name!: \verb!em_max!
\item datatype: adql:DOUBLE
\item size: NULL
\item units: m
\item utype: \darkgreen{\footnotesize{Characterisation.SpectralAxis.Coverage.Bounds.limits.Interval.HiLim}}
\item UCD: em.wl;stat.max
\item principal: 1
\item indexed: implementor decides
\item std: 1
\end{itemize}

The \verb!em_max! column contains the maximum energy observed, expressed as (Barycentric? vacuum?) wavelength.

\subsection{Spectral Resolution (Resolving Power)}

\begin{itemize}	
\item \verb!column_name!: \verb!em_res_power!
\item datatype: adql:DOUBLE
\item size: NULL
\item units: NULL
\item utype: \darkgreen{\footnotesize{Characterisation.SpectralAxis.Resolution.ResPower.refVal}}
\item UCD: spec.resolution
\item principal: 1
\item indexed: implementor decides
\item std: 1
\end{itemize}
%ok for Respower.refval
The \verb!em_res_power! column contains the typical or characteristic resolving power of the energy axis. The value is dimensionless (e.g. delta lambda /lambda).

\subsection{Observable Axis Description}

\begin{itemize}	
\item \verb!column_name!: \verb!o_fluxucd!
\item datatype: adql:VARCHAR
\item size: implementor decides
\item units: NULL
\item utype: \darkgreen{\footnotesize{Characterisation.ObservableAxis.ucd}}
\item UCD: meta.code
\item principal: 1
\item indexed: implementor decides
\item std: 1
\end{itemize}

The \verb!o_fluxucd! column contains a UCD \cite{ucd} describing the nature of the observable within the data product. The observable is the measured quantity, for example photon counts or photon density stored in the pixel value within an image. 

TODO: Doug and Arnold to provide a list of observables, Francois and Mireille to map this to a list of typical UCDs by 2010-06-15.

\subsection{Additional Columns}
\label{sec:extracols}
Service providers may include additional columns in the ivoa.ObsCore table to expose additional metadata. These columns
must be described in the \verb!TAP_SCHEMA.columns! table and in the output from the VOSI-tables resource. Users may access these columns by examining the column metadata for individual services and then using them explicitly in queries or by selecting all columns in the query (e.g. "select * from ivoa.ObsCore ..." in an ADQL query).
In order to provide homogeneity in the keywords used as optional fields, we recommend to use the items defined in the data model and flagged as optional.

% ********************************** CHAPTER: Registration ***************************************************
\section{Registering a TAP Service with the ObsCore Model}
\label{registering}

TAP services that implement the ObsCore model should be registered to indicate this fact so that users can easily find all
services that accept ObsCore queries. Initially, this can be done by using the keyword "ObsCore" to describe the service.
Fine-grained registries may include the complete VODataService \verb!tableset! description, but this is not available through
all searchable registries. The TAP extension schema [to come] allows service providers to specify which IVOA data models are used 
within a TAP service. Services that implement the ObsCore model should use the registry extension schema mechanism to publicise
their support with:
\begin{verbatim}
standardID="ivo://ivoa.net/std/ObsCore/v1.0"
\end{verbatim} 
placed (somewhere) in the TAP Registry extension schema [REF], where \\ \verb!ivo://ivoa.net/std/ObsCore/v1.0! is the IVOA identifier of the StandardRegExt [REF] defined in this document. 

\bibliographystyle{plain}
\bibliography{obscore}

% ********************************** APPENDIX: USE CASES ***************************************************
\newpage
\appendix

\section{Use Cases in detail}
\label{app:use-cases}
\input{usecases}

% ********************************** APPENDIX: OBSTAP QUERY EXAMPLES ***************************************************
\section{Some Use cases translated as Obs/TAP ADQL queries}
\label{app:examples}
Here we consider a very general use case from the list (1.1) :\\
Show me a list of all data that satisfies:
\begin{enumerate}
	\item[1.1.1] Datatype=any
	\item[1.1.2] contains RA=16.0 and DEC=40.0
\end{enumerate}

These data would be searched on all VO services by sending the following query:
\begin{verbatim}
SELECT * FROM ivoa.Obscore WHERE
CONTAINS(POINT(16.0,40.0),s_region)
\end{verbatim}


More constraints can be added in the following use-case (1.3): \\
Show me a list of all data that satisfies
\begin{enumerate}
\item[1.3.1] DataType=Image
\item[1.3.2] Spatial resolution better than 0.3 arcseconds
\item[1.3.3] Filter = J or H or K
\item[1.3.4] RA between 16 hours and 17 hours
\item[1.3.5] DEC between 10 degrees and 11 degrees
\end{enumerate}

Such a query needs to compute RA in degrees, extract information from Filter and adjust spectral intervals for search.
\begin{verbatim}
SELECT * FROM ivoa.Obscore_ext WHERE 
dataproduct_type=``Image.2D'' 
AND  s_resolution < 0.3
AND s_ra > 240 AND s_ra < 255 AND 
s_dec > 10 AND s_dec < 11 
AND 
(em_min > 2.1 AND em_max < 2.4) OR 
(em_min >= 1.6 AND em_max <= 1.8) OR 
(em_min >= 1.2 AND em_max <= 1.4)
\end{verbatim}

%\newpage
% ********************************** APPENDIX: DATA MODEL DOCUMENTATION ***************************************************
\newpage
\section{Data Model detailed description}
\label{app:dmdesc}


Here is a full description of the concepts used in the model, the classes and attributes used to support them.
We also mention the corresponding ``short name'' that would be used in the OBS/TAP implementation of this model and is part of Table \ref{tab:dmsummary}.
The prefix for all utypes in the Observation Core Components DM is \textit{\textbf{'obs:'}} but has been eluded here for a simpler reading.

\subsection{ Observation} 
This class is a place holder that gathers all metadata relative to an observed and distributed dataset. It points to existing classes of Spectrum DM and types from VODataservice \cite{VODataService}. 
 
\subsubsection{Type of Observation} \verb!dataproduct_type! 

The model defines a \textit{data product type} attribute for the Observation Class. It is the type of observation the user queries for or selects for retrieval. This is coded as a string that conveys a general idea of the content and organisation of a dataset.
Possible values for this string are described in section \ref{sec:newconcepts}.
The short name for this attribute is \verb!dataproduct_type! in the implemented Ivoa.ObsCore table, as shown in Table \ref{tab:tscolumns}.

\subsubsection{Calibration level}  \verb!calib_level!

It is a convention we suggest to use to classify the different possible calibration status of an observed dataset. 
These 4 categories allow to distinguish 4 levels of calibration and would be sufficient for 80 \% of the data collections. This will be up to the data provider to consider how to map his own internal classification to the suggested scale here.
 
Following examples can help to find the most appropriate value for the \textit{calibLevel} attribute.

\begin{itemize} 
	\item Level 0:\\
	raw instrumental data, possibly in proprietary internal provider format, that need specific tools to be handled. 
	\item Level 1:\\
	Instrumental data in a standard format(FITS, VOTable, SDFITS, ASDM, etc. The data may or may not be calibrated. Standards tools can handle it. 
	\item Level 2: \\ Science ready data , with instrument signature removed, and calibration status defined on all physical axes.
	\item Level 3: \\ Enhanced data products like mosaics, improved co-added image cubes, resampled or drizzled images, etc. spectra with calibrated velocity axis at a particular line rest frequency. 
	In such case, the improved calibration procedure is described by the data provider in some way, progenitors of such a data product can be identified into the reduction pipeline.
\end{itemize}
This classification is simple enough to cover all regimes. Data providers will adjust the mapping of their various internal levels of calibration to this general frame, with the knowledge of the PIs for each project.
%some parts from Doug
\subsubsection{ Target}
	This is the astronomical object of interest for which the observation was performed. Only the name is used in the Core Components model but the Target object is fully designed in the Spectrum data model. Serendipitous archives or surveys may not contain this information for all observations, so this can be 'NULL' if necessary or have a value like 'DARK' for instance to specify a dark field and not a pointed observation.

\paragraph{Target Name} \verb!target_name! \\
The first attribute is the name of the target and is flagged as mandatory.

\paragraph{Class of the Target source/object} \verb!target_class! \\
Indicates the type of object that was pointed for this observation. It is a string with possible values defined in a special vocabulary set to be defined : list of object classes (or types) used by the SIMBAD database, f.i., or defined in another IVOA vocabulary. 

A simple set can be defined for this data model version :
{STAR, GALAXY, QUASAR, RADIO SOURCE, LINER, GLOB.CLUSTER, DARK, OTHER} \\
TO BE completed

\subsection{Observation regime} \verb!em_domain!

The spectral regime to which an observation belongs is modeled as \\ \textsl{Observation.waveband} with short name :  \verb!em_domain! . This re-uses the waveband definition from the VODataService IVOA standard and the same set of enumerated strings: {RADIO, MILLIMETER, INFRARED, OPTICAL, UV, EUV, X-RAY, GAMMA-RAY} available at \cite{VODataService}.

\subsection{Identification/DataID Class}\verb!!

%\verb!! just to force a CR after a paragraph
After acquisition and reduction an observation is uniquely identified by its creator and gets a creator dataset identifier. This information is defined in the Spectrum data model in the DataID class. We re-use this class and the `Observation.DataID.CreatorDID' Utype string in order to distinguish two dataset curated by two different services (archives) but originating from the same creator. 
When broadcasting a query to multiple servers, the response may contain multiple copies of the same dataset, with a unique \verb! obs_creator_did! but possibly different \verb!obs_publisher_did! ( given by the data provider).
Therefore a unique identifier is needed here.
In the ObsCore model, the short name associated to this ID should be \verb!obs_id! (to be checked).

The second identifier used in this model is the one given by the data provider, and defined in the Curation Class.


\subsubsection{Creator name} \verb!creator_name! \\
This parameter gives the name of the institution which created the dataset.

\subsubsection{Creation date} \verb!creation_date! \\
The creation date parameter gives the date when the dataset has been created.

\subsection{Curation metadata} \verb!!

The Curation Class inherits from the Spectrum data model and VOResource concepts too. 
The various attributes for ObsCore are:

\paragraph{Publisher Dataset ID} \verb!publisher_did! \\
This is the identifier the publisher provides for this observation . It may differ from the original id given by the creator of the data set. ( new reduction , new version , etc..). 
The corresponding Utype mapped from Spectrum DM is \footnotesize{Observation.Curation.PublisherDID} and relates to the same definition as in the Resource Metadata standard.

\paragraph{Publisher Name} \verb!publisher_name! \\
The name of the service providing data : archive, data center, etc..
Not fully standardised. The corresponding Utype is \footnotesize{Observation.Curation.Publisher}.

\paragraph{Publisher identifier} \verb!publisher_id! \\
The IVOA ID for the data provider. 

\paragraph{Bibliographic reference} \verb!bib_reference! \\
Url or Bibcode for documentation. This is a forward link to publications which reference the data set.This is an exact re-use of the SSA definition. See ~\cite{SSA} Curation Metadata 4.2.5.6

\paragraph{Data Rights} \verb!data_rights! \\
This parameter allows to mention protected, public and not yet released data.
The date of the future release can be added for data set in the PI proprietary period.
Possible values are : PUBLIC, PROTECTED, PROPRIETARY ( check see VODataService? or VOresource)
	

\subsection{Number of observed segments(or components)} \verb!num_seg! 
Although we mostly deal with single observations, some data products are the result of combinations of multiple original observations. This is the case for the SED of a source, for drizzled, co-added, mosaic images for instance. This will also be the case for IFU data cubes obtained by summing dozens of short poses, etc.

This field provides the number of original progenitors used to build up the observation of interest.
It is stored as an integer value. This information is necessary for a full description of the characterisation of complex data products, but also from a provenance point of view.This clearly belongs to the features of an Observation but can be re-used within the Characterisation class of a global complex result of an observation combination process. 
How this field is shared between the two classes will be defined in a new revision (v 2.0) of the Characterisation data model.
The corresponding Utype is \footnotesize{ Observation.numSeg }.
%Additionally, an array of url pointing to individual progenitors can be set by the data provider.

\subsubsection{ Description of physical axes: Characterisation classes} 

	As mentioned in the use-cases, selection criteria for an observation depend on the physical axes contained in the dataset especially the position, band , time, and the type of observed quantity, that we call ``observable'' in the data model and can represent various types of flux but also velocity, etc.
	Such a description was tackled in the IVOA Characterisation data model from which we re-use mainly the 2 first level of details except for the spatial coverage where the support region (level 3) is used too.  
\paragraph {Spatial axis}

   	\begin{itemize}
    	\item The observation reference position
    	
    	Two coordinates in position are used to identify a reference position ( typically the center) of an observation in the sky, attached to a coordinate system definition. 
    	\begin{itemize}
    	\item Coordinate system \\
    	The coordinate system defined in the Characterisation DM is based on the STC:Coordsys class.
    	The model in principle supports all kind of coordinate systems defined in the STC reference list. However, the Obs/Tap implementation of the model mandates that queries expressed in ICRS should be supported in an Obs/TAP service. This allows a general query to be sent to multiple archives or data centers , but requires some interpretation /conversion of coordinates at the server side. Still this is efficient for the large data  discovery strategy we need to provide.
Data in a specific coordinate system will be available in a second step via client applications that would do the conversions or adapt the coordinate system to some specific servers.

 			\item Coordinates\\
    	The model uses the Location Class from the Characterisation DM, with the 2 Utypes values:\\    		
    	\scriptsize{Char.SpatialAxis.Coverage.Location.coord.Position2D.Value2.C1} \\
    	\scriptsize{Char.SpatialAxis.Coverage.Location.coord.Position2D.Value2.C2} \\    	
    	whose short names in the ObsCore table and ICRS system  are \verb! s_ra! and \verb! s_dec!.
    	\end{itemize}
    	
    	\item The covered region\\
    	The Coverage class along the spatial axis provide 2 possible concepts:
    	\begin{itemize}
    	\item the Bounds: a bounding box that can estimate very coarsely the coverage of an observation.
    	It is modeled as a couple of intervals on each coordinates with Utypes: \\ 
    	 \begin{scriptsize}
    	 Char.SpatialAxis.Coverage.Bounds.limits.Interval.LoLimit2Vec.C1\\
    	 Char.SpatialAxis.Coverage.Bounds.limits.Interval.HiLimit2Vec.C1\\
    	
    	 Char.SpatialAxis.Coverage.Bounds.limits.Interval.LoLimit2Vec.C2\\
    	 Char.SpatialAxis.Coverage.Bounds.limits.Interval.HiLimit2Vec.C2\\
    	 \end{scriptsize}
    	 
    	\item the Support \verb!s_region!\\ 
    	A precise region description of spatial footprint of the dataset using region types Circle, polygon, etc , provided in STC.
    	\footnotesize{Char.SpatialAxis.Coverage.Support.Area} and 	 \footnotesize{Char.SpatialAxis.Coverage.Support.AreaType} define this region, and STC-S can be used to serialise the values.
    	
    	\end{itemize}
    	    	
    	\item The spatial resolution \verb! s_resol! \\
The minimal size that can be distinguished along the spatial axis, \verb! s_resol! is stored in arcseconds and is called \footnotesize{Char.SpatialAxis.Resolution.RefVal} in the observation Utype list.
    		
     \item The astrometric calibration status   \verb!astrometric_calStatus! \\
A string to encode the calibration status along the spatial axis (astrometry). 
provide examples.
%Some observations only provide the raw pointing position (uncalibrated). Some other may have a raw linear relationship between the pixel coordinates and the world coordinates (RAW). Some other are really accuratly calibrated.
Possible values could be {Calibrated, Normalized, Absolute, Relative} cf charac.
TBD
    	\item Astrometric precision \verb!astrom_precision_stat! \\
This parameter gives an estimate of the astrometric statistical error after the astrometric calibration phase. 
The corresponding Utype will be \footnotesize{Char.SpatialAxis.Accuracy.statError.refval}.

    	\item Spatial sampling \verb!s_pixel_scale! \\
   This corresponds to the sampling precision of the data along the spatial axis. It is stored as a real number corresponding to the spatial sampling period , i. e., the distance in world coordinates system units between two pixel centers.
May contain two values if the pixels are rectangular. 
    	
    	\end{itemize} %spatial
		
\paragraph{Time axis}\verb!!\\
		Three time stamps are used:  \verb! t_start! ,  \verb! t_stop! , and  \verb! t_span! the elapsed time.
		A format like MJD is useful for easy calculations and preferred for the Observation Core components model.
		The elapsed time is expressed in days.\\	 	
    \begin{itemize} 
    
    \item Time resolution \verb!t_resolution! \\
    This item, mapped to the \footnotesize{Char.TimeAxis.Resolution.RefVal} Utype provides an estimate or average value of the temporal resolution. 
    
    \item Time Calibration Status \verb!t_cal_status!\\
    This parameter gives the status of time axis calibration. This is specially useful for time series.

		\item Time Calibration Error \verb!t_staterr!\\
     A parameter used if we can estimate a statistical error on the time measurements (for time series again).
		\end{itemize} %time
		
\paragraph{Spectral axis}\verb!!\\
\label{par:specaxis}
This axis is generally used to represent different kinds of physical measurements: wavelength, energy, frequency
or some interpretation of this with respect to a reference position like velocity.
		
		The data model distinguishes the various flavors of this axis using the ucd attached to it, \footnotesize{Char.SpectralAxis.ucd} or \verb! em_ucd! .
		Possible values for this ucd are defined in the Spectrum DM \cite{IVOASpectrum} in section 4.1
		
		\begin{itemize} 
		\item Spectral Bounds \\
		\verb! em_min! and \verb! em_max! gives the bounds of data present along the spectral axis.
		Such values are expressed as frequencies but using meters as units as it is easily convertible.
		
		\item Spectral Resolution \verb!em_resol!\\
       A mean estimate of the resolution (FWHM of the LSF). 
This is can be used for narrow range spectra whereas in the majority of cases , the resolution power is preferable due to the LSF variation along the spectral axis.

		\item Spectral Resolution Power \verb! em_res_power! \\
		The resolution power along this axis is called  \verb! em_res_power! and maps to the \\ \footnotesize{Char.SpectralAxis.Resolution.ResolPower} Utype in the Characterisation DM. 
		
		\item Resolution Power limits \\
		\verb! em_res_power_min! and \verb! em_res_power_max! parameters give the limit of variation of the resolution power in the observation.

		\end{itemize} %spectral

\paragraph{Observable axis} 
Most observations measure some flux quantity depending on position and/or wavelength/energy/frequency or depending on time.
Here we consider a more general axis: the ``Observable axis'' that can be either flux or any other quantity, the nature of which is given by the UCD attached to this axis.

%For instance :
%\begin{table}[htbp]
%\begin{tabular}{|l|l|l|}
%\hline
%\textbf{Observable} &\textbf{Ucd} &\textbf{Units} \\ \hline
%flux& phot.flux &  Jy \\ \hline
%radial velocity&phys.veloc& km/s \\ \hline
%Stockes U &phot.flux;phys.polarisation & Jy \\ 
%\hline
phot.flux.density;phys.polarization.stokes.I		
%\end{tabular}
%\end{table}

The possible UCD values are part of the UCD1+ vocabulary \cite{ucd}.
You can find simple flux classes like : phot.flux, phot.flux.density, phot.count, phot.mag, or more complex combinations like phot.flux.density;phys.polarization.stokes.I .
Table \ref{tab:listofobservable}	provides a list of possible triplets( observable name, ucd, units) for various observables data providers may want to describe in their archive. 

\includepdf[pages={1},angle=90, addtotoc={28, appendix, 1, List of UCD/Units for observables ,tab:listofobservable }]{./images/ListForObservable25Oct2010.pdf}
%, addtotoc={28, appendix, 1, Data Model Summary,\ tab:dmsummary} ne fonctionne pas 
\label{tab:listofobservable}
	\subsection{Provenance}
	Provenance contains a class to represent all the Observing configuration used to acquire an observation.
	Instrumental parameters are gathered here.
	
	\subsubsection{Instrument name} \verb!instrument! 	This class provides the name of the instrument used for the acquisition of the observation. It is given in the model as \\ \footnotesize{Provenance.ObsConfig.instrument.name} and encoded as a string.	
 	
 	\subsubsection{Access to the data}
 	
 	The data format as well as the url to access the data set are provided by the \textbf{Access} Class inherited from the SSA Utype list and mapped to \verb!access_format!  and \verb!access_url! short keywords respectively. 	
The estimated size of the data file is also given by the Size attribute and corresponds to  	


% ********************************** APPENDIX: SAMPLE SQL DDLs  ***************************************************
\newpage
\section{Obs/TAP Schema: Examples of table definitions in SQL Data Definition Language}
\label{app:sample-ddl}

All TAP services that implement the ObsCore model will have the same table and column descriptions in their \verb!TAP_SCHEMA!,
with the exception of the service-specific column descriptions as described in \ref{sec:impl}. Specifically, implementations
may change the description, the indexed flag, and the size (in the case of variable width columns).

Note: The following SQL was modified but not tested to match the requirements in section \ref{sec:impl}. A tested version will be included as soon as possible.

\textit{EDITOR TEMPORARY COMMENTS:Mir check/validate with existing implementation }

\begin{verbatim}
delete from TAP_SCHEMA.columns where table_name = 'ivoa.ObsCore';

delete from TAP_SCHEMA.tables where table_name = 'ivoa.ObsCore';

delete from TAP_SCHEMA.schemas where schema_name = 'ivoa';

insert into TAP_SCHEMA.schemas (schema_name,description) values
( 'ivoa', 
  'collection of tables for IVOA data models with standard relational mappings');

insert into TAP_SCHEMA.tables (schema_name,table_name,utype,description) values
( 'ivoa', 'ivoa.ObsCore', 'Observation', 'ObsCore 1.0' );

-- NOTE: values in the size column are implementation-specific 
--       and should be set by providers
-- NOTE: values in the description may be changed/enhanced by providers
-- NOTE: all other columns have values mandated by the specification
insert into TAP_SCHEMA.columns 
(table_name,column_name,datatype,unit,utype,ucd,size,principal,indexed,std,description) 
values
( 'ivoa.ObsCore', 'dataproduct_type', 'adql:VARCHAR', NULL,
  'Observation.DataProductType', NULL, 128, 1,0,1,
  'type of data product (image, spectrum, timeseries, visility, eventlist, cube)' ),

( 'ivoa.ObsCore', 'calib_level', 'adql:VARCHAR', NULL,
  'Observation.calibLevel', NULL, NULL, 1,0,1,
  'calibration level (0,1,2,3)' ),

( 'ivoa.ObsCore', 'target', 'adql:VARCHAR', NULL,
  'Observation.Target.name', NULL, 128, 1,0,1,
  'name of the intended target' ),

( 'ivoa.ObsCore', 'obs_collection', 'adql:VARCHAR', NULL,
  'Observation.DataID.collection', NULL, 128, 1,0,1,
  'name of the archive or data collection' ),
( 'ivoa.ObsCore', 'obs_id', 'adql:VARCHAR', NULL,
  'TBD:Observation.DataID.creatorDID', NULL, 128, 1,0,1,
  'IVOA dataset identifier assigned by the publisher' ),
( 'ivoa.ObsCore', 'obs_publisher_did', 'adql:CLOB', NULL,
  'Observation.Curation.PublisherDID', NULL, NULL, 128, 1,0,1,
  'IVOA dataset identifier assigned by teh publisher' ),

( 'ivoa.ObsCore', 'access_url', 'adql:CLOB', NULL,
  'Observation.Access.reference', NULL, NULL, 1,0,1,
  'URL to download the data' ),
( 'ivoa.ObsCore', 'access_format', 'adql:VARCHAR', NULL,
  'TBD:Observation.Access.format', NULL, 128, 1,0,1,
  'file format or structure of downloaded product(s)' ),
( 'ivoa.ObsCore', 'access_estsize', 'adql:INTEGER', 'KB',
  'TBD:Observation.Access.size', NULL, NULL, 1,0,1,
  'estimated download size' ),

( 'ivoa.ObsCore', 's_ra', 'adql:DOUBLE', 'deg',  
  'Characterisation.SpatialAxis.Coverage.Location.coord.Position2D.Value2.C1', 
  NULL, NULL, 1,0,1,
  'ICRS Right Ascension of central coordinates' ),
( 'ivoa.ObsCore', 's_dec', 'adql:DOUBLE', 'deg',
  'Characterisation.SpatialAxis.Coverage.Location.coord.Position2D.Value2.C2', 
  NULL, NULL, 1,0,1,
  'ICRS Declination of central coordinates' ),
( 'ivoa.ObsCore', 's_region', 'adql:REGION', 'deg',
  'Characterisation.SpatialAxis.Coverage.Support.Area', 
  NULL, NULL, 1,0,1,
  'region bounded by observation' ),
( 'ivoa.ObsCore', 's_resolution', 'adql:DOUBLE', 'arcsec',
  'Characterisation.SpatialAxis.Resolution.refVal.Cresolution', 
  NULL, NULL, 1,0,1,
  'typical/representative/characteristic spatial resolution' ),

( 'ivoa.ObsCore', 't_min', 'adql:DOUBLE', 'd',
  'Characterisation.TimeAxis.Coverage.Bounds.limits.Interval.StartTime', 
  NULL, NULL, 1,0,1,
  'start time of observation (MJD)' ),
( 'ivoa.ObsCore', 't_max', 'adql:DOUBLE', 'd',
  'Characterisation.TimeAxis.Coverage.Bounds.limits.Interval.StopTime', 
  NULL, NULL, 1,0,1,
  'end time of observation (MJD)' ),
( 'ivoa.ObsCore', 't_exptime', 'adql:DOUBLE', 'sec',
  'Characterisation.TimeAxis.Coverage.Support.Extent', 
  NULL, NULL, 1,0,1,
  'exposure time of observation' ),
( 'ivoa.ObsCore', 't_resolution', 'adql:DOUBLE', 'sec',
  'Characterisation.TimeAxis.Resolution.refVal', 
  NULL, NULL, 1,0,1,
  'typical/representative/characteristic Time resolution' ),

( 'ivoa.ObsCore', 'em_min', 'adql:DOUBLE', 'm',
  'Characterisation.SpectralAxis.Coverage.Bounds.limits.Interval.LoLim', 
  NULL, NULL, 1,0,1,
  'minimum wavelength' ),
( 'ivoa.ObsCore', 'em_max', 'adql:DOUBLE', 'd',
  'Characterisation.SpectralAxis.Coverage.Bounds.limits.Interval.HiLim', 
  NULL, NULL, 1,0,1,
  'maxmimum wavelength' ),
( 'ivoa.ObsCore', 'em_res_power', 'adql:DOUBLE', NULL,
  'Characterisation.SpectralAxis.Resolution.ResPower.refVal', 
  NULL, NULL, 1,0,1,
  'typical/representative/charateristic spectral resolution' ),

( 'ivoa.ObsCore', 'o_fluxucd', 'adql:VARCHAR', NULL,
  'Characterisation.ObservableAxis.ucd', NULL, 32, 1,0,1,
  'UCD describing the observable axis (pixel values)' )
;
\end{verbatim}


% ******************************************* APPENDIX: OBS/TAP Implementations ***********************************************
\section{OBS/TAP Implementations}
\label{obstap-implementations}
\subsection{OBS/TAP implementation at CADC}
tbc
\subsection{OBS/TAP service for XMM SSC catalog Data in the framework of Saada}
TBC, Laurent

CDS and XMM/SSC catalog team implemented a version of the Obs/TAP service on the XCatDB database, one of the public interfaces to the XMM-Newton catalogue \cite{Xmm}
The XCatDB contains millions of X-ray data files (time series, spectra, images, catalogs) distributed in
various formats (FITS, HTML, PNG, PDF, ZIPPED folders). 
An extension to the Saada application \cite{Saada}, supporting TAP access and using the Observation data model core components, allows managing this heterogeneous data collection.
 
Fig. \ref{fig:xmmwf} shows the workflow launched for a query on the service developped at CDS by G.Mantelet.
\begin{figure}[!ht]
\begin{flushleft}  
\includegraphics[width=\textwidth]{./images/sscXMMObsTAP}
\caption{\footnotesize The workflow executed by the SSC XMM Obs/Tap service for resolving a query. It has two execution modes as required for TAP: a synchronous one , top branch and an asynchronous one, using the UWS protocol. }
\label{fig:xmmwf}
\end{flushleft}
\end{figure}

\begin{figure}[!ht]
\begin{flushleft}  
\includegraphics[width=\textwidth]{./images/sscXMMObsTAP-UI}
\caption{\footnotesize User interface to the XMM SSC catalog Obs/TAP service in Strasbourg, with a specified query}
\label{fig:xmmui}
\end{flushleft}
\end{figure}

\begin{figure}[!ht]
\begin{flushleft}  
\includegraphics[width=\textwidth]{./images/sscXMMObsTAP-Result}
\caption{\footnotesize An example of a query response from the XMM SSC catalog Obs/TAP service to the query submitted in Fig.\ref{fig:xmmui}}
\label{fig:xmmqr}
\end{flushleft}
\end{figure}
This implementation is based on a smart ADQL editor developed on high level Javascript
features (Drag and Drop, Ajax).
The query responses are generated by default as VOTable XML documents displayed, via XSLT transformation, in a user-friendly panel, gathering lists of active urls to point to each individual result. Fig.\ref{fig:xmmui} shows how a user can submit a query with this interface, and Fig.\ref{fig:xmmqr} exposes the interface to the query response. 
More output formats for the query response are available : HTML, CSV,TSV.
This implementation takes benefits of extended access formats for data products. For instance, making spectrum plots available in visual format( pdf) allows end-users to have a quick look before downloading larger bulks of data.
%Choosing HTML in a simple web page allows to redirect users to the XCatDB pages from where the whole database can be
%browsed.??
Spectra are delivered in ZIP balls containing all data files relative to an observation and the attached calibration/instrumental files needed to achieve a complete calibration.
This service remains very relevant although only a few number (24?) of the ObsCore table fields can be populated due to the X-ray data specificity.

This TAP service implementation is available at \url{http://xcatdb.u-strasbg.fr/2xmmidr3/tap}. It is based on a
Java ADQL parser and UWS libraries , both to be released for IVOA usage on the Saada Web site \cite{Saada}.

% ********************************** APPENDIX: UPDATES OF THE DOCUMENT ***************************************************
\newpage
\section{Updates of the document}
\label{app:docupdate}
\begin{itemize}
\item version 0.2 to 1.0
	\begin{itemize}
    \item include implementation part in section 3
    \item fix underscore character in most places 
		\item include data model summary table for all fields in appendix A
		\item add a status column for each field M or R or O
		\item update tables \verb! em_domain! moved up ( minimal change)
	\end{itemize}
\item version 1.0 to 1.1
	\begin{itemize}
    \item section 5: re-write XMM SSC Obs/TAP service description 
		\item introduce use case shortly at beginning and point to appendix 
		\item moved data model summary table back to data model section 
	\end{itemize}
\end{itemize}
\end{document}

