| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Changed: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
< < | ObsLocTAP protocol | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
> > | ObsLocTAP project page | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Changed: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
< < | The Observation Locator Table Access Protocol (ObsLocTAP) specification describes the necessary data model elements and the access protocol to discover metadata about observations for a given Astronomical Observatory through a uniform interface within the VO framework. The used data model makes reference to IVOA Observation Data Model elements (Mireille, et al., 2017), removing the ones associated to datasets access, as these elements are not available yet for future observations that are planned, scheduled, performed but not archived. In this way, present standard is focused on the access to metadata related to the planning of a certain observatory, more than in the access to the scientific data products of a certain observation. Also, the data model described in the present standard will be focused on metadata discovery, useful for multiwavelength coordination observations. However, and in order to prevent coupling between ObsCore and ObsLocTAP standards, ObsLocTAP defines its own data model.The access protocol will be expressed by the implementation of an IVOA Table Access Protocol (TAP) (Patrick, Guy, & Doug, 2010), that, as described in the use cases, allows a powerful discovery process and, also, the implementation of the use cases proposed in the present specification. ObsLocTAPservices could be registered using a TAP VOResource Extension (Demleitner, Dowler, Plante, Rixon, & Taylor, 2015) providing the relevant ObsLocTAP capability. Exact details of the VOResource Extension will be defined in another appropriate standard definition.In the case of a simple use case, planned observations of a certain target can be discovered via a TAP and ADQL (Iņaki, et al., 2008) query that searches for sky regions (or FOV) that overlap a certain sky coordinate. Adding a certain radius to the sky coordinates and/or filtering by instrument identifier could expand this simple use case. As described in TAP, the service will return a list of performed observations of the given target and also the future planned and/or scheduled observations in VOTable format or, optionally, in other tabular serialization format, e.g. JSON. The information about the planned observation, in the near future, will be subject to changes due to any operational issue (re-planning, ToO alerts, etc...). Thus, an implementation of the service may support additional search parameters (some of which may be custom to that particular service) to more finely control the selection of the observation information. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
> > | This page summarises how to implement an ObsLocTAP service. The full description of this protocol can be found under the IVOA (International Virtual Observatory Alliance) domain at: | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Changed: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
< < | ObsLocTAP Datamodel
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
> > | http://www.ivoa.net/documents/ObsLocTAP/index.html | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Added: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
> > | The Observation Locator Table Access Protocol (ObsLocTAP henceforth) specifies in a standard format the services to retrieve information about planned, scheduled and performed observations. By sharing this information, the scientific community can monitor the planning activities of the observatories, prepare coordinated observations or to allow the preparation of better proposals from the scientific point of view.
Following the approach of the newer IVOA protocols, ObsLocTAP is a Tabular Access Protocol (TAP) service. This service exposes a database table using a query language close to SQL but with astronomical operators that allow better exploration of astronomical resources. This language called Astronomical Data Query Language (ADQL)
http://www.ivoa.net/documents/REC/ADQL/ADQL-20081030.pdf
is a standard for the virtual observatory.
The implementation of a full compatible TAP service from scratch is cumbersome. However, there are a set of toolkits that allow the creation of this services without major knowledge of the underlying standards and very fast. We will describe how to implement a TAP service using a very well toolkit called taptuto but there are others. Implementors could take a look into the different alternatives and give them a try.
Other very well known toolkits are:
ObsLocTAP COMPONENTSThree basic components are needed to implement an ObsLocTAP service
DATABASE INSTALLATIONThere are several pages where we can find how to install a PosgreSQL server. In most of the cases, just following the instructions of https://www.postgresql.org/download/ should be enough to have a PosgreSQL server up and running. In fact, many of the different server images have already a PosgreSQL instance that can be used. Although PosgreSQL v10 contains some interesting features, v9.6 can guarantee at this stage operational performance and compatibility with extensions. There are multiple tutorials to install PosgreSQL depending on the operating system on the web. It is recommendable to implement periodic dumps and backups in order to restore an operational database in case of failure (see https://www.postgresql.org/docs/9.6/backup.html). Also, it is recommendable to use a special account (e.g. postgres) for the database instance. Once you have your PosgreSQL instance up and running, there is one spatial extension that would be needed for the TAP service instance. There are other spatial indexing extensions but the TAP service instance used in point 2 is making use of pgsphere. Installation of the extension following the instructions at http://pgsphere.github.io/download.html is more or less transparent.CREATION OF OBSLOCTAP COMPLAINT TABLE (DATABASE)Create a database and a table with the following structure:CREATE SCHEMA ivoa; CREATE TABLE ivoa.obsplan ( t_planning double precision NOT NULL, target_name character varying, obs_id character varying NOT NULL, obs_collection character varying NOT NULL, s_ra double precision, s_dec double precision, s_fov double precision, s_region scircle, s_resolution double precision, t_min double precision NOT NULL, t_max double precision NOT NULL, t_exptime double precision, t_resolution double precision, em_min double precision, em_max double precision, em_res_power double precision, o_ucd character varying NOT NULL, pol_states character varying, pol_xel integer, facility_name character varying NOT NULL, instrument_name character varying NOT NULL, obs_release_date timestamp, t_plan_exptime double precision NOT NULL, category character varying, priority integer, execution_status character varying NOT NULL ); CREATE INDEX i_obsplan_fov ON ivoa.obsplan USING gist (s_region);Please notice that the fov is defined as a pgsphere object. Other types different than polygon could be used (see https://pgsphere.github.io/doc/types.html) (a typical type used for footprints is circle) but two different types of pgsphere objects cannot be combined in the same database object. The last create index sentence defines how to create a gist index on the pgsphere object to increase the performance. In this case, we have defined s_region as a scircle pgsphere object. Typical value would be a spoly. However, there is not an easy way to mix pgsphere circles and polygons in the same column as they do not have a generic data type. One possible solution is to store everything as spoly and convert the circles in approximated polygons. The output of a select on s_region is typically a string representation of the Field of View. The conversion from a pgsphere output to a STC-S output when this column is used on a SELECT condition is typically done by the TAP server. For example, polygon is defined as: Polygon J2000 158.408362 57.781849 158.501991 57.812066 158.558573 57.762136 158.465026 57.73196 where the points of the polygon is defined after J2000 in pairs of ra dec values. Whenever there is some uncertainty on the final pointing, a good option is to define the stc_s representation just as a circle, with the central position and the radius defined as: Circle J200 <s_ra> <s_dec> <s_fov> where the values of s_ra, s_dec and s_fov are the other columns in the obsplan table. Whenever the Field of View is better defined, an upgrade from the circle to something more detailed could be done. See more info at http://ivoa.net/documents/STC-S/ It would be a good idea to populate this table with some dummy rows for testing during next steps. If you are familiar with docker, we have created a docker instance with posgreSQL, pgsphere and the ivoa.obsplan table with some dummy fields for testing. https://hub.docker.com/r/jsalgadodocker/postgres9.5-pgsphere-obsplan TAP SERVICE INSTANCEThere are some frameworks to be used to create a TAP instance. We recommend to use the one created by Gregory Mantelet at CDS as it is quite well documented and it has been updated to the latests ADQL (Astronomical Data Query Language) changes. Installation instructions can be found here: http://cdsportal.u-strasbg.fr/taptuto/ Some notes:
PUBLICATION OF TABLES THROUGH TAP SERVERTAP_SCHEMA should contain the description of the published table(s) through the TAP service. Using the SQL script provided in the previous framework (or by other method), the characterization of the table in the TAP_SCHEMA should contain the following information: | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Deleted: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
< < | Possible prototypes | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Changed: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
< < | Some observatories have offered the possibility to collaborate on this standard by the creation of prototypes | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
> > | TESTING THE SYSTEM | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Changed: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
< < |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
> > | Once the service is up and running, some queries can be used to test the system. Some examples are included in the specification but, as an example, we present some queries used for the ObsLocTAP Integral Observatory service. The service URL for this service is: | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Deleted: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
< < |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Changed: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
< < | Comments | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
> > | https://iladev.esac.esa.int/tap-dev/tap/sync? | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Added: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
> > |
For other services, this service URL will be different although the rest of parameters (REQUEST, PHASE, LANG, QUERY) are the typical TAP service one so these will be common for the rest of services.
6.1. SEARCH OF OBSERVATIONS ON A CERTAIN TIME RANGEADQL Query:SELECT o.s_ra, o.s_dec, o.target_name, o.t_min, o.t_max FROM ivoa.obsplan AS o WHERE o.t_max > 56783 ORDER BY o.t_minService invocation: https://iladev.esac.esa.int/tap-dev/tap/sync?REQUEST=doQuery&PHASE=RUN&LANG=ADQL&QUERY=select+o.s_ra%2C+o.s_dec%2C+o.target_name%2C++o.t_min%2C+o.t_max+from+ivoa.obsplan+as+o+where+o.t_max%3E56738+order+by+o.t_min 6SEARCH OF OBSERVATIONS ON A CERTAIN ASTRONOMICAL AREAADQL Query:SELECT * FROM ivoa.obsplan WHERE 1=INTERSECTS(s_region, CIRCLE('ICRS', 114.8251, 1.6179, 0.016666))Service invocation: https://iladev.esac.esa.int/tap-dev/tap/sync?REQUEST=doQuery&PHASE=RUN&LANG=ADQL&QUERY=SELECT%20%2A%20FROM%20ivoa.obsplan%20WHERE%201%3DINTERSECTS%28s_region%2C%20CIRCLE%28%27ICRS%27%2C%20114.8251%2C%201.6179%2C%200.016666%29%29 COMBINATION OF TIME AND GEOMETRICAL CONDITIONSADQL Query:SELECT * FROM ivoa.obsplan WHERE t_max < 58502 AND 1=INTERSECTS(s_region, CIRCLE('ICRS', 114.8251, 1.6179, 0.016666))Service invocation: https://iladev.esac.esa.int/tap-dev/tap/sync?REQUEST=doQuery&PHASE=RUN&LANG=ADQL&QUERY=SELECT%20%2A%20FROM%20ivoa.obsplan%20WHERE%20t_max%20%3C%2058502%20AND%201%3DINTERSECTS%28s_region%2C%20CIRCLE%28%27ICRS%27%2C%20114.8251%2C%201.6179%2C%200.016666%29%29 7. PLAYING WITH DOCKERSAlthough it is not recommended to use docker containers for an operational archive, we have defined two dockers containers for the DB and the TAP server that you can use to have a ObsLocTAP server up and running in minutes in you have the docker command line client installed. You just need to:docker pull jsalgadodocker/pgsphere-obsplan docker pull jsalgadodocker/tapserver docker network create --driver=bridge db-network docker image ls docker run -p 8080:8080 --net=db-network --name tap <tapserver_image_id> docker run -p 5432:5432 --net=db-network --name db <pgsphere-obsplan_image_id>You will have a TAP server with an obsplan table with some Integral records as test data running on your localhost. It could be tested by http://localhost:8080/tap/tap/ Also, you can connect to port 5432 of your localhost with a dabase client (e.g. pgadmin3) to see the records or to ingest your own ones. In order to make public your service, you can e.g. create a proxy pass redirect from an apache web server, use port 80 and open it to internet, etc. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ObsLocTAP Datamodel
Possible prototypesSome observatories have offered the possibility to collaborate on this standard by the creation of prototypes
Comments |