Siav2RFC < IVOA

IVOA Web>IvoaVOTable>RfcPage>Siav2RFC (2015-12-18, MatthewGraham) (raw view)
<br /> <!--
      * Set ALLOWTOPICRENAME = IVOA.TWikiAdminGroup
-->
---++ SIA v2.0 Proposed Recommendation: Request for Comments

Public discussion page for the IVOA Simple Image Access (version 2.0) Proposed Recommendation.

Latest version of the IVOA SIAV2.0 can be found at:
   * http://www.ivoa.net/documents/SIA/20150730/index.html

---+++ Reference Interoperable Implementations

(Indicate here the links to at least two Reference Interoperable Implementations)
---++++ AMIGA SIAv2 Archive Prototype

The <a target="_blank" href="http://amiga.iaa.es:9000/">AMIGA SIAv2 Archive Prototype</a> provides discovery and access interfaces for two collections of radiointerferometric single-line emission datacubes of galaxies. The <a target="_blank" href="http://amiga.iaa.es:9000/bodega/">B0DEGA collection</a> is composed of 30 velocity datacubes of southern nearby galaxies obtained using the Submillimeter Array (SMA) for CO(2-1) emission of the circumnuclear regions (1 arcmin). The <a target="_blank" href="http://amiga.iaa.es:9000/whisp/">WHISP collection</a> provides 33 velocity datacubes of the more extended <a target="_blank" href="http://www.astron.nl/wow/testcode.php?project=3">public WHISP survey</a> observed for HI 21cm emission line. RESTful interfaces may be tested with HTML forms that were initially developed for debugging purposes.

Interfaces provided by AMIGA SIAv2 Archive Prototype:

   * <span style="background-color: transparent;">HTML </span><a target="_blank" href="http://amiga.iaa.es:9000/search/">http://amiga.iaa.es:9000/search/</a>
   * <span style="background-color: transparent;">SIAv2 Discovery Service (Endpoint + Debug Web Form)</span> 
      * <a target="_blank" href="http://amiga.iaa.es:9000/search/sia">http://amiga.iaa.es:9000/search/sia</a>
      * <a target="_blank" href="http://amiga.iaa.es:9000/form/SIA2" style="background-color: #ffffff;">http://amiga.iaa.es:9000/form/SIA2</a>
   * <span style="background-color: transparent;">Full Characterization Metadata Service (Endpoint + Debug Web Form)</span> 
      * <a target="_blank" href="http://amiga.iaa.es:9000/search/sia/metadata" style="background-color: #ffffff;">http://amiga.iaa.es:9000/search/sia/metadata</a>
      * <a target="_blank" href="http://amiga.iaa.es:9000/form/metadata" style="background-color: #ffffff;">http://amiga.iaa.es:9000/form/metadata</a>
   * <span style="background-color: transparent;">Adhoc cube extraction services (Endpoint + Debug Web Form)</span> 
      * <a target="_blank" href="http://amiga.iaa.es:9000/accessdata" style="background-color: #ffffff;">http://amiga.iaa.es:9000/accessdata</a>
      * <a target="_blank" href="http://amiga.iaa.es:9000/form/accessdata" style="background-color: #ffffff;">http://amiga.iaa.es:9000/form/accessdata</a>
   * <span style="background-color: transparent;">DataLink Service</span> 
      * <a target="_blank" href="http://amiga.iaa.es:9000/datalink">http://amiga.iaa.es:9000/datalink</a>
      * <a target="_blank" href="http://amiga.iaa.es:9000/form/datalink">http://amiga.iaa.es:9000/form/datalink</a>
DALI/VOSI compliant endpoints for /capabilities and /availability are also provided for SIAv2 Discovery Service (as well as for DataLink Service), the content of these responses (e.g. <em>standardID </em>values) may be modified to final agreed values accordingly.

Service descriptors for DataLink and Adhoc cube extraction services are provided in the SIAv2 discovery responses (as well as in the Full Characterization Metadata Service and DataLink responses)

AMIGA SIAv2 discovery service can accept other input params (very specific for searching in HTML form interface: LINE, VELOCITY and CHANWIDTH) which does not prevent it from beeing compliant with last SIAv2 draft. AMIGA <span style="background-color: transparent;">SIAv2 discovery service is compliant with recommendations for the use of COOSYS in VOTables. </span>

-- IVOA.JoseEnriqueRuiz - 2014-11-18
---++++ CADC implementation of the SIA-2.0

The CADC implementation has been updated to support all the query parameters in the PR document. Resources:

http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/sia/availability

http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/sia/capabilities

http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/sia/v2query

The landing page (http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/sia/) provides some example queries. The access_url in the output is currently a link to the CADC DataLink service (as described by the access_format value); this may change in the future to be a direct link to science data. We also include a DataLink service descriptor in the response (VOTable) document that can be used to generate calls to the DataLinks service using ID values from the table.

Implementation note: This implementation is really just a front-end for the CADC TA service. It generates an ADQL query on the ivoa.ObsCore table and submits that as a query job to TAP sync endpoint; the caller gets the outpu directly from the CADC TAP service. All of the code to implement such a front-end is available in the cadcSIA module at https://code.google.com/p/opencadc/ -- the code here can be used to parse query params, generate ADQL, and provide a "TAP-frontend resource" using the sync support in the cadcUWS library.

---++++ Client implementation

The last beta version of Aladin is able to query the test implementation servers and to ingest the query reponse.

-- IVOA.FrancoisBonnarel - 2015-06-08

---++++ DALServer implementation of the SIA-2.0

An implementation of the current RFC SIAV2 interface, including an accessData prototype, is in progress within the DALServer framework. Details will be posted here once this becomes available. The DALServer framework was developed with NVO and VAO resources within the US. Earlier (2014) prototype SIAV2 and accessData services, including multidimensional image data from a half dozen or so projects, can be viewed and exercised at <a href="http://vaosa-vm1.aoc.nrao.edu/ivoa-dal/siav2query.html">http://vaosa-vm1.aoc.nrao.edu/ivoa-dal/siav2query.html</a>

-- IVOA.DougTody - 2015-02-25

---++++ CASDA implementation of SIA-2.0

The CSIRO ASKAP Science Data Archive (CASDA) project has implemented a "minimal viable product" level SIA2, Datalink and SODA 'caravan' for retrieval of multi-dimensional FITS images and CASA format visibility files. See my presentation on the InteropOct2015DAL page for more detail. This was released on 7 Nov 2015. The next release will be in Feb 2016 which will include support for the full set of SIA2 parameters, async access and a cutout facility.

   * SIAv2 Discovery Service (Endpoint supporting POS and BAND queries) 
      * <a target="_blank" href="http://casda.csiro.au/casda_vo_tools/sia2/query">http://casda.csiro.au/casda_vo_tools/sia2/query</a>
-- IVOA.JamesDempsey - 2015-12-02
---+++ Validators

(If any, indicate here the links to Implementations Validators)

---+++ RFC Review Period: 2014-07-31 - 2014-09-09

<!-- REMOVE comment-out for TCG Review Period
---+++ TCG Review Period: TCG_start_date - TCG_end_date
-->
---
---

---++ Comments from the IVOA Community during RFC period: 2014-07-14 - 2014-09-09
---+++ Comments from José Enrique Ruiz

   * Abstract 
      * "SIA provides capabilities for image discovery and *access*, but capabilities for access are defined elsewhere." (?!)
   * 1.2.1. Simple Data Discovery 
      * "find data within a range of exposure (integration) time" I guess this requirement directly translates to searching data with flux in a specified range. I do care about integration time only if all observations have been made with the same instrumental set-up/sensitivity, which is not the case in broadcasted discovery queries.
      * <span style="background-color: transparent;">I can think on other requirements related to spectral radio velocity datacubes.</span> 
         * "find data observed for a specific spectral line"
         * "find data within a specified range of velocity for a specific spectral line"
   * 2. Resources 
      * In the table, what does the asterisk mean in no* ?
Changed to "reserved for future use". -- IVOA.PatrickDowler - 2014-10-20
   * <span style="background-color: transparent;">2.1.2 BAND</span> 
      * This is a major point. <span style="background-color: transparent;">I would use this param to search for datasets that have been *observed* in a specific range of wavelengths. The doc states that "values used in the BAND parameter are always assumed to (vacuum) wavelength with UNKNOWN reference position" I have to admit that I do not fully understand this sentence. Does it mean that I have to provide in my query *corrected from redshift values* instead of the actual frequency values in the instrumental set-up of the observational dataset? </span>If this is the case, this perfectly answers how could I search for spectral velocity radio datacubes that have been observed for a specific emission line (independently of its redshift) But what if I am searching for data within a specified range of velocity for a specific spectral line? My proposition is to keep BAND as *observed* wavelength, and add a param for LINE, and another one for VELOCITY.
   * 2.1.6 SPATRES 
      * I just draw your attention to a particular point in radio interferometry observations. There is an instrumental param called "Maximum Angular Scale" that provides the maximum angular scale structure that may be recoverable, which results in the fact the larger structures in the sky are "resolved out" and cannot be detected. A potential use case could be to find observations performed with a particular "Maximum Angular Scale" so we are sure we do not miss any structure in the sky smaller that this param. I guess this use case cannot be addressed with values for SPATRES.
   * <span style="background-color: transparent;">2.1.7 EXPTIME</span> 
      * As I said previously, I only see this param useful in the case I'm searching for observations that have been made with the same instrumental set-up/sensitivity, which in general it's not the case for broadcasted discovery queries.
   * <span style="background-color: transparent;">2.1.8 ID</span> 
      * Do we have a use case for broadcasted discovery using obs_publisher_did?
      * "should this parameter override the normal case-sensitive string equality comparison?" I would say yes.
   * COLLECTION, FACILITY, INSTRUMENT, TARGET 
      * I think it is important to reach an agreement on what to do wrt. case-sensitivity and strict-equality for these string-valued params. I personally see them more in the realm of use cases for search of services in the registry, with the exception of target that could be translated to coordinates by a name/coords look-up service like Sesame.
   * <span style="background-color: transparent;">2.1.16 SPECRP</span> 
      * In spectral velocity radio datacubes, resolving power (more used in optical wavelength observations) may have its analogue in the concept of "channel width" usually measured (again) in units of velocity.
   * <span style="background-color: transparent;">2.1.18 UPLOAD</span> 
      * It could be good to have an example of how to reference (from any input param) a column in a VOTable provided by UPLOAD (I guess only tabular data in VOTables allowed?).
Left undefined in 2.0. -- IVOA.PatrickDowler - 2014-10-20
   * <span style="background-color: transparent;">page 19</span> 
      * "Note that the {query} resource does not have to be named as shown in the accessURL(s) above; in fact, they could share the same accessURL since they also differ in the value of the REQUEST parameter..." Since REQUEST param has been removed, all this text should be revisioned, and {metadata} resource declared in VOSI capabilities..
All query parameters are defined in terms of the ObsCore data model and we now have params for all the interesting mandatory fields. Fixed the bit about REQUEST. -- IVOA.PatrickDowler - 2014-10-20

   * side point: logical operators 
      * It is said in the doc: "The constraints from multiple values of a parameter are combined with logical OR operator. The constraints from different parameters are combined with a logical AND operator." Given the number high number of params, should we start thinking (for a future version) on a mechanism to define logical operators for the combination of multi-valued params in multi-queries? I'm thinking on the potential use case of working with inclusion AND operators for a list of ranges in the POS params (overlapping in spatial dimension), for example. Moreover, this will help to combine different multi-valued params at the same time..
This is something that could be added in 2.1; for example some particiants are experimenting with passing query parameters in a document (e.g. json) and it would be simple enough to add logical operators and brackets to that. -- IVOA.PatrickDowler - 2014-10-20 -- IVOA.JoseEnriqueRuiz - 2014-10-08

<span style="color: #630000; font-size: 17px; font-weight: bold; line-height: 1em; background-color: transparent;">Comments from Mark Taylor</span>

Looks mostly well-written, reasonable and comprehensible to me (though I don't have a strong interest in the details of image access). Some small comments just to prove I read it:

   * One or two parts have not been prepared to the level of PR readiness: 
      * Sec 1.1: TODO include architecture diagram
      * Sec 1.3: "queryData and get-gory-details ... to be renamed in diagram before PR"
      * Sec 2.1.8 and 2.2.1: TBDs concerning ID case-sensitivity
      * The placeholders on the RFC page for implementations and validators are blank.
   * <em>Fixed. </em>-- IVOA.PatrickDowler - 2014-10-01

   * Minor suggestions/queries: 
      * Sec 2.1.1: Is POS longitude always in the range 0..360 degrees, or are services permitted/required to make sense of values in the range -180..0 as well? (possibly this is covered by one of the other referenced standards, but it might be good to add a short note here in any case).
      * Sec 2.4: "in fact, they could share the same accessURL since they also differ in the value of the REQUEST parameter that invokes their standard behaviour." I don't quite understand which resources are being referred to here as potential sharers, but in any case I *think* this comment is invalidated since the removal of the REQUEST parameter from the {query} resource. Probably this clause can just be removed.
      * It would be nice either in the Changes section or somewhere in the introduction to include a short comment on the difference between SIA v1 and v2.
   * _Added valid coordinate value range to text (longitude in [0,360], latitude in [-90,90] since we only support ICRS RA and DEC). Removed the bit about REQUEST. IVOA. -- PatrickDowler - 2014-10-01_
   * _SIA v1 and v2 are quite different so we will add a_ paragraph in the Introduction to explain the major changes. -- IVOA.PatrickDowler - 2014-10-05

   * Tiny typos: 
      * Sec 2: "will return detailed metadata can be used" - missing "that"?
      * Sec 2.1.4: "ther !ObsCore data model"
      * Sec 2.1.5: "The FOV parameter define the range(s)" -&gt; "defines"
   * <em>Fixed. </em>-- IVOA.PatrickDowler - 2014-10-01
-- IVOA.MarkTaylor - 2014-08-31

---+++ Comments from IVOA.FrancoisBonnarel

   1 1.4 POL (page 13)
POL=I POL=Q POL=U

means dataset contains "I OR Q OR U"

There is currently no way to say that a dataset contains "I AND Q AND U"

Currently implementations understand POL=I as "contains I" and not as "strictly equal to I".

We do not recommand to change this for version 2.0

---

I have other personal comments. One (2.2) is a real question

   * 1.1 I have an interrogation about "SIA defines a separate resource for accessing the complete ImageDM metadata for a single dataset. See below .

   * Page 7 we still have get-gory-details in the diagram and page 8 in the text. I think metadata is fine as a final name, although FullMetadata, or ImageMetadata could be more precise.
   * _Fixed name in diagram and text. -- IVOA.PatrickDowler - 2014-10-01_
* 2.1 (page 9) "all parameters for the {query} resource defined below are required.". ---&gt; I assume this means that all parameters must be interprateted by the service, but they may be absent from the URL built by the client/user. Otherwise ... Should we say that or is this obvious ?
   * _Clarified. -- IVOA.PatrickDowler - 2014-10-01_
* 2.1.3 (page 11) BAND Find data that includes 21 cm : this should be BAND = 0.21. BAND = 0.20/0.22 finds data which overlaps 0.20/0.22. It may be a range of 0.195/0.205 and in that case 0.21 will be outside ... (because we have said Intervals are intersecting )
   * _Fixed. -- IVOA.PatrickDowler - 2014-10-01_
* 2.1.17 (page 17)

I don't think FORMAT = .... can be used when access_format has a DataLINK value. How are we sure we know the content-type values for the considered dataset ?
   * _Removed FORMAT query parameter._ -- IVOA.PatrickDowler - 2014-10-01
* 2.2 {metadata} resource

The choice made there is to access metadata for one single dataset (except if we put several IDS values in the ID parameter)

In that case {metadata} is very close to {accessdata} resource not in the actual content but in the relationship to query. The output (one to several full descriptions) is then let open with a single format. I had in mind that a totally different mechanism could be usefull: Discovery with full metadata . In that case the {metadata}resource shares the same input parameters with {query}. But the ouptut table (or tables) should be consistent with full Image DM. It is a "long query" in some way ! In other words, WCS is directly available in the response obtained by the full parameter query.
   * _Postpone to 2.1 -- IVOA.PatrickDowler - 2014-10-01_
-- IVOA.FrancoisBonnarel - 2014-07-31

---+++ Answer by IVOA.DougTody -

Since the full metadata (WCS etc.) for a single dataset can be very large, and a discovery query may find many datasets, I think getMetadata should be a separate operation, restricted to a single image. Also, for reasons of complexity, only queryData should do discovery, and getMetadata and accessData should be limited to single datasets.

However, if we want basic WCS info in the discovery query response then that is possible as we probably do not need full image metadata. The minimal stuff we have proposed for ObsTAP 1.1, plus s_region, already comes close. If we were to add just a bit more (e.g. reference point and projection type) then it would suffice for the typical nsubarrays=1 use case.

-- IVOA.DougTody - 2014-07-31

---+++ Comments from IVOA.WalterLandry -2014-09-26

* <em> Reminder: the parameters you are critiquing are from the SIAv2 query capability. This capability is designed to query for datasets that match the specified conditions, so the parameters are inherently those that express the conditions and not (necessarily) those that describe some ideal (construct-able) data. </em>-- Answer by IVOA.PatrickDowler - 2014-09-29

* _Comment/response inline_ -- Answer by IVOA.PatrickDowler - 2014-09-29

Back in July, I sent a note to this list about some issues I had with SIA 2.0. Since then, we have implemented a synthetic image generation service for the Planck satellite. We tried to implement this is in a way consistent with SIA 2.0, but we had some difficulties.

1) BAND

The Planck satellite detector bands are all specified in GHz: 30, 44, 70, etc. These are nice, integer numbers. Mapping to wavelength leaves me with numbers that are not exactly representable in floating point. This means that every search has to give a range. It would be nice if I could specify the frequency instead of the wavelength. We ended up using the keyword FREQ in MHz, since I do not know of any astronomical observations of EM radiation that go lower than that.

* _We've had this argument so many times and it always comes own to "just pick something". In ObsCore-1.0 the em_min and em_max fields are wavelength in meters, so that is what SIA-2.0 query uses to query those fields. No matter what you pick for the standard, someone has to transform nice looking values into something with scientific notation... Implementing FREQ in addition is fine for you; requiring it in the standard is more work for all services; implementing FREQ instead means services are not compm&gt;atible or (at best) clients have to get/grok service capabilities before being able to call them._ -- Answer by IVOA.PatrickDowler - 2014-09-29

2) RANGE

As I mentioned in July, RANGE is prohibitively expensive for this data set. So we do not support it, will never support it, and I still think it should not be part of the spec.

* _I don't disagree with this; at best it is a convenience for making some common (large) polygons._ -- Answer by IVOA.PatrickDowler - 2014-09-29

3) POS

Since this is a synthetic image generation service, it would be nice to make rectangular images. The current SIA 2.0 spec has no great circle rectangles. The client has to construct the polygons themselves, which is non-trivial. Given the negative reaction I got to Box's last time, we ended up ditching SIA 2.0 for this entirely and using SIA 1 syntax: POS, SIZE, CFRAME, CDELT (though SPATRES would have been fine). I would really prefer a better mechanism than this.

* _Well, I basically grok what you re trying to do and it is just using a small bit of FTS WCS to describe what you want to get back. That seems to me to fit much better in an AccessData-ish service and not in a pure data discovery service. Since it seems to be driven from real data, there is obviously some part of the usage that would involve discovery... let's make sure to discuss this kind of usage in Banff next week._ -- Answer by IVOA.PatrickDowler - 2014-09-29

4) TARGET vs OBJECT

Why does SIA 2.0 use the TARGET parameter? OBJECT is an existing standard FITS convention.

* _The query result is ObsCore; in there the name of the field being constrained is target_name, hence TARGET._ -- Answer by IVOA.PatrickDowler - 2014-09-29

5) Syntax

Consider these issues:

a) In July, I highlighted a problem with the syntax of POS parameters. It requires spaces, which must be URL encoded or things silently break. Silent breakage is the worst kind of breakage.

* _Syntax requires encoding, yes. Even without syntax, parameter values must be encoded to be safe or strange things happen. How many times have I cursed the IAU naming convention that includes + sign? Lost count Must encode._ -- Answer by IVOA.PatrickDowler - 2014-09-29

b) Polygon searches use a straight list of numbers. It would be better to have a list of pairs to make typos more obvious.

* _It is possible to make syntax errors. More syntax solves it?_ -- Answer by IVOA.PatrickDowler - 2014-09-29

c) We need to be able to select multiple detectors at once, so we would like to have an array of strings.

* _Not sure I follow... you said synthetic but now are talking about multiple detectors. If the underlying data is some kind of mosaic camera then you have several choices on what constitutes a single ObsCore entity (been there, we can discuss off-line), but describing the complexity of 1 observation -&gt; N subarrays is not in the scope of ObsCore-1.0 so not in the scope of SIA-2.0 query. There is ObsCore-1.1 work underway, plus the ImageDM and consequent SIAv2 "metadata" capability for exposing it._ -- Answer by IVOA.PatrickDowler - 2014-09-29

d) There is no way to add arbitrary parameters. COORD was the way to do that in old versions of SIA 2.0. Now I have to use up a keyword and hope it does not accidentally conflict with new versions of the standard. This is not going to scale.

* _SELECT and COORD were never part of SIA-2.0 query; they were part of WD-AccessData-1.0 to show a way that SimDAL could be supported within that spec._ -- Answer by IVOA.PatrickDowler - 2014-09-29

e) We have a smart client doing searches on behalf of the user. In general, we would like to set arbitrary metadata that are not necessary for the search but convenient for the user.

* _You can always add custom fields to your ObsCore output. If they aren't really custom, but just in the optional fields of the appendix, then chosing standard names would be a good thing to do._ -- Answer by IVOA.PatrickDowler - 2014-09-29

This prompted me to use a more general syntax to express queries. Specifically, I used json5

https://github.com/aseemk/json5

It is an extension of JSON to make it friendlier to write. It is a strict superset of JSON and a strict subset of Javascript. So every valid JSON file is valid json5, and eval() will still work for those of you foolish enough to run it on unverified user input To be specific, a sample query would be

http://irsa.ipac.caltech.edu/cgi-bin/Planck_TOI/nph-planck_toi_sia?POS=[0.053,-0.062]&CFRAME='GAL'&ROTANG=90&SIZE=1&CDELT=0.05&FREQ=44000&ITERATIONS=20&INSTRUMENT=['24m','24s']&TIME=[[0,55300],[55500,Infinity]]&USER_METADATA={CLIENT:'IRSA Smart Client'}

Note that the service is not public yet, so this URL will not work for you yet.

Internally, every parameter is converted into a json5 element. So this would turn into the json5 document

{ POS:[0.053,-0.062], CFRAME:'GAL', ROTANG:90, SIZE:1, CDELT:0.05, FREQ:44000, ITERATIONS:20, INSTRUMENT:['24m','24s'], TIME:[[0,55300],[55500,Infinity]], USER_METADATA:{CLIENT:'IRSA Smart Client'} }

Modulo whitespace, this is just replacing '&' with ',' and '=' with ':'. We also support submitting a json5 document directly

http://irsa.ipac.caltech.edu/cgi-bin/Planck_TOI/nph-planck_toi_sia?{POS:[0.053,-0.062],CFRAME:'GAL',ROTANG:90,SIZE:1,CDELT:0.05,FREQ:44000,ITERATIONS:20,INSTRUMENT:['24m','24s'],TIME:[[0,55300],[55500,Infinity]],USER_METADATA:{CLIENT:'IRSA Smart Client'}}

* _This is interesting and I remember talking about json5 at the last interop. Hopefully we can see/discuss this further in Banff._ -- Answer by IVOA.PatrickDowler - 2014-09-29

On a side note, I have used JSON (not json5) as input for simulations in geology [1]. The SAMRAI Adaptive Mesh Refinement framework for massively parallel simulations [2] also uses a format that is almost indistinguishable [3] from json5 for input files. So I would claim that json5 would be able to cover any needs that SIMDAL would need in specifying model parameters.

Given all of the issues that I ran into, it is not clear to me that it would be a good idea to ratify SIA 2 as it is now. Whether or not you like the changes I made, there seem to be some major deficiencies that need to be addressed before the standard can be accepted.

[1] http://geodynamics.org/cig/software/gale [2] https://computation-rnd.llnl.gov/SAMRAI/index.php [3] You can separate elements in an array or object with newlines instead of commas. -- IVOA.WalterLandry 2014-09-26
---+++ Comments from José Enrique Ruiz 2014-10-28

Comments for [[http://www.ivoa.net/documents/SIA/20141024/index.html][SIAv2 PR 2014-10-24]]

   * <span style="background-color: transparent;">1.2.1. Simple Data Discovery. Last item </span><span style="background-color: transparent;">"find data within a range of exposure (integration) time"</span>
<span style="background-color: transparent;">I guess this requirement directly translates to searching data with </span><strong style="background-color: transparent;">flux </strong><span style="background-color: transparent;">in a specified range, i.e. finest details of an object detected. Integration time is not so useful when observations have been made with different instrumental set-up/sensitivities. I would prefer a scientific requirement taking exposure (integration) time as a threshold, then the delivered response would need subsequent filtering. </span>

   * <span style="background-color: transparent;">2.1 {query} resource. "All query parameters are multi-valued. The constraints from multiple values of a parameter are combined with a logical OR operator. The constraints from different parameters are combined with a logical AND operator." </span>
I think readers will appreciate a more developed paragraph, crearly saying that a single invocation of the service with several multivalued input params result in a complex discovery task issued from the multiple combination of the values among the parameters, so multi-queries are possible with a single invocation of the service.

   * 2.1 {query} resource. Last paragraph "The units for numeric values are are..."
Typo

   * 2.1.2 BAND. Last paragraph "<span style="background-color: transparent;">Energy values used in the BAND parameter are always assumed to be (vacuum) </span><span style="background-color: transparent;">wavelength in meters. The default interpretation is with an UNKNOWN reference </span><span style="background-color: transparent;">position [9]."</span>
More clarity would be appreciated. i.e. "Energy values used in the BAND parameter are always assumed to be <strong>observed </strong>wavelength in meters"

   * 2.1.4 POL. Last paragraph "Possible values for the POL parameter are defined in, and the POL parameter..."
Something is missing after "are defined in.." ?

   * 2.1.8 ID
I guess this param prevails over the whole rest. If a value for ID is provided, the rest of input params are dismissed?

   * 3.2 Errors. "If the requested format is VOTable..."
DALI RESPONSEFORMAT param is not present in the doc. I guess the resquested format could still be passed in this "not mandatory" param.

-- IVOA.JoseEnriqueRuiz - 2014-10-28

---+++ Answer by IVOA.FrancoisBonnarel

I go back to this discussion between Walter and Pat (see above). I think the point has been clarified at Banff and in the new version.

-- Walter said 3) POS

Since this is a synthetic image generation service, it would be nice to make rectangular images. The current SIA 2.0 spec has no great circle rectangles. The client has to construct the polygons themselves, which is non-trivial. Given the negative reaction I got to Box's last time, we ended up ditching SIA 2.0 for this entirely and using SIA 1 syntax: POS, SIZE, CFRAME, CDELT (though SPATRES would have been fine). I would really prefer a better mechanism than this.

* Well, I basically grok what you re trying to do and it is just using a small bit of FTS WCS to describe what you want to get back. That seems to me to fit much better in an AccessData-ish service and not in a pure data discovery service. Since it seems to be driven from real data, there is obviously some part of the usage that would involve discovery... let's make sure to discuss this kind of usage in Banff next week. -- Answer by PatrickDowler - 2014-09-29

** This has been clarified in the introduction (differences SIAV1 and SIAV2)

" The capabilities for dynamic access to image datasets are expanded in scope, but are separated from data discovery and download of whole image datasets. A separate "AccessData" specification currently under development will define the more advanced dynamic data access functionality. Automated virtual data generation and discovery (as in SIA- 1.0) is not currently supported but is being considered for a future version of SIA."

Actually, AccessData 1.0 allows to extract cutouts. Regridding, changes of WCS will be covered by AccessData 1.1. Discovery of virtual data in the query phase will be covered by SIAV2.1

---
---

---++ Comments from TCG member during the TCG Review Period: 2014-11-20- 2015-09-13

WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

---+++ TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham_ )

   * In Section 1.1, the text states that "<span style="background-color: transparent;">SIA services may be registered in an IVOA Registry using the SimpleDALRegExt [11] extension schema." If so, then SimpleDALRegExt should appear in the architecture diagram.</span>
* _OK. It's done_ -- IVOA.PatrickDowler 2015-06-08

   * For recommendation, there needs to be two interoperable reference implementations. Two reference implementations are described but interoperability is not shown.
* _This is done by implementing a client which query on both servers_ -- IVOA.FrancoisBonnarel 2015-06-08

   * For recommendation, there needs to be an implementation validator. Is there one?
* _This was dependant from the new version of the draft (exemple) _ -- IVOA.FrancoisBonnarel 2015-06-08

-- IVOA.SeverinGaudet - 2014-12-18
---+++ Applications Working Group ( _Pierre Fernique, Tom Donaldson_ )

   * The reading of this document is a little bit hard as it requires to consult several related documents already standardized or still in maturation (such as DALI, ObsCore, TAP, DataLink, AccessData,...). For a easier general understanding, we would suggest to add a full example with an SIA v2 query URL and a VOTable response.
* _This is done_ -- IVOA.FrancoisBonnarel 2015-06-08

   * The new section about the difference between SIA v1 and SIA v2 helps a lot. May be, it could be moved at the beginning of the introduction rather at the end.
* _It has been moved in this direction (but the very beginning was not appropriate)_

-- IVOA.PierreFernique - 2014-12-15
---+++ Data Access Layer Working Group ( _François Bonnarel, Marco Molinaro_ )

---+++ Data Model Working Group ( _Jesus Salgado, Omar Laurino_ )

Overall I found that this document goes

a long way in meeting its goal, though there are some areas that need some improvement. I hope my

feedback is helpful in meeting that goal.

*Reference Implementations.*

It looks like the reference implementations are both server implementations. In what sense are they interoperable? It would be good to have a client implementation, and probably there is one already, although it's not listed on the RFC page. This would show that a) the services are interoperable (with each other and with an independent client), and b) that the use cases are fulfilled.

* _There is now a client implementation within Aladin (see above)_ -- IVOA.FrancoisBonnarel 2015-06-08

*Examples*

The document does not present any examples that show possible fulfillments of the use cases (i.e. the triple: use case, example query, example response).

* _This is done now (see above)_ -- IVOA.FrancoisBonnarel 2015-06-08

*Syntax*

I agree with Walter that it is discouraging to see yet another syntax defined by the spec. This may limit uptake, as it increases the burden on both server and client implementors, and makes the spec less future-proof. I don't know if there is still room for reviewing this decision. Adopting an existing syntax would also make more advanced uses of the specification easier (including the ones declared as "future" features, in particular related to ImageDM, inputs based on complex metadata, or request-based data manipulation). The concrete risk is that the standard will need to define more than one syntax for simple and advanced cases falling into its scope.

* - two alternatives answers (to be discussed in Sesto) : The syntax is not fairly new. It is an adaptation for images to the SSA Syntax defined several years ago. having SSA and SIA syntax close together is a strong driver. -- IVOA.FrancoisBonnarel

Changing to the VOTable-consistent syntax would mostly resolve this issue, with cost of changing the parameter syntax and inroducing new spatial params _ IVOA.Patrick.Dowler 2015-06-08

*Mandatory features*

There has been strong feedback about some mandatory features that came from implementors during the RFC period (I am thinking about RANGE in particular). Would it be possible to make the problematic feature(s) optional, and maybe add some clarifications to the problematic keywords in the spec? As the original comment is actually deeper than that, I am also concerned that not taking the comment into account will result in more issues down the road.

* _It would be much easier to make spatial coordinate range query optional if it was a separate parameter rather than a separate keyword in the value of POS. In the RFC review my response was that RANGE could be dropped since it adds burden for services and only convenience for clients... but we never revisisted that in Banff since there were bigger issues... it isn't too late to drop spatial coordinate range_ -- IVOA.Patrick.Dowler 2015-06-08

*Response serialization (and relationship with Data Models).*

It seems that the document does not directly specify the serialization format for service responses, leaving this to referenced specs. This may be a major issue for implementors.

* _It is actually similar to ObsTAp response. There are now examples available._ -- IVOA.FrancoisBonnarel - 2015-06-09

The abstract claims that SIA is based upon the Image DM. However, later in the text, the relationship with ImageDM seems to be part of "future" updates while the spec explicitly states that SIA is based on ObsTAP. * _Hopefully this has been clarified now (see below)_ -- IVOA.FrancoisBonnarel - 2015-06-09

Notice that in general I would suggest to limit references to "future" work as much as possible, as such references can possibly be confusing. (see "what's in 2.0 and what is not" below).

* _We have taken this remark into account._ -- IVOA.FrancoisBonnarel - 2015-06-09

More specifically, when SIAv2 states (3.1 Successful Query): "The response [...] is a table with instances of the ObsCore data model [that] specifies all the (VOTable) field names, utypes, UCDs, and units to use in the response"

I believe it is referring to the specific "4 Implementation of ObsCore in a TAP service" section in the ObsTAP document, plus it assumes a VOTable serialization (why the parentheses btw?), as in section 2.9 of the TAP spec, although the TAP spec is not referenced. It takes too much work, in my opinion, to figure out what exactly the response should be, and the lack of examples does not help either. Obviously this is not an issue for those who have followed the discussions or implemented ObsTAP already, but we cannot assume the regular reader knows what the spec implies.

* _The paragraph has been modified for clarification._ -- IVOA.FrancoisBonnarel - 2015-06-09

The aforementioned statement in section 3.1 (assuming I correctly interpreted it) clashes with the different semantics of similar sentences in other sections, e.g. 1.1 The Role in the IVOA Architecture:

"The metadata returned [...] is defined by the ObsCore data model; this may be extended with additional metadata (columns) from ImageDM in the future" (more on the "in the future" part later).

ImageDM is abstract and serialization-agnostic, thus it does not, by itself, specify the VOTable field names, UCDs, etc. (This seems better suited for a DAL specification or a standardized serialization specification). For instance, the "ObsCore in TAP" specification limits the semantics of the DM (e.g. central coordinates must be in ICRS), as it is expected by a DAL spec, but again this does not apply to ImageDM.

ImageDM is agnostic on how a DAL protocol response specification (or any other serializations) may restrict its semantics and make a "column" out of one of the leaves in its hierarchy.

* _section 1.1 has been modified to clarify this point._ -- IVOA.FrancoisBonnarel - 2015-06-09

On the other hand, section 2 Resources is more clear by itself (except for the "future" part, again more on this later), but again the context is ambiguous:

"an optional {metadata} resource for obtaining [...] metadata conforming to the ImageDM."

How is this statement to be interpreted? As in what applies to "ObsCoreDM in ObsTAP", i.e. a flattened serialization in a specific format with specific semantics, or as in what applies to ImageDM , i.e. any valid serialization of the abstract concepts defined by the DM?

In this case the answer is probably "we don't know yet, as this is unspecified", which segue perfectly to the next comment.

* _The subsection describing metadata resource has been suppressed and the content partially moved to the introduction._ -- IVOA.FrancoisBonnarel - 2015-06-09

In conclusion, modulo the contents of the next comment, I would suggest that the document replaces the references to ObsCoreDM with less ambiguous phrasing, as proposed below.

In the abstract, I would suggest to remove the reference to ImageDM (unless I am missing something ImageDM will be used to support future versions).

* _This has been changed and hopefully clarified but not removed._ -- IVOA.FrancoisBonnarel - 2015-06-09

Assuming that a SIA v2.0 response is identical to an ObsTAP response (is this true?), in section 1.1 I would suggest to remove the reference to ImageDM and I would rephrase the reference to ObsCoreDM as follows:

"The metadata returned [...] is defined by the ObsCore data model components [7] and serialized according to the ObsTAP specification [ibid., section 4], and [missing reference to TAP, section 2.9].

* _Done_ -- IVOA.FrancoisBonnarel - 2015-06-09

In section 3.1, instead of:

"The response [...] is a table with instances of the ObsCore data model [that] specifies all the (VOTable) field names, utypes, UCDs, and units to use in the response"

I would suggest:

"The response is a table consistent with ObsTAP responses as described in [7, section 4] and [missing reference to TAP, section 2.9]. Examples are provided in Appendix &lt;X&gt;."

* _Changed along these lines_ -- IVOA.FrancoisBonnarel - 2015-06-09

*What is in 2.0 and what is not.*

It is not trivial for the reader to keep track of what is defined in 2.0 and what is left to later releases. It is indeed true that the text tries to specify it, but the references to next releases are dispersed in the document, which will also make a future review hard to implement. I would suggest to add a section with a title like "Notable Absences", or "Future support", to explicitly notify the reader of what was left out of this version and will be done in the future. This should also make future reviews easier to accomplish.

* _This has been integrated to the introduction._ -- IVOA.FrancoisBonnarel - 2015-06-09

Similarly, it is unclear which Use Cases are addressed by v2.0 and which are not. In some cases it is clearly stated that a use case is not even supposed to be covered by the spec. One may probably work that out by cross-matching the specified features with the use cases section, but that seems too much work for implementors just so they can figure out whether what they want to achieve is supported or not by the standard. As the list of use cases is supposed to be a list of the "motivating" drivers, it's probably fine to keep all of them in the same section, but I would suggest to clearly mention in the title whether they are covered in the 2.0 specs or not. If it is true that some use cases are not going to be covered eventually, I suggest those use cases to be removed.

* _This has been clarified in the "Scope and Related documents section"_ -- IVOA.FrancoisBonnarel - 2015-06-09

*Other, possibly minor, issues (I am marking the minor ones with (M)):*

(M) 1. SIA-2.0 specification will make use of features...

Why the future tense? Does it or does it not make use of those features?

* _Does. Fixed _ -- IVOA.FrancoisBonnarel - 2015-06-09

(M) 1.1 SIA is a data discovery and metadata service that works with other DAL services...

I would have been less surprised to read that SIA is a *protocol* with relationships with other protocols and standards. Applications need to implement the SIA standard from the client side as much as services need to implement it from the server side.

* _Fixed_ -- IVOA.FrancoisBonnarel - 2015-06-09

(M) 2.1 Repetition: "units for numeric values are *are* specified"

* _Fixed_ -- IVOA.FrancoisBonnarel - 2015-06-09

2.1 "multi-valued" parameters. I guess this refers to multiple values of the same GET/POST parameter. However, it is unclear what the syntax is here, and as far as I can tell the HTTP does not define a standard one. DALI seems to define "multiple values" as multiple occurrences of the "parameter=value" pair, e.g. in 3.1.3 (Multiple Values) and 3.2.5 (UPLOAD), while the "," syntax is left to separate values of the single occurrence of a parameter (e.g. UPLOAD in 3.2.5, where the table, uri pair is separated by commas). This is consistent with what is specified in section 2.1.4 (POL). SIAv2 should probably clarify the syntax for multiple values and maybe refer to DALI here.

* _Hopefully clarified_ -- IVOA.FrancoisBonnarel - 2015-06-09

(M) 2.1.4 POL. The specification here is redundant, as the syntax for multiple values was already defined in the 2.1 preamble (or it should have been), and does not apply to POL only. The sentence "possible values for the POL parameter are defined in [...]" does seems incomplete.

* _Reworded to make this clear_ -- IVOA.PatrickDowler - 2015-06-09

2.2 {metadata}. I am not sure I understand why this is useful. As the resource is not specified and left to future revisions this section seems out of place and I would suggest to remove it.

* _Information has been moved somewhere else. _ -- IVOA.FrancoisBonnarel - 2015-06-09

(M) As most of the query parameters description is boilerplate text, I would suggest to insert more information in the section titles. DataType and Multiplicity are probably enough, but maybe the corresponding ObsCore field name would be useful as well. In my experience this eases implementations a good deal, as most of the relevant information becomes readily available in the TOC.

(M) When referencing other specs (e.g. DALI) it would help a lot to have a specific section referenced, especially in the normative part.

* _I am somewhat reluctant to do that because I would prefer to reference things like DALI and ObsCore in a loose DALI-1.0 or later sort of way and putting section numbers in just breaks at some point. If someone can't find the appropriate bit in DALI the the DALI doc needs improvement as a base reference doc_ -- IVOA.PatrickDowler - 2015-06-09

3.1 Successful Query. The spec should probably refer to the RESPONSEFORMAT parameter defined in DALI (Not necessarily in this section, but this section mentions the response format, so I am bringing this up here). This may also be true for VERSION and possibly other standard parameters from DALI. The spec does redefine standard pieces from DALI, sometimes unchanged (e.g. 2.3 Availability), so it seems appropriate to name all the parts of DALI that should be implemented faithfully with DALI. This is probably just boilerplate, but the kind of boilerplate that helps implementors by guiding them in tracking the relevant sections in different documents.

* _RESPONSEFORMAT sentence has been added with reference to DALI _ -- IVOA.FrancoisBonnarel - 2015-06-09

3.1.1 Related Service Metadata. Again, this section seems to specify what the document claimed is left for future versions (e.g. {metadata}).

* _{metadata} information has been heavily modified elsewhere. should be sufficient _ -- IVOA.FrancoisBonnarel - 2015-06-09

-- IVOA.OmarLaurino - 2014-12-31
---+++ Grid & Web Services Working Group ( _André Schaaff, Brian Major_ )

It would be nice to know, briefly and upfront, what the differences are between ImageDM and ObsCore. There is a suggestion that ObsCore is a more abstract data model in section 1.4, but apart from that it is difficult to understand why there are the two models and what purpose each serves. This, perhaps, could be clarified in a terminology section after the abstract?

* _Modifications made along the lines stressed by the DM WG should help. _ -- IVOA.FrancoisBonnarel - 2015-06-09

An overall easy to read document except for (as Pierre notes above) for the required knowledge of other documents.

* _The example should allow to avoid that._ -- IVOA.FrancoisBonnarel - 2015-06-09

-- IVOA.BrianMajor - 2014-12-23
---+++ Registry Working Group ( _Markus Demleitner, Pierre Le Sidaner_ )

In the following comments, what we've marked with (*) are things we feel must be addressed one way or the other before the document can pass.

General: We miss language on having custom parameters on SIAv2 services, something that was possible in SIAv1 -- and frequently necessary, too. Relatedly, the DALI metadata response (e.g., MAXREC=0) needs to be specified. (*)

We suggest that even for the standard parameters, means for clients to discover what, if any, values of them can be expected to match.

* _a MAXREC section has been added_ -- IVOA.PatrickDowler - 2015-06-09

We note the absence of a validator and a client implementation. For something as use-case driven SIAv2, a client implementation really is indispensable. Also, as we understand SIAv2 was positioned as "something easier to implement for non-IVOA folks than TAP+Obscore", a solid validator for these external implementors should really be provided. (*)

* _The answer to this can be found above (Basically the answer is yes)_ -- IVOA.FrancoisBonnarel - 2015-06-09

For readability, we would prefer if parameter descriptions were grouped in a physically meaningful way (band and specrp together, time, timeres, exposure_time together, etc).

* _Not done yet._ -- IVOA.FrancoisBonnarel - 2015-06-09

1 Introduction: "While the data model is general..." -- We suppose the "data model" here means whatever data model is implied by the protocol response. If that is true, that should be made clear here so as avoid confusion with a future Image DM.

* _I think relationship with ObsCore and Image DM have been clarified now._ -- IVOA.FrancoisBonnarel - 2015-06-09

1.2 Motivating Use Cases: Stylistically, it would be preferable if there were less "probably", "may", etc in that section. Also, as 1.2.4 isn't actually part of SIA2, wouldn't it be better to not have them as two of four motivating use cases, regardless of what follows in 1.3?

* _Ok for "may". For the second part the answer is no because the documents and protocol are separated for convenience but the use cases may actually be inseparable._ -- IVOA.FrancoisBonnarel - 2015-06-09

2.1. {query} resource. As in Datalink, I'd like some guaranteed way to get from the URL of a {query} resource to the capabilities endpoint, e.g., by saying slashes are forbidden in {query}. (*)

* _ It means the {query} resource has to be a sibling of the VOSI resources. I have added this restriction on naming to section 2. _ -- IVOA.PatrickDowler - 2015-06-09

2.1. UNKNOWN frames in BAND and TIME. I believe what's meant here is that the reference positions of the incoming data should be used as-is. It should be put like this in the spec. The prose about UNKNOWN reference position is not helpful -- how to I interpret data as being computed for "UNKNOWN"?. Either way, the document would profit from an analysis on why leaving the reference position open in a discovery protocol is acceptable and doesn't conflict with any requirements (in particular for BAND, that's not obvious to us)

* _Yes it means that it should be used as-is and is assumed to be the same as the internal one_ -- IVOA.PatrickDowler 2015-06-09

2.1.1 POS. It might be wise to stress even here that all values are decimal.

* _Done_ -- IVOA.FrancoisBonnarel - 2015-06-09

2.1.3 TIME. Is it *really* necessary to allow both MJD and ISO time stamps? What additional use cases are enabled by this? It seems that allowing both is a bad compromise on the backs of service implementors, needlessly requiring a certain amount of guessing in parameter interpretation. As SIAv2 is design-- IVOA.FrancoisBonnarel - 2015-06-09ed as a relatively thin layer on top of obscore, we suggest going for MJD exclusively.

* _still in discussion among authors : ISO, is for compatibility with SSA I Guess. MJD is natural with OBsCore. So I would say no for this demand_ -- IVOA.FrancoisBonnarel - 2015-06-09

* _I actually agree with this ; the backwards compatibility comes at a price because allowing both MJD and ISO means that the param datatype can't be double_ -- IVOA.PatrickDowler - 2015-06-09

* _I actually agree with this ; the backwards compatibility comes at a price because allowing both MJD and ISO means that the param datatype can't be double_ IVOA.FrancoisBonnarel - 2015-06-09

2.1.8 ID. The text should say: "Note that IVORNs MUST be compared case-insensitively. As PubDIDs in the VO generally are IVORNs, implementations will usually have to use case-insensitive comparisons here."

* _Done_ IVOA.FrancoisBonnarel - 2015-06-09

2.1.9....2.1.12, 2.1.14, 2.1.17: Do the string-valued parameters make any sense at all without a means to discover what values might be valid on a given service? Also, experience shows that people will want to use some sort of regular expression/wildcard scheme on these, and that we'll have implementations accepting them, leading to interoperability problems when no sane standard exists. It would be preferable if some provision were made for both discovery and patterns on enumerated/string-valued parameters. (*)

* _Since I had to add a little MAXREC subsection, I also included there the usefulness of providing a self-describing service descriptor (example 4.6 in Datalink) where once could list values for the string params. This provides a mechanism to do this and indirectly to describe all supported params including custom ones (exactly why we added that example to datalink). I can try to implement this and then be able to provide a good MAXREC=0 output example, or at worst work up an example by hand next week_ -- IVOA.PatrickDowler - 2015-06-09

2.1.9 COLLECTION. Obscore has here: "In practice this is not a very precisely defined field" -- this, of course, makes discoverability even more important. Or SIAv2 should grasp the chance to define best practices: maybe IVORNs for the collections after all since there's a Resource type for them now? Or maybe propose some list of common short names of the data collections, perhaps maintained on the IVOA wiki?

* _I don't think we can really do something valuable right now. may be reconsidered with version 2.1_ -- IVOA.FrancoisBonnarel - 2015-06-09

2.1.10 FACILITY. Obscore has here "For combined observations stemming from multiple facilities the name may contain a list of comma separated strings..." SIAv2 *really* should comment about what that means for interpreting FACILITY queries. Also, having some sort of vocabulary or at least a means of discovering values supported by the service appear preconditions for having this work.

* _same answer than for COLLECTION_ -- IVOA.FrancoisBonnarel - 2015-06-09

2.1.12 DTYPE. Is there any strong reason not to include other, fairly image-like data like visibilities (ok, their edges in space may be a bit fuzzy, but strictly speaking you have stray like on CCDs, too) or event lists?

* _Visibilities and event list can be discovered using ObsTap. Or via DataLink if they are related to the cube. Do we really want to discover with SIAV2 ? AccessData will be difficult for such Datasets. I would say no for this version_ -- IVOA.FrancoisBonnarel - 2015-06-09

2.1.17 FORMAT. Previous incarnations of FORMAT included lots of funky magic terms for ("compliant", "native", "image"...). Just to be sure: There's no intention of have them any more?

* _No there isn't_ -- IVOA.FrancoisBonnarel - 2015-06-09

2.1.18 UPLOAD. The text appears a bit too vague for us. Couldn't this just be "The DALI UPLOAD parameter is not used by this version of SIA. The use case of uploading lists of coordinates is covered by the multiple-valued parameter values." or so?

* _Done_ -- IVOA.FrancoisBonnarel - 2015-06-09

2.2. {metadata} resource. If we've learned one thing from VO standards development it is that predictions are tricky, in particular when the concern the future. Non-normative or not, this section may have to do fairly little with what will actually happen, and so more likely than not will only confuse future readers. And we don't think it helps a lot right now.

* _This has been moved (see above)_ -- IVOA.FrancoisBonnarel - 2015-06-09

2.4 Capabilities, example. The Registry-approved canonical namespace prefix for VODataService is vs, not vod. The example document also needs the prefix declaration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" in addition to what's already there. (*)

* _Done_ -- IVOA.FrancoisBonnarel - 2015-06-09

2.4 Capabilities. Do we really intend to drop the special SIA capability? If not, we'd be happy to assist in drawing up a Registry extension for SIAv2. It seems very clear to us that maintaining these in a separate document (as for SimpleDALRegExt) is not a sustainable thing, and so we propose that *if* extra capability metadata is intended for SIAv2, it should go into this document. If it is dropped, we would strongly suggest some other means (examples endpoint?) is defined that allows validators to discover a set of parameters that should return data.

* _I know that one thing that is missing with SimpleDALregExt is the ability to say which version of ObsCore the query interface uses. If ObsCore-1.1 is accepted at the interop and we can tweak SIA_2.0 to depend on it (or later) &ndash; and I think we should - then I think this won't be an problem for quite some time_ -- IVOA.PatrickDowler - 2015-06-09

3.1. Successful Query. We had the thing with the "recommended HTTP Response Headers" in Datalink already, and we think the table and associated text should go here, too, for identical reasons.

3.1. Successful Query. We suggest that the reference to Obscore might be a bit too terse for implementors. Anyway, there should be an example response in the document. This could be together with a sample query string, which might also be helpful. (*)

* _Done_ -- IVOA.FrancoisBonnarel - 2015-06-09

3.2. Error "If the requested format is VOTable, the error document should be VOTable..." -- this "should" is bad from a validation perspective. This must be a MUST of some sort, or our validators cannot flag services that don't do that. It's clear that exceptional errors can result in other error document formats, but such services are then just broken, and we should flag them. Please have text here that gives us the legal basis to do so. (*)

* _An attempt was made_ -- IVOA.FrancoisBonnarel - 2015-06-09

---+++ Semantics Working Group ( _Norman Gray, Mireille Louys_ )

I think the document offers a clear and precise list of features to be present to implement the SIAv2 protocol. Units and UCD semantic tags are defined via the other re-used specifications, namely ObsCore, VOTable, so there are no conflicting issues concerning the Semantic WG approval.

An implementation section could be added directly as appendix of the specification , in order to keep their examples content close to the description of the specifiactions.

* _We added the examples section_ -- IVOA.FrancoisBonnarel - 2015-06-09

A few minor comments and suggestions :
   * In order to help the reader , it would be valuable to add the referenced section inside each referenced documents:
from SIAv2 doc --&gt; to ...

sec. 1.1 --&gt; VODataservice, sec.? --&gt; SimpleDalRegext, sec.?

sec 2 , sec 2.1 --&gt; DALI sync, sec. nb ??

sec 2.1.18 --&gt; DALI upload, sec nb?

etc.

* _See answer by Pat above (Data Model section)_ -- IVOA.FrancoisBonnarel - 2015-06-09

   * p14. Typos on last sentence in page 14? "Pol parameter are defined in , and the POL parameter"....?
* _ Clarified the text_ -- IVOA.FrancoisBonnarel - 2015-06-09

   * Figure p8, could have a caption to explain more precisely how Siav2 combines/chains the various capabilities offered by the DAL layer.The current layout, with Datalink in the middle might suggest that accessing data files needs to use the Datalink box , which is not true. More text would just help and avoid people to interpret the figure as a data flow .
* _Don't agree because what is considered is AccessData and a DataLink of some sort may be needed to go from SIAV2 and ObsTap to it._ -- IVOA.FrancoisBonnarel - 2015-06-09

   * p9 . An updated version of VOTable is used : be more precise , like 2.1 and above ... or 1.3
* _Done_ -- IVOA.FrancoisBonnarel - 2015-06-09

... where ObsCore specified standard UCD ( *ref UCD1+*) and Utype ...

"VO image data model " --&gt; VO Image data model

* _OK_ -- IVOA.FrancoisBonnarel - 2015-06-09

standardised / standardized

* _OK_ -- IVOA.FrancoisBonnarel - 2015-06-09

   * p11. middle of the page : "The units for numeric values are <em><strong>are </strong></em>specified in the ...".
* _OK_ -- IVOA.FrancoisBonnarel - 2015-06-09

I would emphasized : " specified <strong>in the next sections</strong>" ...

   * p.20 "if the differ" --&gt; "if they differ"
* _Done_ -- IVOA.FrancoisBonnarel - 2015-06-09

-- IVOA.MireilleLouys - 2015-03-16

---+++ Education Interest Group ( _Massimo Ramella, Sudhanshu Barway_ )

---+++ Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick_ )

---+++ Data Curation & Preservation Interest Group ( _Françoise Genova_ )

---+++ Knowledge Discovery in Databases Interest Group ( _George Djorgovski_ )

---+++ Theory Interest Group ( _Franck Le Petit, Rick Wagner_ )

---+++ Standards and Processes Committee ( _Françoise Genova_)

--&gt;
---

<br /> <!--
      * Set ALLOWTOPICRENAME = IVOA.TWikiAdminGroup
-->

---++ Comments from TCG member during the TCG Review Period extension: 2015-09-14- 2015-09-30

WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

---+++ TCG Chair & Vice Chair ( _Matthew Graham, Pat Dowler_ )

This document is approved.

-- IVOA.MatthewGraham - 2015-12-18
---+++ Applications Working Group ( _Pierre Fernique, Tom Donaldson_ )

We approve this document.

Just to fix a problem p19: there is a empty section 2.1.18

* _Done_ --IVOA.FrancoisBonnarel - 2015-10-28

-- IVOA.PierreFernique - 2015-10-26
---+++ Data Access Layer Working Group ( _François Bonnarel, Marco Molinaro_ )

Approved as a version 2.0

-- IVOA.FrancoisBonnarel 2015-09-16

---+++ Data Model Working Group ( _Mark Cresitello, Laurent Michel_ )

I believe this version addresses the previous comments by the DM working group of 2014-12-31.

A couple general notes of my own:
   * Referenced documents 
      * reference for DataLink is the PR from 2014, should be updated to the 2015 rec.
* _Done_ -- IVOA.FrancoisBonnarel 2015-10-28

   * 
      * reference for ObsCore is the 1.0 Rec from 2011. (conflict with Example.. see below)
   * Section 4: Examples 
      * The UTypes used in the example match the ObsCore-1.1 WD, not the 1.0 REC. For SIAV2 to go to REC, I'd expect these would have to conform to the REC (1.0) version. Is this a problem? A future minor version (2.1) would want to base on the ObsCore-1.1 Utypes which have some notable differences (not just additions). example: the utypes for dataproduct_type, s_resolution, t_min, t_max, t_resolution, em_min, em_max in the example all change between 1.0 and 1.1
* _about the above. I can still update the document by replacing with Obscore-1.0 as soon as possible. But this will also miss some image and cube specific dimensions. and will be obsolete examples as soon as Obscore 1.1 will become a recommendation. Advice?_ -- IVOA.FrancoisBonnarel 2015-10-29

** Pat explained the viewpoint regarding the utypes, which is these were "all accepted to be bugs in ObsCore-1.0; if ObsCore-1.1 was not imminent then an errata document would have been issued to fix them". This resolves my concern with the examples. -- IVOA.MarkCresitelloDittmar - 2015-11-09

* AccessData
   * 
      * there are several references to the content of, and current state of the AccessData spec, which is a WD. This seems rather odd, but I did not notice any direct dependence on AccessData for the content of SIAV2.. is that correct?
* _SIAV2.0 in itself specifies a pure DataDiscovery service. It is not dependant on any other specification. But as a part of a full AccesData chain it may be combined with other specs such as DataLink and AcessData. I changed a curious reference to the AccessData "draft" at some point._ -- IVOA.FrancoisBonnarel 2015-10-29

   * Section 2.1 
      * I note that there has already been discussion about NaN vs Inf on the dal list, so no need to elaborate here.
* _The replacement of NaN by -Inf,+Inf has been done_ -- IVOA.FrancoisBonnarel 2015-10-29

   * Minor tidbits 
      * I agree with the earlier comments about limiting mention of 'future' work and capabilities in the core document content. I do not consider it an issue, just stating an opinion.
* _There was a forgotten mention of future version in section 2. I removed it. All mentions of future work aree now concentrated in the introduction section and specially in a dedicated subsection. The Hawai interop defined a roadmap for DataLink, AccessData and SIAV2 with a simple first step and future additions. It's important to find a trace of this somewhere in the document in such a way that readers can understand why the spec is sometimes incomplete_ -- IVOA.FrancoisBonnarel 2015-10-29

   * 
      * Section 1.2, last bullet "SIAV2 will support the new DataLink" should not be future tense.
* _Done_ -- IVOA.FrancoisBonnarel 2015-10-28

   * 
      * Section 1.5, 'by tseparate capabilities' --&gt; 'by *the* separate capabilities'
* _Done_ -- IVOA.FrancoisBonnarel 2015-10-28

   * 
      * Section 1.5, the phrasing of the last sentence could maybe be improved.. the ObsCore and Cube models do not provide capabilities, they define a framework which the DataLink, TAP, SIA specifications can use to define capabilities.
* _Done_ -- IVOA.FrancoisBonnarel 2015-10-28

   * 
      * Empty Sections 2.1.10 and 2.1.18 (already mentioned by Markus)
* _Done_ -- IVOA.FrancoisBonnarel 2015-10-28

   * 
      * Section 5.1: 'so that patameters' --&gt; s/b 'parameters'
* _Done_ -- IVOA.FrancoisBonnarel 2015-10-28

   * 
      * Section 5.1: 'Moved section compring' --&gt; s/b 'comparing'
* _Done_ -- IVOA.FrancoisBonnarel 2015-10-28

   * 
      * Section 5.2: 'SIS-1.0' --&gt; s/b 'SIA-1.0'
* _Done_ -- IVOA.FrancoisBonnarel 2015-10-28

-- IVOA.MarkCresitelloDittmar - 2015-09-30

   * Section 2.1 (P13) 
      * The open-ended interval (-infinity,infinity) (all values) is / : You may specify that this interval select all values which are set (not NaN).
* _I think that with the change from NaN to -Inf +Inf this good remark is no more appropriate_ -- IVOA.FrancoisBonnarel 2015-10-29

   * Section 2.2.18 MAXREC 
      * The second part of the section (about Datalink) should be in a specific subsection.
* _Done_ -- IVOA.FrancoisBonnarel 2015-10-28

-- IVOA.LaurentMichel - 2015-10-07

I approve this document. -- IVOA.MarkCresitelloDittmar - 2015-11-09

---+++ Grid & Web Services Working Group ( _Brian Major, Giuliano Taffoni_ )

I approve this document.

-- IVOA.BrianMajor - 2015-10-26
---+++ Registry Working Group ( _Markus Demleitner, Theresa Dower_ )

Two minor new points:

   * There's empty sections 2.1.10 and 2.1.18.
* _Dropped. Thanks_ -- IVOA.FrancoisBonnarel 2015-10-29

   * Regardless of wher Identifiers 2.0 goes, we'd like to have the term IVOID whereever there's now IVORN in this document.
* _Fixed_ --IVOA.FrancoisBonnarel 2015-10-29

Not related to our previous review but to the standards process as such, we believe that Walter Landry's public RFC comment on dropping RANGE has not been sufficiently addressed.

* _The discussion with Walter is old. (since March 2014). RANGE is a very simple criterium for end user. Very natural and prettu usefull around the low "latitudes". If your internal sytem implementation is not able to deal with such ranges because it works with great circles you can either modify by using your great circle search as a first pruning out thing and then you refine the search. OR you just provide the pruning and nobody will harm you. This is discovery right ? But only discovery._ IVOA.FrancoisBonnarel - 2015-10-09

---+++ 

Finally, points from our original review we believe are not entirely satisfactorily addressed:

(1) François said in his response to our questions on COLLECTION and FACILITY that there's probably nothing valuable that can be done with the two parameters at this point. Given the potential confusion (e.g., commas in obscore's facility_name), I think the logical thing to do is remove them ("Underspecified is worse than unspecified").

* _There was a confusion in undersatnding my answer. Probably my fault. I thought it would be difficult to get a significant valuable VO List from scratch before implementation. As soon as you know (by chance or whatever other ad hoc reason) some COLLECTION and FACILITY values in a service these parameters are allready usefull for discovering datasets in a service. However It would be very intersting to get these list and standradize at the IVOA level after some experience and feddback and I added a slot in the "current and future support" section_ -- IVOA.FrancoisBonnarel 2015-10-29

(2) I'm still not happy with the language in DPTYPE. If we really want to specify that SIAv2 services only return image and cube-typed things, that needs to be said somewhere else, not en passant in the description of a parameter.

* _the restriction was allready stated several times in the introduction. I made it explicitly more clear in the "current and future support" section_ -- IVOA.FrancoisBonnarel 2015-10-29

Markus is all for dropping this restriction entirely. Let operators decide what they want to return. On François' objection that they might be trouble in server side data processing services ("AccessData"): From Markus' experience in writing such services, you'll have to be very flexible in their parameters anyway, even with your run-of-the-mill array-typed data sets. Markus happens to believe the proposed three-factor semantics is powerful enough. And either way for SIAv2 itself that's a secondary consideration.

* _I think it will be interesting to have visibilities and event list in a future version. A cutout on visibility is really not trivial at then moment. How to provide a POS cutout ? Do we want to put this in the AccessData thing today ? and a cutout on u,v coordinates values (spatial frequencies) should be done. How ? For event list the recent discussion in Obscore show it is not easy to define relationship between the event list concept and the sparse cube concept._ -- IVOA.FrancoisBonnarel 2015-10-29

(3) We're still missing the facility to let validators find out parameters that should return values (the old testQuery thing). It's too late to address this properly now, but we in Registry would be happy to lift the standardId locks in StandardsRegExt if DAL asked for that. In that case, you could say here something like:

_For SIAv2, it is recommended to use sia:SimpleImageAccess-typed capabilities as soon as their standardIDs are no longer fixed to the SIAv1 value._

We'd promise to fix SimpleDALRegExt as quickly as possible. Without a testQuery, it's really hard to build automated validators. We believe we really want those.

* _I think I agree with that. Where should I put this ? In capabilities section ?_ -- IVOA.FrancoisBonnarel 2015-10-29

* _Has been now added to VOSI capability section_ -- IVOA.FrancoisBonnarel 2015-11-01

---+++ Semantics Working Group ( _Mireille Louys, Alberto Accomazi_ )

A few typos need to be fixed as far as Semantics is concerned :
   * p. 23: VOTable example uses unit=&rdquo;sec&rdquo; for TIME quantities, while VOUnits would recommand unit=&rdquo;s&rdquo;.
   * p.28 Mjd=55708 --&gt; MJD=...
* _Corrections Done_ -- IVOA.FrancoisBonnarel 2015-11-01

__Section 2.1.10 ID__

mentions IVOIds, but there is no reference to the definition of an IVOA ID nor to the "IVOA identifiers specification".

* _Reference to IVOids specification has been added_ -- IVOA.FrancoisBonnarel 2015-11-01

I think this would improve readibility for people starting to implement VO protocols.

Otherwise I think the specification is clear and compatible to VO defined Semantics standards.

-- IVOA.MireilleLouys - 2015-10-31
---+++ Operations Working Group ( _Tom McGlynn, Mark Taylor_)

While we recognize that it is important to make progress on this standard, the lack of a validator is troubling. We strongly urge that validation of the new SIA capabilities be treated as a very high priority. DAL standards are particularly appropriate for validation since they have well defined inputs and outputs. Validation helps assure robust services and provision of a validator makes it easier for standard implementers to develop their services by making it easy to identify subtle bugs and possible misinterpretations of the standard.

* _This question has been adressed during the september TCG teleconf. The "VO Paris data center" group was working on that again since a couple of weeks_ --IVOA.FrancoisBonnarel 2015-10-29

---+++ Education Interest Group ( _Massimo Ramella, Sudhanshu Barway_ )

---+++ Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick_ )

---+++ Data Curation & Preservation Interest Group ( _Françoise Genova_ )

---+++ Knowledge Discovery in Databases Interest Group ( _George Djorgovski_ )

---+++ Theory Interest Group ( _Franck Le Petit, Carlos Rodrigo_ )

---+++ Standards and Processes Committee ( _Françoise Genova_)

---+++ 

--&gt;
---

<br /> <!--
      * Set ALLOWTOPICRENAME = IVOA.TWikiAdminGroup
-->
Topic revision: r48 - 2015-12-18 - MatthewGraham
IVOA
Log in or Register
IVOA.net
Wiki Home
WebChanges
WebTopicList
WebStatistics
Twiki Meta & Help
IVOA
Know
Main
Sandbox
TWiki
TWiki intro
TWiki tutorial
User registration
Notify me
Working Groups
Interest Groups
Committees
Stds&Procs
www.ivoa.net
Documents
Events
Members
XML Schema