IVOA Web>SimpleDALRegExt10RFC (revision 7)~~EditAttach~~

Request for Comment: SimpleDALRegExt v1.0

This document serves as the RFC center for the Proposed Recommendation entitled "Describing Simple Data Access Services, Version 1.0". The specification can be found at http://www.ivoa.net/Documents/SimpleDALRegExt/20120517/.

RFC Review period: May 23, 2012 - June 27, 2012
TCG Review period:
Exec Approved for REC:

To add a comment on the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the Resource Registry mailing list, registry@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Notes on Implementations and Validaters

The SimpleDALRegExt page includes several compliant sample VOResource instances that use the extension schemas defined in the spec.. Instances can be validated with an XML Schema validater such as the one provided with Junx (which is based on Xerxes).

Previous versions of the extensions for ConeSearch, SIA, and SSA, which the PR versions are backward-compatible with, have been in production use by IVOA-compliant registries for several years.

Comments from the community

Sample comment by WikiName
- Response (by WikiName)

Comments from MarkTaylor

A couple of comments about the schema:

Data types: Several quantities are declared as xs:int type, for instance the sia:SIACapRestriction maxFileSize and maxRecords elements. Is this intentional? xs:int is a 32-bit quantity, which at least for maxFileSize seems like an unnecessary/unwise restriction. xs:positiveInteger or xs:unsignedLong might be more appropriate. Elsewhere xs:float and xs:double are used in different places, e.g. siaSkyPos uses xs:double and sia:SkySize uses xs:float. Is there a rationale for which floating point type is used where?
Required elements: Most of the limit-type elements (maxRecords, maxFileSize) are declared as required. Is this a good idea? Some services may have no hard restriction on the number or size of data returned, or may not know at capability specification time what such limits are. In that case services will have to make something up, e.g. 999999999, which may make clients think that a particular limit exists when it doesn't really. Any reason not to make most of these optional? slap:maxRecords seems particularly anomalous: the table in sec 3.4.2 notes that this element is required, and also that if it is not specified there is no predefined hard limit.

And some minor comments on the text, mostly typos:

Abstract: "describe a services" -> "describe a service"
Abstract: fundemental -> fundamental
Abstract: Stadard -> Standard
Sec. 1: "... form a query ... and send it to all konwn services which support that protocol". For cone search at least this is rather a bad idea and not to be encouraged (tools like AstroScope provided, then deprecated this facility), since it submits far too many queries to be scalable or useful. This point could be rephrased to discuss sending the same queries to a moderately-sized group of compliant services matching some additional selection criteria.
Sec. 1: "can be search in one query" -> "can be searched in one query"
Sec. 1: "supports one of the protocol" -> "supports using one of the protocols"
Fig. 1 caption: "SimpleDALExt" -> "SimpleDALRegExt" (twice)
Sec. 2: Missing close parenthesis following "extension schema [VDS]"
Sec. 2: "unambiguosly" -> "unambiguously"
Sec. 3: "each extension schema have" -> "each extension schema has"
Sec. 3: "fixes the value its standardID" -> "fixes the value of its standardID"
Sec 3.1.2: "cs:ConeSearch type is vr:Capability sub-type" -> "cs:ConeSearch type is a vr:Capability sub-type"
Sec 3.1.2: "The custom metadata that the sia:ConeSearch type provides" -> "... cs:ConeSearch ..."
Sec 3.1.2 metadata elements table: the Comments written under the testQuery element should be in the verbosity section instead
Sec 3.1.2.1: "at least matched record" -> "at least one matched record"
Sec 3.2.2: "cs:SIACapRestriction" -> "sia:SIACapRestriction"
Sec 3.2.2.2: "axes may not particularly parallel" -> "axes may not be particularly parallel"
Sec 3.2.2.2: "latitdude" -> "latitude"
Sec 3.3.2: "is vr:Capability sub-type" -> "is the vr:Capability sub-type"
Sec 3.3.3: "for compliance final SSA Recommendation" -> "for compliance with the final SSA Recommendation"
Sec 3.4.1: "The namespace prefix, sia" -> "The namespace prefix, slap"
Sec 3.4.2: "The sia:SimpleLineAccess type" -> "The slap:SimpleLineAccess type"

-- MarkTaylor - 19 Jun 2012

Comments from TheresaDower

(pardon any major formatting issues; twiki is not being kind to my preview/editing attempts)

I will admit I am unsure how much this document is supposed to be an administrative collation of the existing standards and how much we intend to change each of them with an eye on backward compatibility. This may or may not be the stage at which to address inconsistencies between the standards, but I can't think of a better one. (It may also be well past time, as I note we've already gone over the RFC date but are requesting more feedback.)

* For example, the Cone Search test query is required. It is not required for any of the other service types. (Sec 3.2.1:) In practice, I have seen registered cone searches handle this by populating testQuery with bogus values, leading to non-responsive test queries. In practice, I've also seen validators fail non-Cone Search services 100% fullstop for having no optional testQuery. * Likewise, SSA's (Sec 3.3.2)) maxFileSize is optional, SIA's is required, and it's often populated with an intentionally meaningless value of 0. * I know there have been and continue to be good reasons for encouraging data/service providers to provide more and more metadata as the standards grow in complexity, but if there is a time to look back and make a consolidated standards document more internally consistent between standards, this is it.

* These comments are obviously related to MarkTaylor's general ones about the creeping number of required fields. Many of these values are difficult or impossible for service providers to calculate for some catalogs and thus many of these fields get filled with bogus information, which is less than useless to validators and client software. (If I have understood RandyThompson's offline comments correctly, Sec 3.2.2: sia:MaxImageSize/Extent stand out as a particularly difficult example for large catalogs with variably rotated images from different instruments.) Even if we don't want to tackle every field in each standard right now, it would be good to look at some of these and either make them optional or provide guideline default values in more cases than we already do.

-- TheresaDower - 23 Jul 2012

Comments from Randy Thompson

Comments and questions (as sent to Ray last month)

* General:

Why is "test query" required for cone search services and optional for others? I think many people would like this required for all services.
Why is maxFileSize required for SIA and optional for SSA?

* SIAP:

maxImageSize - I see MaxImageFileSize listed in 6.2 of the SIAP document and later in Appendix B as MaxImageSize. Are these all the same value? I guess this is really a SIAP document issue.

ImageFormats - the MIME types for provided image formats is listed as required for registering SIAP services (section 6.2 of SIAP V1.0 doc.) but I don't see it mentioned in this document.

ImageSize 3.2.2 - I've always assumed that maxImageSize was simply the maximum image size in pixels along the x and y axis, so registering a service describing an archive of 10,000, 500 x 200 pixel images would just have an image size of 500 x 200. However the definition seems to require that the sizes be given along the axes of RA and Dec which means the orientation of the images becomes significant and the calculation more involved. Plus the orientation may vary from image to image making the definition more confusing. For example, if the orientation for the above image archive can vary such that the long axes is parallel to the RA axis in one observation and parallel to the Dec axes in another, what is the value of maxImageSize?

Related to this, it might help to reword this sentence in 3.2.2.2:

   "The sky coordinate axes may not particularly parallel with the image axes
   (e.g. due to rotation or projection effects); in this case, the
   longitudinal side should be considered the side of the image that it
   cuts the smallest angle with."

Why not just define maxImageSize as the x,y dimensions in pixels of the largest image that can be requested? I suspect this is what most data providers do anyway.

RA,Dec vs Long,Lat - When a metadata value specifically requires RA and Dec, I'm confused why they're labelled as Long and Lat. It would certainly make sense if galactic or ecliptic coordinates were allowed, but if RA and Dec are required, why not call them RA and Dec?

maxImageExtent,maxImageSize,maxFileSize,maxRecords 3.2.2 - I'm surprised to see these all listed as required even though its stated as such in the SIAP document (section 6.2). I believe these have never been enforced in the VAO.

* SSAP:

* supportedFrame 3.3.2 - this is listed as required, but I don't recall seeing it used in any registered services. Perhaps it is considered optional if only ICRS is supported?

* ProtoSpectralAccess 3.3.3 - We have several registered services supporting both SSAP V1.0 and its predecessor (frequently referred to as version 0.4). We list them under separate capabilities within one resource as was discussed with VO staff a few years ago. In section 3.3.3 it states:

"A VOResource resource description must not include both a
    ssap:SimpleSpectralAccess capability and a
    ssap:ProtoSpectralAccess capability that describe the same
    service base URL, as given by the <interface>'S<accessURL>."

I'm not sure what