TWiki> IVOA Web>SimpleDALRegExt10RFC (revision 14)EditAttach

Request for Comment: SimpleDALRegExt v1.0

This document serves as the RFC center for the Proposed Recommendation entitled "Describing Simple Data Access Services, Version 1.0". The version reviewed during the RFC can be found at http://www.ivoa.net/Documents/SimpleDALRegExt/20120517/. The version revised for TCG Review can be found at http://www.ivoa.net/Documents/SimpleDALRegExt/20121116/.

RFC Review period: May 23, 2012 - June 27, 2012 Completed
TCG Review period: 4 Dec 2012 - 4 Jan 2013
Exec Approved for REC:

Attention TCG: if you prefer printing out a PDF version for review, consider this abbreviated version. It omits the inclusion of the appendices listing the full XML schemas, resulting in 50% fewer pages.

To add a comment on the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the Resource Registry mailing list, registry@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

Notes on Implementations and Validaters

The SimpleDALRegExt page includes several compliant sample VOResource instances that use the extension schemas defined in the spec.. Instances can be validated with an XML Schema validater such as the one provided with Junx (which is based on Xerxes).

Previous versions of the extensions for ConeSearch, SIA, and SSA, which the PR versions are backward-compatible with, have been in production use by IVOA-compliant registries for several years.

Comments from the community

Comments from MarkTaylor

A couple of comments about the schema:

  • Data types: Several quantities are declared as xs:int type, for instance the sia:SIACapRestriction maxFileSize and maxRecords elements. Is this intentional? xs:int is a 32-bit quantity, which at least for maxFileSize seems like an unnecessary/unwise restriction. xs:positiveInteger or xs:unsignedLong might be more appropriate. Elsewhere xs:float and xs:double are used in different places, e.g. siaSkyPos uses xs:double and sia:SkySize uses xs:float. Is there a rationale for which floating point type is used where?
    • RayPlante responds:I agree with moving to xs:positiveInteger and xs:double for these numeric items; will change in next version. I note that this can be considered a backward-compatible change.
  • Required elements: Most of the limit-type elements (maxRecords, maxFileSize) are declared as required. Is this a good idea? Some services may have no hard restriction on the number or size of data returned, or may not know at capability specification time what such limits are. In that case services will have to make something up, e.g. 999999999, which may make clients think that a particular limit exists when it doesn't really. Any reason not to make most of these optional? slap:maxRecords seems particularly anomalous: the table in sec 3.4.2 notes that this element is required, and also that if it is not specified there is no predefined hard limit.
    • *RayPlante responds:* If I were to try to recall the rational, I would bet it would be historical. Nevertheless, I agree, based on our experience now, that making these optional is better than 9999999 for the reasons Mark describe; will change in next version. I note that this can be considered a backward-compatible change.
And some minor comments on the text, mostly typos:
  • Abstract: "describe a services" -> "describe a service"
  • Abstract: fundemental -> fundamental
  • Abstract: Stadard -> Standard
  • Sec. 1: "... form a query ... and send it to all konwn services which support that protocol". For cone search at least this is rather a bad idea and not to be encouraged (tools like AstroScope provided, then deprecated this facility), since it submits far too many queries to be scalable or useful. This point could be rephrased to discuss sending the same queries to a moderately-sized group of compliant services matching some additional selection criteria.
  • Sec. 1: "can be search in one query" -> "can be searched in one query"
  • Sec. 1: "supports one of the protocol" -> "supports using one of the protocols"
  • Fig. 1 caption: "SimpleDALExt" -> "SimpleDALRegExt" (twice)
  • Sec. 2: Missing close parenthesis following "extension schema [VDS]"
  • Sec. 2: "unambiguosly" -> "unambiguously"
  • Sec. 3: "each extension schema have" -> "each extension schema has"
  • Sec. 3: "fixes the value its standardID" -> "fixes the value of its standardID"
  • Sec 3.1.2: "cs:ConeSearch type is vr:Capability sub-type" -> "cs:ConeSearch type is a vr:Capability sub-type"
  • Sec 3.1.2: "The custom metadata that the sia:ConeSearch type provides" -> "... cs:ConeSearch ..."
  • Sec 3.1.2 metadata elements table: the Comments written under the testQuery element should be in the verbosity section instead
  • Sec 3.1.2.1: "at least matched record" -> "at least one matched record"
  • Sec 3.2.2: "cs:SIACapRestriction" -> "sia:SIACapRestriction"
  • Sec 3.2.2.2: "axes may not particularly parallel" -> "axes may not be particularly parallel"
  • Sec 3.2.2.2: "latitdude" -> "latitude"
  • Sec 3.3.2: "is vr:Capability sub-type" -> "is the vr:Capability sub-type"
  • Sec 3.3.3: "for compliance final SSA Recommendation" -> "for compliance with the final SSA Recommendation"
  • Sec 3.4.1: "The namespace prefix, sia" -> "The namespace prefix, slap"
  • Sec 3.4.2: "The sia:SimpleLineAccess type" -> "The slap:SimpleLineAccess type"
    • *RayPlante responds:* All good catches; will change.
-- MarkTaylor - 19 Jun 2012



Comments from TheresaDower

(pardon any major formatting issues; twiki is not being kind to my preview/editing attempts)

I will admit I am unsure how much this document is supposed to be an administrative collation of the existing standards and how much we intend to change each of them with an eye on backward compatibility. This may or may not be the stage at which to address inconsistencies between the standards, but I can't think of a better one. (It may also be well past time, as I note we've already gone over the RFC date but are requesting more feedback.)

  • For example, the Cone Search test query is required. It is not required for any of the other service types. ( Sec 3.2.1:) In practice, I have seen registered cone searches handle this by populating testQuery with bogus values, leading to non-responsive test queries. In practice, I've also seen validators fail non-Cone Search services 100% fullstop for having no optional testQuery.
    • *RayPlante responds:* This is good feedback based on experience; will make this optional
  • Likewise, SSA's ( Sec 3.3.2)) maxFileSize is optional, SIA's is required, and it's often populated with an intentionally meaningless value of 0.
    • *RayPlante responds:* Agreed; will change as per above response to MarkTaylor.
  • I know there have been and continue to be good reasons for encouraging data/service providers to provide more and more metadata as the standards grow in complexity, but if there is a time to look back and make a consolidated standards document more internally consistent between standards, this is it.
    • *RayPlante responds:* We'll have to strike a balance here. We can certainly provide consistency with regard to what's required as recommended in these comments. However, an important aim of this standard has been expediency: to provide endorsement for practices that are already in place. This implies preferring backward compatible changes. Further consolidation that might be useful (like refactoring to collect together common metadata) would open up questions that are not easily/quickly resolved (like, is ConeSearch still encouraged, and do we need to tweak the underlying protocol documents.). If we pass on these larger questions, some inelegance will inevitably remain.
  • These comments are obviously related to MarkTaylor's general ones about the creeping number of required fields. Many of these values are difficult or impossible for service providers to calculate for some catalogs and thus many of these fields get filled with bogus information, which is less than useless to validators and client software. (If I have understood RandyThompson's offline comments correctly, Sec 3.2.2: sia:MaxImageSize/Extent stand out as a particularly difficult example for large catalogs with variably rotated images from different instruments.) Even if we don't want to tackle every field in each standard right now, it would be good to look at some of these and either make them optional or provide guideline default values in more cases than we already do.
-- TheresaDower - 23 Jul 2012



Comments from Randy Thompson

Comments and questions (as sent to Ray last month)

* General:

  • Why is "test query" required for cone search services and optional for others? I think many people would like this required for all services.
  • Why is maxFileSize required for SIA and optional for SSA?
* SIAP:

  • maxImageSize - I see MaxImageFileSize listed in 6.2 of the SIAP document and later in Appendix B as MaxImageSize. Are these all the same value? I guess this is really a SIAP document issue.
    • *RayPlante responds:* The error is in the SIA document, specifically in Appendix B. The purpose of this appendix is to show the registration terms identified in the SIA 6.2 get represented in VOResource; the names in column 1 of the table in the appendix do not necessarily match what's actually used in as shown in column 3. The error is that "MaxFileSize" in column 1 should read "MaxFileSize". (Admittedly, they all probably should say "MaxImageFileSize".)

  • ImageFormats - the MIME types for provided image formats is listed as required for registering SIAP services (section 6.2 of SIAP V1.0 doc.) but I don't see it mentioned in this document.
    • *RayPlante responds:* Formats should be specified via the VODataService element, "format". (SIA Appendix B should say this but does not.) I'll add a note to this effect.

  • ImageSize 3.2.2 - I've always assumed that maxImageSize was simply the maximum image size in pixels along the x and y axis, so registering a service describing an archive of 10,000, 500 x 200 pixel images would just have an image size of 500 x 200. However the definition seems to require that the sizes be given along the axes of RA and Dec which means the orientation of the images becomes significant and the calculation more involved. Plus the orientation may vary from image to image making the definition more confusing. For example, if the orientation for the above image archive can vary such that the long axes is parallel to the RA axis in one observation and parallel to the Dec axes in another, what is the value of maxImageSize?
Related to this, it might help to reword this sentence in 3.2.2.2:
   "The sky coordinate axes may not particularly parallel with the image axes
   (e.g. due to rotation or projection effects); in this case, the
   longitudinal side should be considered the side of the image that it
   cuts the smallest angle with."
   

Why not just define maxImageSize as the x,y dimensions in pixels of the largest image that can be requested? I suspect this is what most data providers do anyway.

    • *RayPlante responds:* I think the text you quoted reflects my struggle trying to interpret the original text from SIA S6.2, which is terse. I agree that your interpretation--where which number is long and which is lat is unimportant--makes more sense. The best fix would be to not use "long" and "lat", which would be a non-backward compatible change.

      I note that non-backward compatitble changes are not out of scope. In the coming months registry maintainers will need to upgrade many of these records due to changes in VODataService; we could fold in changes in these schemas at the same time. I should send a note to the list about this.

  • RA,Dec vs Long,Lat - When a metadata value specifically requires RA and Dec, I'm confused why they're labelled as Long and Lat. It would certainly make sense if galactic or ecliptic coordinates were allowed, but if RA and Dec are required, why not call them RA and Dec?
    • *RayPlante responds:* made moot by previous response.

  • maxImageExtent,maxImageSize,maxFileSize,maxRecords 3.2.2 - I'm surprised to see these all listed as required even though its stated as such in the SIAP document (section 6.2). I believe these have never been enforced in the VAO.
    • *RayPlante responds:* see previous response re: required metadata.
* SSAP:

* supportedFrame 3.3.2 - this is listed as required, but I don't recall seeing it used in any registered services. Perhaps it is considered optional if only ICRS is supported?

    • *RayPlante responds:* This point was a point of discussion at the Pune interop and it was concluded that making this required made things a bit more future proof.
* ProtoSpectralAccess 3.3.3 - We have several registered services supporting both SSAP V1.0 and its predecessor (frequently referred to as version 0.4). We list them under separate capabilities within one resource as was discussed with VO staff a few years ago. In section 3.3.3 it states:

"A VOResource resource description must not include both a
    ssap:SimpleSpectralAccess capability and a
    ssap:ProtoSpectralAccess capability that describe the same
    service base URL, as given by the <interface>'S<accessURL>."

I'm not sure what

"<interface>'S<accessURL>"

means, but does this sentence imply our services are no longer allowed?

    • *RayPlante responds:* That snippet might be hard to parse: it means the accessURL within the interface element; I'll re-word. The intention is not to disallow services of type ProtoSpectralAccess. This stipulation just reflects the fact that you cannot have an SSAP service, as identified by its accessURL, that is simultaneously compliant with both versions.
-- RandyThompson - 05 Aug 2012




Comments from TCG members during the TCG Review Period: ?? ???? 2012 - ?? ???? 2012

WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

TCG Chair & Vice Chair ( _Christophe Arviset, Séverin Gaudet)

Applications Working Group ( _Mark Taylor, Enrique Solano)

Data Access Layer Working Group ( Patrick Dowler, Mike Fitzpatrick)

Data Model Working Group ( _Jesus Salgado, Omar Laurino)

Grid & Web Services Working Group ( Andreas Wicenec, Andre Schaaff )

Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)

Semantics Working Group ( _Sebastien Derriere, Norman Gray)

VOEvent Working Group ( _Matthew Graham, John Swinbank)

Data Curation & Preservation Interest Group ( Alberto Accomazzi)

Knowledge Discovery in Databases Interest Group ( Giuseppe Longo)

Theory Interest Group ( _Herve Wozniak, Franck Le Petit)

Standards and Processes Committee ( Francoise Genova)


Edit | Attach | Watch | Print version | History: r28 | r16 < r15 < r14 < r13 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r14 - 2012-12-04 - RayPlante
 
This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback