(r9) DataLink11RFC < IVOA

IVOA Web>IvoaTCG>DataLink11RFC (revision 9) (raw view)~~EditAttach~~
---++ DataLink 1.1 Proposed Recommendation: Request for Comments

<br />%TOC{depth="3"}%
---+++ Introduction
---

DataLink describes the linking of data discovery metadata to access to the data itself, further detailed metadata, related resources, and to services that perform operations on the data.

The main changes in v1.1 are
   * Generalize by adding use cases for links to content other than data files
   * VOSI-availability and VOSI-capabilities endpoints are now optional
   * Service descriptors can include exampleURL and contentType param(s), as well as DESCRIPTION, name, etc...
   * Added optional link_auth and link_authorized to signal whether authentication is necessary to use the link
   * INFO element with standardID mandatory in {links} response
   * Added content_qualifier FIELD to inform on the nature of the link target
   * Added local_semantics to identify similar links in the same DataLink service for different IDs
   * Mechanisms to recognize {links} endpoints outside ObsCore

Latest version of DataLink can be found at:
   * https://www.ivoa.net/documents/DataLink/20230413/index.html
The GitHub repository for issues and source can be found at:
   * https://github.com/ivoa-std/DataLink
Detailed discussion towards 1-1 can also be found on this ivoa twiki page ( last update by FrancoisBonnarel - 2023-05-04):
   * https://wiki.ivoa.net/twiki/bin/view/IVOA/DataLink-1_0-Next

---+++ Reference Interoperable Implementations

---++++ Server side

GAVO implemented the following changes = content_qualifier and local_semantics, service descriptor additional features such as DESCRIPTION? name, content_type, etc....

As a matter of example, a couple of links response for various obscore/sia/ssa tables in GAVO server
   * *rosat.images* : http://dc.zah.uni-heidelberg.de/rosat/q/dl/dlmeta?ID=ivo%3a%2f%2forg.gavo.dc%2f~%3frosat%2fimage_data%2frda_4%2fwg400138p_n1_p1_r2_f2_p1%2frp400138n00_im3.fits.gz where some links have content_qualifier = #image, others have content_qualifier = #event or #timeseries
   * *lamost6.lrs* : http://dc.zah.uni-heidelberg.de/lamost6/q/sdl_lrs/dlmeta?ID=ivo%3a%2f%2forg.gavo.dc%2f~%3flamost6%2flrs%2f373006237 where some links have content_qualifier = #spectrum
   * *califadr3.cubes* : http://dc.zah.uni-heidelberg.de/califa/q3/dl/dlget?ID=ivo%3A//org.gavo.dc/~%3Fcalifa/datadr3/COMB/ARP220.COMB.rscube.fits where some links have content_qualifier = #cube
   * *ppakm31.maps* : http://dc.zah.uni-heidelberg.de/ppakm31/q/cdl/dlmeta?ID=ivo%3a%2f%2forg.gavo.dc%2f~%3fppakm31%2fdata%2fPPAK_M31_F5_cube.fits wher some links have local_semantics filled up and content_qualifier equals #image or #cube

All these are combined with various semantics or content_type values
---

CADC has implemented the following in https://ws.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/caom2ops:
   * INFO element with standardID in links response
   * new optional fields in links response: local_semantics (no content yet but can be populated with default vocab in most cases), content_qualifier (no content, not likely to use), link_auth, link_authorized
   * contentType param in service descriptors (where applicable)
IRIS image: https://ws.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/caom2ops/datalink?ID=ivo://cadc.nrc.ca/IRIS?f212h000/IRAS-25um shows link_auth=optional and link_authorized=true because one can authenticate but the data is public.

new CFHT data: anonymous use of https://ws.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/caom2ops/datalink?ID=ivo://cadc.nrc.ca/CFHT?2773629/2773629o shows link_auth=optional and link_authorized=false because the data is still proprietary and the caller is anonymous; if an authorized user makes the call they will see authorized=true. It's hard to demonstrate that for a general audience.

The core CADC implementation is available as a library (cadc-datalink-server) in MavenCentral with source code at https://github.com/opencadc/dal.git; the caom2-specific logic is available in a library (caom2-datalink-server) with source at https://github.com/opencadc/caom2service.git -- the core lib is also used in ALMA DataLink service but may not yet be released with the latest features.

---++++ Client side

TOPCAT v4.8-8 and later displays additional features in service descriptor and makes use of additional links table FIELD such as content_qualifier, authorization and local_semantics; The tool behavior is adapted to the content of these new FIELDS. For example Activation Actions suggested by topcat for the links not only depend on content_type but also content_qualifier, and local_semantics is used to guess which link a user is interested in based on previous selections.

AladinDesktop is going to adapt to those new features too. [[%ATTACHURL%/AladinDesktopDataLink1-1.png][(see prototype screenshot)]]

The CADC DownloadManager (https://github.com/opencadc/apps.git) includes a simple DataLink client class so it can resolve publisherID values into 1..* URLs for download; this code hasn't changed as a result of DataLink -1.1. The CADC AdvancedSearch web portal makes calls to the above caom2ops/datalink service to find previews and download info for each row (publisherID): it makes use of link_authorized to decide to display the download options (or not), which prevents users from selecting downloads/links when they are not authorized and the request will be rejected later.
---+++ Implementations Validators

The following validators are available for DataLink

   * [[http://www.starlink.ac.uk/stilts/sun256/datalinklint.html][datalinklint]] which is part of STILTS.  From STILTS v3.4-8 there is a =version= parameter that can be set to 1.0 or 1.1 (effectively defaults to v1.1)
   * [[https://dc.g-vo.org/shomydl/q/f/form][Show my DataLink ]] which is part of DaCHS

---
---

---++ Comments from the IVOA Community during RFC/TCG review period: 2023-04-21 - 2023-05-18

The comments from the TCG members during the RFC/TCG review should be included in the next section.

In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your Wiki Name so that authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.

Additional discussion about any of the comments or responses can be conducted on the WG mailing list. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document

   * Sample comment by TWiki.WikiName
      * Response (by TWiki.WikiName)

---
---

---++ Comments from TCG member during the RFC/TCG Review Period: 2023-04-21 - 2023-05-18

WG chairs or vice chairs must read the Document, provide comments if any (including on topics not directly linked to the Group matters) or indicate that they have no comment.

IG chairs or vice chairs are also encouraged to do the same, althought their inputs are not compulsory.

---+++ TCG Chair & Vice Chair

---+++ [[IvoaApplications][Applications Working Group]]

---+++ [[IvoaDAL][Data Access Layer Working Group]]

---+++ [[IvoaDataModel][Data Model Working Group]]

---+++ [[IvoaGridAndWebServices][Grid & Web Services Working Group]]

---+++ [[IvoaResReg][Registry Working Group]]

---+++ [[IvoaSemantics][Semantics Working Group]]

---+++ [[IvoaDCP][Data Curation & Preservation Interest Group]]

---+++ [[IvoaEducation][Education Interest Group]]

---+++ [[IvoaKDD][Knowledge Discovery Interest Group]]

---+++ [[IvoaOps][Operations Interest Group]]

I don't strictly speaking speak for Operations IG as of this week, but since I did most of the review before my term expired, I'll fill it in here; the TCG can decide whether this counts as an Ops endorsement or not.

As one of the authors I'm basically happy with this document, but I will draw attention to one or two issues.
   * Section 2.2 defines the standardID for this standard as =ivo://ivoa.net/std/DataLink#links-1.0=, followed by the comment _"Note this is applicable to endpoints following any version 1.* of the !DataLink standard, to avoid backward compatibility problems."_  In my opinion the backward compatibility problems are not sufficient to justify this choice, and the minor version should be reflected in this standard ID, i.e. it should be ="...#links-1.1"=. This has been discussed in the open github [[https://github.com/ivoa-std/DataLink/issues/96][Issue #96]], and other authors seem to agree.  A fix will require at least an update to the !StandardsRegExt record, and also changes in the document to places where the key is referenced, including Section 2.2 and, especially, Section 3.3.1 as well as related example text.  This change would amongst other things make it possible for validators to check which minor version they are supposed to be validating against.  PatDowler has volunteered to write a Pull Request addressing this issue, but I can have a go if he doesn't.
   * Section 3.3.1 REQUIRES an INFO defining a suitable standardID for links response tables.  The example shows such an INFO element as a child of the =RESOURCE/@type="results"= element, but it's not clear what restrictions there are on the location - does it have to go there, or can it be elsewhere in the VOTable?  This should be clarified.  If it's not required to be a child of the results resource, the example text in this section should probably be cut down.
   * Section 3.2.2: The final sentence says _"No other additional parameters or client handling are allowed."_  I don't understand what is meant here.  Should this sentence be removed?
   * As mentioned in [[https://github.com/ivoa-std/DataLink/issues/82][Issue #82]], the recommended MIME type =application/x-votable+xml;content=datalink= is using a content-type parameter for VOTable not endorsed in the VOTable standard, which is a bit questionable; but this is not new in this version of !DataLink and it will hopefully be addressed in VOTable 1.5, so turning a blind eye is probably OK.

   -- MarkTaylor - 2023-05-15

---+++ [[IvoaRadio][Radio Astronomy Interest Group]]

---+++ [[IvoaSS][Solar System Interest Group]]

---+++ [[IvoaTheory][Theory Interest Group]]

---+++ [[IvoaVOEvent][Time Domain Interest Group]]

---+++ [[IvoaStdsDocsProc][Standards and Processes Committee]]

---

---++ TCG Vote : 2023-05-19 - 2023-06-01

If you have minor comments (typos) on the last version of the document please indicate it in the Comments column of the table and post them in the TCG comments section above with the date.

| Group | Yes | No | Abstain | Comments |
| TCG | | | | |
| Apps | | | | |
| DAL | | | | |
| DM | | | | |
| GWS | | | | |
| Registry | | | | |
| Semantics | | | | |
| DCP | | | | |
| Edu | | | | |
| KDIG | | | | |
| Ops | | | | |
| Radio | | | | |
| SSIG | | | | |
| Theory | | | | |
| TD | | | | |
| &lt;nop&gt;StdProc | | | | |

---

<br /> &lt;!--<br />      * Set ALLOWTOPICRENAME = IVOA.TWikiAdminGroup<br />--&gt;
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
png	AladinDesktopDataLink1-1.png	r1	manage	1209.4 K	2023-04-19 - 16:15	FrancoisBonnarel
Topic revision: r9 - 2023-05-15 - MarkTaylor
IVOA
Log in or Register
IVOA.net
Wiki Home
WebChanges
WebTopicList
WebStatistics
Twiki Meta & Help
IVOA
Know
Main
Sandbox
TWiki
TWiki intro
TWiki tutorial
User registration
Notify me
Working Groups
Interest Groups
Time Domain
Committees
Stds&Procs
www.ivoa.net
Documents
Events
Members
XML Schema