UserAgentUsage < IVOA

---+ UserAgentUsage

*NOTE: this content has been included in the !SoftID draft Note ([[https://github.com/msdemlei/softid]]), currently under review.*

-- IVOA.MarkTaylor - 2021-03-05

---++ Identifying operations-related service requests


---+++ Introduction

Client software may make requests to VO services for various reasons.
The most straightforward case is when a science user makes a query
in order to obtain data for scientific purposes.
However clients may also wish to connect for operational purposes,
for instance in order to perform service validation or monitoring.
If service providers are gathering statistics on service usage,
they may wish to distinguish these different classes of request.

---+++ User-Agent header standard usage

The HTTP =User-Agent= header may be used by clients to identify
their nature or origin.  The definiition and usage of this header
is described in
[[https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.43][RFC2616 section 14.43]],
with additional text on
syntax in sections 3.8 and 2.2.  The basic rule is that the content
of this field should consist of a sequence of tokens, where
each token is either a product name (with an optional version
indicator), or a free-text comment enclosed in parentheses.
Formally (BNF from RFC2616):
<pre>
   User-Agent      = "User-Agent" ":" 1*( product | comment )
   product         = token ["/" product-version]
   product-version = token
   comment         = "(" *( ctext | quoted-pair | comment ) ")"
   ctext           = &lt;any TEXT excluding "(" and ")"&gt;
</pre>
(the =quoted-pair= business here is a character escaping mechanism).

Additional rules/conventions are that more-significant
tokens should appear earlier in the sequence, and that the content
should be "short and to the point".

---+++ User-Agent field IVOA recommendations

The Operations IG endorses and encourages use of these rules
concerning the User-Agent header,
and adds a further convention, which does not conflict with the above rules:
clients whose primary purpose relates to operations within the VO *should* indicate that purpose
by including a comment token of the form
"<tt>(IVOA-<i>&lt;op-purpose&gt;</i> <i>&lt;optional-extra-text&gt;</i>)</tt>".

Suggested =op-purpose= values are currently:
   * =validate= : client is performing service validation,
                   e.g. to assess service conformance to standards
   * =monitor= : client is performing service monitoring,
                 e.g. to assess service availability or performance
   * =harvest= : client is harvesting records
                 e.g. to populate a registry or registry-like database
This list may evolve in the future (suggestions should be discussed on
the ops@ivoa.net mailing list).
Custom =op-purpose= values are permitted.
Case is not significant in =op-purpose= values or in its "<tt>IVOA-</tt>" prefix.

The =<optional-extra-text>= *may* (*should*?) be used to indicate
a URL at which more information about the client, or perhaps about
the results it is gathering from the current request, can be found.

Formally:
<pre>
   ivoa-comment  = "(IVOA-" op-purpose *( ctext | quoted-pair | comment ) ")"
   op-purpose   = "validate" | "monitor" | "harvest" | token
</pre>

Tokens of the form =ivoa-comment= *should not* appear in the user-agent
field if the request is a "normal" user science query.
There are obviously grey areas between operational and science requests;
this convention does not attempt to provide a rigid definition of
these categories.

This arrangement allows service operators to test in their logs for
User-agent values whose content matches the sequence "<tt>(IVOA-</tt>",
or perhaps
"<tt>(IVOA-validate</tt>", and adjust their usage statistics appropriately.
Note however that it is not feasible to force operational clients to
follow this convention, so service operators will still need to be
careful in analysing their usage statistics.

---+++ Examples

A science query from the STILTS =tapquery= TAP client might contain
the HTTP header
<pre>
   User-Agent: STILTS/3.1-4 Java/1.8.0_181
</pre>
while a query from the STILTS =taplint= TAP service validator might
contain the header
<pre>
   User-Agent: STILTS/3.1-4 (IVOA-Validate) Java/1.8.0_181
</pre>
or maybe
<pre>
   User-Agent: STILTS/3.1-4 (ivoa-validate http://validators.org/results) Java/1.8.0_181
</pre>

---+++ Implementations

Here is a list of operational clients that declare themselves using this convention. Please update this list if you know of such clients not listed here:
   * [[http://www.starlink.ac.uk/stilts/sun256/taplint.html][STILTS taplint]], as "<code>(IVOA-Validate)</code>" since version 3.3-3 (05/2019)


---+++ History

This topic was originally raised by Alberto Micol, and
[[http://wiki.ivoa.net/internal/IVOA/InterOpNov2018Ops/valid-id.pdf][discussed]]
in the Operations IG session at the November 2018 Interop meeting
in College Park.

This page was written up by Mark Taylor following that discussion to facilitate further consideration
by the community.  If there is agreement, it may make sense to write it up as an IVOA Note at some point.

-- IVOA.MarkTaylor - 2018-11-22
This topic: IVOA > WebHome > IvoaOps > UserAgentUsage
Topic revision: r5 - 2021-03-05 - MarkTaylor