VOUnitsRFC < IVOA

IVOA Web>IvoaSemantics>VOUnitsRFC (2014-10-27, NormanGray) (raw view)
---+ VOUnits 1.0 RFC

<span style="color: orangered;">The VOUnits document is now an [[http://www.ivoa.net/documents/VOUnits/index.html][IVOA REC]]; please do not edit this page</span>


---

<span style="color: orangered;">THE TEXT BELOW IS HISTORICAL, FOR REFERENCE ONLY.</span>

VOUnits 1.0-20130724 is now ready for [[#TcgReview][TCG review]]. The review will run for four weeks, from XXX.

The document is available at http://www.ivoa.net/documents/VOUnits/ . This incorporates (somewhat overdue) changes after the RFC process represented by the comments below. *Update, 2013-08-12*: there has been fairly extensive discussion on the semantics@ivoa list, and there will probably be a further draft released semi-formally within a week or so; keep an eye on the semantics list if you're interested in this. *Update, 2013-09-22*: after more on-list discussion, there has been another out-of-cycle release; that is now at the document repository above.

The VOUnits document is intended to describe existing and recommended practice for writing units specifications in IVOA protocols:

   * _Abstract_: This document describes common practices in manipulating units in astronomical metadata and will define a means of consistent representation within VO services. The core of the document contains the rules for writing string representations of unit labels, called VOUnits.
Existing commentary:

   * There has been some discussion of the document during its drafting on the [[http://www.ivoa.net/pipermail/semantics/][Semantics mailing list]]
   * There was some early discussion at DiscussionOnVOUnits.
*Note on implementations*: This document does not define a protocol, but summarises existing practice, and recommends good practice for future IVOA specification. In consequence, the existing software which parses units specifications (summarised in the document) can be regarded as 'implementations' of the document. In addition, the [[https://bitbucket.org/nxg/unity][Unity]] library, which was written in part to validate some of the document's conclusions, can be regarded as an implementation of the standard.

---

---++ Comments

Please follow roughly the following pattern:

---+++ Off-line comments (2012 August)

Norman Gray received some direct comments from Tim Jenness and Markus Demleitner. These were in part typo-level comments, but included some more significant points about the substance of the document. Many thanks to both.

   * Mismatch between table 8 and appendix A.4 (noted: to be fixed in next version)
   * Various style/readability points about included grammars (clarified)
   * Unclear discussion about what units are permitted/recommended/deprecated (included new summary table of unit recommendations)
   * Perhaps mention ISO-80000-13 and IEEE-1541 binary prefixes (kibi-, mibi-, etc) (noted, and text added to the document to confirm that we don't, and don't intend to, talk about these)
   * Discussion of candela vs centi-day and similar (noted, and text _possibly_ clarified; not completely sure what to do here)
   * No discussion of records/arrays of units (noted, and text clarified)
-- IVOA.NormanGray (2012-09-02)

---+++ Additional comments from Markus (2012-09-03)

   * We should at least consider allowing "modern-style" floats (2.2e19 rather than 2.2 10+19); it's still more ambiguity, but in this case it's probably helping a lot towards what people expect.
   * We definitely should allow float mantissas to have no dot at all, i.e., have them match [0-9]+(\.[0-9]*)?. It's the right thing mathematically, and it'll save a world of confusion.
-- IVOA.MarkusDemleitner - 2012-09-03

I'm reluctant to be innovative here, since part of the goal of the VOUnits work is to say "If you write in the VOUnits subset, then others have no excuse for not being able to parse what you write". -- IVOA.NormanGray - 2013-04-29

---+++ Comments from Mark Taylor

Looks good. A couple of very minor things
   * comments on document status in "Status of this document" and "Acknowledgements" sections can probably be adjusted
   * "unit parameter" -&gt; "unit attribute" in sec 1.1?
   * "problem of try to best accomodate" -&gt; "problem of trying to best accommodate" in sec 3.6?
   * some problem with the accents in the BIPM reference in the bibliography
-- IVOA.MarkTaylor - 2012-09-03

All fixed, I think. I don't follow the issue about the "Status of this document" section, but this has been tidied up a bit, so perhaps the problem has been fixed. -- IVOA.NormanGray - 2013-04-29

---++++ <br />Comments from Matthew Graham / VAO (2012-10-01)

Generally there was a feeling that the language in the specification could be clearer on what the basis of this recommendation is, i.e., it's essentially the FITS units convention. Also:

   * There is no mechanism for identifying which units convention one might be using, especially this one - how can I distinguish between using FITS vs VOUnits in a data set. There should be some way of stating this.
   * A list of approved units is not present in the document. This is primarily for reference or quick lookup and could be included as an appendix, along with some examples of usage.
   * The information about the supporting services and libraries is buried in the document and could be more upfront.
   * What is the policy for existing data sets which use unit conventions that are not compliant with VOUnits?
Some more minor things:

   * Solidus and slash are both used to refer to "/"
   * How should electron (counts) be referred to, e.g., e-/ADU?
   * The units data model expresses relationships between certain units, e.g., degree -&gt; arcmin -&gt; arcsec. A conversion model then needs to be stated so that deg/pixel can be calculated given arcsec (as mentioned). This is not just millisec to sec which can be guessed from the prefix alone.
-- IVOA.MatthewGraham - 2012-10-01

   * Clarity: point taken. The recommendation is essentially the intersection of the various existing conventions, plus clarification about what those conventions actually are. This goal has been stated more starkly.
   * Identifying convention: I think this isn't this document's job; each syntax will need something like this. Unless you're suggesting that this document should define or register some well-known names for the syntaxes.
   * List of units; supporting services: these should both now be more visible.
   * Existing data sets: again, I think this goes beyond what the document is aiming for; what this document aims for is a clear target for conformance tests.
   * Solidus/slash: fixed (in favour of solidus)
   * ADU: the 'known units' table is now more prominent
   * Conversions: there are no conversions discussed in this document. We believe we've made this exclusion more prominent.
-- IVOA.NormanGray - 2013-04-29

Thanks for various comments, everyone. See [[https://code.google.com/p/volute/source/browse/#svn%2Ftrunk%2Fprojects%2Fstd-vounits][volute]] revision 2143.
---++++ Comments from Baptiste Cecconi / VOParis (2013-05-16)

We need mass and distance units related to solar system objects, such as: Mass of the Sun, Mass of Earth, of Mars, of Jupiter, of Titan, etc; Radius of Sun, Radius of Earth, of Mars, of Jupiter, of Titan, etc.

Should we propose a list of bodies for which we need it ? And should we propose a convention for naming these units ?

For instance, using the three first letters of the corresponding body, as it is already proposed for the Sun in the current version of recommendation:

   * Mass of Jupiter : jupMass
   * Radius of Jupiter : jupRadius
Additional comment:

What about units in dB ?

A very common unit in low frequency radioastronomy is dB(V^2/Hz) or dB(W/m^2/Hz)

There is the possibility of using the log function as in log(V^2/Hz).

Should we use 10.log(V^2/Hz) instead of dB(V^2/Hz) ?

-- IVOA.BaptisteCecconi - 2013-05-17

Thank you for these comments, Baptiste.

I agree that such a set of units would be desirable, to some extent, but I confess that I don't really know what to do regarding them. We could add a large collection of extra units for various planets, but then we'd feel obliged to add dwarf planets, and satellites, and I don't know where we'd stop. We could encourage or endorse a pattern such as mass(Jupiter), which looks like function application; but again, where do we stop?

The current version of the document (and the associated Unity library) now accepts functions of units. It recommends log, ln, exp and sqrt, because the FITS specification does, but recommends that parsers accept unrecognised functions, even if they subsequently deprecate them in some way. In that way, your 'db(xxx)' example would work. Do you think this needs more specification, so that 'dB' is specifically defined _as a function_ in the document (it's currently defined as a special-case and unprefixable unit).

Given this treatment of functions and unrecognised units, the pattern 'mass(Jupiter)' would therefore, we suggest, be _syntactically_ acceptable to any conforming parser, even though there wouldn't be any recommended semantics attached to it.

-- IVOA.NormanGray - 2013-07-24

---

#TcgReview

---++ Comments from TCG (WG and IG chairs)

---+++ TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham_ )

---+++ Applications Working Group ( _Mark Taylor, Pierre Fernique_ )

Good doc for me. Easy to read, easy to understand.

Just a few remarks and suggestions.

*Page 9 - 2.4 Scale factors*
   * <em>"The SI prefixes of Tables 10 on page 20 always refer to multiples of 1000, even when applied to binary units such as bit or byte.<br /></em>=&gt; Certainly already discussed inside the author and editor group, but could we have a justification or a reference concerning this big distortion of the usual usage ? The fact to use multiples of 1000 rather than 1024 for bytes and bits will certainly generate a lot of errors.<br /> =&gt; I suggest to discourage the usage of K,T,P,E,Z,Y for bits and bytes in the VOUnit contexts.
*page 13- 2.7 Mathematical expressions containing symbols*

   * One problem that VO developers encounter with value+unit parsing is the fact that the value can be also expressed in scientific notation (ex: 10.6E-9g). And in this case the "E" or "e" character can be a problem for parsing correctly the quantity.<br /> =&gt; I just suggest to insert a valid example using scientific notation just to open the mind about this potential problem. May be a short remark could be also helping.<br /> =&gt; "<em>Examples of valid VOUnits are 2.54cm, 10+8m, 3.45 10**(-4)Jy, 10.6E-9g</em>
Answer :<em> Scientific notation example included as suggested. Thanks</em>
   * -- IVOA.MireilleLouys - 2013-07-18
*Page 14 - 2.8 Remarks and good practices*
   * <em>"Expressions such as "d:m:s" ... are not valid VOUnits. This should not be a problem, as existing VO standards already recommend that coordinates be expressed in decimal degrees"<br /></em>=&gt; Is this assertion really true ?<br /> =&gt; Concretely, a lots of VOTable providers use "h:m:s" and "d:m:s" units in "unit" VOTable XML attribute. This sentence will have the consequence that VOTables expressing coordinates in two columns in sexagesimal will be no longer "VO valid" or they will lost their units (in contradiction with 3.4 - 1 - "<em>All data providers should be encouraged to supply units for each column of a table</em>")<br /> =&gt; I would suggest to remove this example, and/or add an exception of unit usage in VOTable REC.
*Page 16 - 3.4 Query languages and 3.5 Broader use in the VO*

   * These 2 last sections appear as a set of reflexions and elements of works but not really more.<br /> =&gt; I would suggest to move them in an appendixes, or just remove them for this release.
-- IVOA.PierreFernique - 2013-04-30

   * There are a couple more relevant standards that could be added to the IVOA architecture diagram in section 1.1: 
      * TAP (the =TAP_SCHEMA.columns= table has a =units= column, sec 2.6.3)
      * <nop>VODataService (the =BaseParam= type contains a =unit= element, sec 3.5)
      * Maybe Relational Registry, but that's still WD, and the units dependency is more or less via <nop>VODataService
-- IVOA.MarkTaylor - 2013-05-30

Thanks for these comments, Pierre. I hope that the rationale in the current version of the document is clearer than it was. I've incorporated changes which arrived via Sébastien and Mireille. -- IVOA.NormanGray - 2013-07-24

All of the above remarks are suggestions for the consideration of the authors to act on if they agree. Apps WG approves the document in any case.

Answer :<em>The Architecture diagram is updated in the new revision of the document (Proposed Recommendation 2013-07-01)</em><br />[[%ATTACHURL%/PR-VOUnits-1.0-20130701.pdf][PR-VOUnits-1.0-20130701.pdf]]

-- IVOA.MireilleLouys - 2013-07-11
---+++ Data Access Layer Working Group ( _Patrick Dowler, Francois Bonnarel_ )

Table 1: Is gramme accidentally in French or is it intended to be the alternate British English spelling? I would have expected gram instead.

Table 2: AU = astro'l unit looks odd; when I first saw it I thought it was astrological unit. I would suggest either the whole word or, if you really need to avoid an ugly use of space, maybe an abbreviation like "astron. unit".

Approved.

-- IVOA.PatrickDowler - 2013-09-27
---+++ Data Model Working Group ( _Jesus Salgado, Omar Laurino_ )

As said in the document, the scope of the present specification is the standardization of the string format for the units in the VO context. Taking into account this scope, the document is clear and complete.

As you probably know, I do not fully share that the only standardization needed for units is the strings. Although it is possible to parse unit strings (we all have a library that, more or less, implements this task) string parsing is, in my view, not recommended and a set of possible strings to be used is limited.

String format for units is important but I would have preferred an approach closer to a data model. In this way, we should not care of the jupiterMass or any other kind of peculiar units as the string itself could be ignored for software clients.

A complete, flexible and powerful data model for units is not simple. As said in the past, an initial step could be to describe the scaling and the dimensional equation but more things could be needed so the scaleq/dimeq info should have been an starting/initial point. This possible data model would have described, also, a serialization.

I approve the document within the aforementioned scope.

-- IVOA.JesusSalgado - 2013-06-14

Thanks, Jesus.

I agree that there is some demand for a standardisation of _quantities_, and possibly some data model which incorporates all of the associated subtleties, and which therefore supports things like unit conversions. However 'quantities' have been contentious in the IVOA in the past, we wanted to avoid re-opening that discussion, and so we concentrated on a specific part of that question, namely the specification of only the unit strings.

-- IVOA.NormanGray - 2013-07-24

Thanks Norman but, in my view, I am not talking about quantities but units. If we say, these are 15 JupMass, I would like to know that 1JupMass=1.898E27 kg (I am talking about what is a JupMass

not the "15" jupiter Masses or the error for this "15"), 1JupMass=1.898E27 kg is the definition of the unit, i.e., this defintion should be part of its data model.

In any case, as said before, I understand this spec is only trying to describe unit strings, so I approve the document within the aforementioned scope.

-- IVOA.JesusSalgado - 2013-07-30
---+++ Grid & Web Services Working Group ( _Andreas Wicenec, Andre Schaaff_ )

Approved -- IVOA.AndreSchaaff - 2013-09-22

---+++ Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner_)

Taking into account the comment of Baptiste Cecconi by proposing "good practice" as for other body of the solar system <br /> page 13 "The symbol Sun is used to express ratios relative to solar values," maybe additional possibility of using other solar system body <br /> with it's 3 first letter Jup, Moo, etc.<br /> <br /> This will help the solar system community to keep compliance with their <br /> services, not waiting for next version.

-- Approved with these comments IVOA.PierreLeSidaner IVOA.GretchenGreene - 2013-09-20

-- Approved IVOA.PierreLeSidaner IVOA.GretchenGreene - 2014-05-15
---+++ Semantics Working Group ( _Norman Gray, Mireille Louys_)

---+++ Data Curation & Preservation Interest Group ( _Alberto Accomazzi, Françoise Genova_ )

---+++ Education Interest Group ( _Massimo Ramella, Sudhanshu Barway_ )

---+++ Knowledge Discovery in Databases Interest Group ( _George Djorgovski_ )

---+++ Theory Interest Group ( _Franck Le Petit, Rick Wagner_ )

As I already said, the document is quite comprehensive. I think that is a good progress for InterOperability to finally have some prescriptions on units in the VO.

I agree with the document. Most of points I noticed in the document have already been mentionned or corrected during the recommendation process.

Approved.
---+++ Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick_ )

I am happy with the content of this document (version 2013-07-24), and have only minor comments:
   * Can we avoid referring to arrows by colour in figure 2? They are clearly distinguished as solid/dashed, but both appear grey on my (monochrome) printer.
   * Appendix A5: the convoluted history of AIPS++/CASA/casacore is out of scope for this discussion. However, AIPS++ is no longer independently developed or availble. I think a reference to both CASA and casacore is all that's needed here.
   * There is a missing quotation mark before "pix" in section 2.3.
-- IVOA.JohnSwinbank - 2013-08-05
---+++ Standards and Processes Committee ( _Françoise Genova_ )

<br /> <!--
      * Set ALLOWTOPICRENAME = IVOA.TWikiAdminGroup
-->
   * [[%ATTACHURL%/PR-VOUnits-1.0-20130701.pdf][PR-VOUnits-1.0-20130701.pdf]]: updates for last TCG comments

   * [[%ATTACHURL%/VOUnits-PR-1.0-20130724.pdf][VOUnits-PR-1.0-20130724.pdf]]: VOUnits PR 1.0-20130724
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
pdf	PR-VOUnits-1.0-20130701.pdf	r1	manage	687.0 K	2013-07-11 - 10:11	MireilleLouys	updates for last TCG comments
pdf	VOUnits-PR-1.0-20130724.pdf	r1	manage	852.2 K	2013-07-25 - 16:50	NormanGray	VOUnits PR 1.0-20130724
Topic revision: r34 - 2014-10-27 - NormanGray
IVOA
Log in or Register
IVOA.net
Wiki Home
WebChanges
WebTopicList
WebStatistics
Twiki Meta & Help
IVOA
Know
Main
Sandbox
TWiki
TWiki intro
TWiki tutorial
User registration
Notify me
Working Groups
Interest Groups
Committees
Stds&Procs
www.ivoa.net
Documents
Events
Members
XML Schema