VOTable-JSONdiscussion < IVOA

IVOA Web>TWikiUsers>SteliosVoutsinas>VOTable-JSONdiscussion (2022-05-04, PaulHarrison) (raw view)
<br /> <!--
      * Set ALLOWTOPICRENAME = IVOA.TWikiAdminGroup
-->'

---+ VOTable-JSON output discussion

<div id="_mcePaste">As discussed in the IVOA Interop in South Africa, many of the service providers are either already using, or have thought of using JSON as a complimentary output format for TAP (as well as other data access services). Many large projects such as LSST have often expressed a preference for JSON over XML, and it seems to be worth discussing whether we should, and if so how to propose a votable-JSON standard output for TAP services.</div>

<div id="_mcePaste">Below are some of the different approaches that have been discussed on the DAL mailing list:</div>

---++++ 1) Simplistic approach suggested by [Brian McKlaveren, Thomas Boch, Stelios Voutsinas, Grégory Mantelet]

<span style="color: #7d2727; font-family: Consolas, Menlo, Monaco, "> </span>{
<div style="padding-left: 30px;" id="_mcePaste">"columns": [</div> <div style="padding-left: 60px;" id="_mcePaste">{"name":"foo", "datatype":"int"},</div> <div style="padding-left: 60px;" id="_mcePaste">{"name":"bar", "datatype":"long"}</div> <div style="padding-left: 60px;" id="_mcePaste">{"name":"baz", "datatype":"double"}</div> <div style="padding-left: 30px;" id="_mcePaste">],</div> <div style="padding-left: 30px;" id="_mcePaste">"data" : [</div> <div style="padding-left: 60px;" id="_mcePaste">[ 1, "1.12", 2.123]</div> <div style="padding-left: 60px;" id="_mcePaste">[ 2, "1.34", 2.345]</div> <div style="padding-left: 60px;" id="_mcePaste">[ 3, "1.56", 2.678]</div> <div style="padding-left: 30px;" id="_mcePaste">]</div>

}

*Pros:*
<div id="_mcePaste">+ Extremely easy to deal with in javascript</div> <div id="_mcePaste">+ Human Readable</div>

<div id="_mcePaste">i.e:</div> <div id="_mcePaste">var table = $(/* call TAP here via jquery */);</div> <div id="_mcePaste">table.columns.forEach(function(i){ console.log(i.name)}); // log names</div> <div id="_mcePaste">table.rows.forEach( doSomethingWIthRow );</div>

*Cons:*
<div id="_mcePaste">- Missing some metadata? i.e. Query status, overflow, description? (Possible to extend by adding additional metadata in key/array pairs, e.g. "metadata/query/info")</div>

---++++ 2) XML to JSON conversion using badgerfish [Pat Dowler]
<div id="_mcePaste">"I have written some generic code to output a DOM in JSON in addition to the usual XML. It is a set of rules mostly consistent with [[http://badgerfish.ning.com/ ][http://badgerfish.ning.com/]] but with simpler namespace handling and more consistent array/list output. We have it in use in vospace (for sure) and maybe UWS (can't recall if that is released yet). I intend to try outputting VOTable using this code... it will be more complex than the above examples and iirc there are some structures allowed in VOTable that don't map perfectly." &lt;<a href="PatrickDowler" style="background-color: #ffffff;" title="Patrick Dowler">Pat Dowler</a>&gt;</div>

*Pros:*
<div id="_mcePaste">+ Output structure will closely match VOTable?</div>

*Cons:*
<div id="_mcePaste">- More complex object. May lose some of the simplicity of JSON output.</div>

---++++ 3) Mapping based on Boost Property Tree [Walter Landry]
<div id="_mcePaste">Based on Boost Property Tree http://www.boost.org/doc/libs/1_60_0/doc/html/property_tree/parsers.html#property_tree.parsers.json_parser</div>

<span style="background-color: transparent;">{</span>
<div style="padding-left: 30px;" id="_mcePaste">"VOTABLE":</div> <div style="padding-left: 30px;" id="_mcePaste">{</div> <div style="padding-left: 60px;" id="_mcePaste">"&lt;xmlattr&gt;":</div> <div style="padding-left: 60px;" id="_mcePaste">{</div> <div style="padding-left: 90px;" id="_mcePaste">"version": "1.3",</div> <div style="padding-left: 90px;" id="_mcePaste">"xmlns:xsi": "http:\/\/www.w3.org\/2001\/XMLSchema-instance",</div> <div style="padding-left: 90px;" id="_mcePaste">"xmlns": "http:\/\/www.ivoa.net\/xml\/VOTable\/v1.3",</div> <div style="padding-left: 90px;" id="_mcePaste">"xmlns:stc": "http:\/\/www.ivoa.net\/xml\/STC\/v1.30"</div> <div style="padding-left: 60px;" id="_mcePaste">},</div> <div style="padding-left: 60px;" id="_mcePaste">...</div> <div style="padding-left: 60px;" id="_mcePaste">"RESOURCE":</div> <div style="padding-left: 60px;" id="_mcePaste">{</div> <div style="padding-left: 90px;" id="_mcePaste">"&lt;xmlattr&gt;":</div> <div style="padding-left: 90px;" id="_mcePaste">{</div> <div style="padding-left: 120px;" id="_mcePaste">"type": "results"</div> <div style="padding-left: 90px;" id="_mcePaste">},</div> <div style="padding-left: 90px;" id="_mcePaste">"INFO":</div> <div style="padding-left: 90px;" id="_mcePaste">{</div> <div style="padding-left: 120px;" id="_mcePaste">"&lt;xmlattr&gt;":</div> <div style="padding-left: 120px;" id="_mcePaste">{</div> <div style="padding-left: 150px;" id="_mcePaste">"name": "QUERY_STATUS",</div> <div style="padding-left: 150px;" id="_mcePaste">"value": "OK"</div> <div style="padding-left: 120px;" id="_mcePaste">}</div> <div style="padding-left: 90px;" id="_mcePaste">},</div> <div style="padding-left: 90px;" id="_mcePaste">"TABLE":</div> <div style="padding-left: 90px;" id="_mcePaste">{</div> <div style="padding-left: 120px;" id="_mcePaste">"FIELD":</div> <div style="padding-left: 120px;" id="_mcePaste">{</div> <div style="padding-left: 150px;" id="_mcePaste">"&lt;xmlattr&gt;":</div> <div style="padding-left: 150px;" id="_mcePaste">{</div> <div style="padding-left: 180px;" id="_mcePaste">"name": "cntr",</div> <div style="padding-left: 180px;" id="_mcePaste">"datatype": "int"</div> <div style="padding-left: 150px;" id="_mcePaste">}</div> <div style="padding-left: 120px;" id="_mcePaste">},</div> <div style="padding-left: 120px;" id="_mcePaste">...</div> <div style="padding-left: 120px;" id="_mcePaste">"DATA":</div> <div style="padding-left: 120px;" id="_mcePaste">{</div> <div style="padding-left: 150px;" id="_mcePaste">"TABLEDATA":</div> <div style="padding-left: 150px;" id="_mcePaste">[</div> <div style="padding-left: 150px;" id="_mcePaste">[</div> <div style="padding-left: 150px;" id="_mcePaste">"224910226",</div> <div style="padding-left: 150px;" id="_mcePaste">"s",</div> <div style="padding-left: 150px;" id="_mcePaste">...</div> <div style="padding-left: 150px;" id="_mcePaste">]</div> <div style="padding-left: 150px;" id="_mcePaste">]</div> <div style="padding-left: 120px;" id="_mcePaste">}</div> <div style="padding-left: 90px;" id="_mcePaste">}</div> <div style="padding-left: 60px;" id="_mcePaste">}</div> <div style="padding-left: 30px;" id="_mcePaste">}</div> <div id="_mcePaste">}</div>

*Pros:*
<div id="_mcePaste">+ Direct VOTable (XML) mapping</div>

*Cons:*
<div id="_mcePaste">- Similarly to previous example, result is a more complex object. May lose some of the simplicity of JSON output.</div>

---++ 
---++ Questions to consider:
<div id="_mcePaste">
   * How to we find a compromise between the lightness that JSON provides and the richness the XML/VOTable can contribute?
   * Should we think of mapping based on XML to JSON converters http://badgerfish.ning.com/, http://camel.apache.org/xmljson.html?
   * Do we define a mimetype (application/x-votable+json?)?
   * How many would use it? (Clients? Internal use?)
   * Do we write an IVOA Note for it?
   * Or advise/suggestion as part of the VOTable/TAP docs? "In the MOC standard document, we just suggested one way to serialize MOC in JSON. It is not in the standard part of the document, but just as an advise." &lt;<a href="PierreFernique" title="Pierre Fernique">Pierre Fenrique</a>&gt;
   * https://seriot.ch/projects/parsing_json.html
</div>
Topic revision: r2 - 2022-05-04 - PaulHarrison
IVOA
Log in or Register
IVOA.net
Wiki Home
WebChanges
WebTopicList
WebStatistics
Twiki Meta & Help
IVOA
Know
Main
Sandbox
TWiki
TWiki intro
TWiki tutorial
User registration
Notify me
Working Groups
Interest Groups
Time Domain
Committees
Stds&Procs
www.ivoa.net
Documents
Events
Members
XML Schema