VOTable-JSON output discussion

As discussed in the IVOA Interop in South Africa, many of the service providers are either already using, or have thought of using JSON as a complimentary output format for TAP (as well as other data access services). Many large projects such as LSST have often expressed a preference for JSON over XML, and it seems to be worth discussing whether we should, and if so how to propose a votable-JSON standard output for TAP services.

Below are some of the different approaches that have been discussed on the DAL mailing list:

"columns": [

{"name":"foo", "datatype":"int"},

{"name":"bar", "datatype":"long"}

{"name":"baz", "datatype":"double"}

],

"data" : [

[ 1, "1.12", 2.123]

[ 2, "1.34", 2.345]

[ 3, "1.56", 2.678]

]

+ Extremely easy to deal with in javascript

+ Human Readable

i.e:

var table = $(/* call TAP here via jquery */);

table.columns.forEach(function(i){ console.log(i.name)}); // log names

table.rows.forEach( doSomethingWIthRow );

- Missing some metadata? i.e. Query status, overflow, description? (Possible to extend by adding additional metadata in key/array pairs, e.g. "metadata/query/info")

"I have written some generic code to output a DOM in JSON in addition to the usual XML. It is a set of rules mostly consistent with http://badgerfish.ning.com/ but with simpler namespace handling and more consistent array/list output. We have it in use in vospace (for sure) and maybe UWS (can't recall if that is released yet). I intend to try outputting VOTable using this code... it will be more complex than the above examples and iirc there are some structures allowed in VOTable that don't map perfectly." <Pat Dowler>

+ Output structure will closely match VOTable?

- More complex object. May lose some of the simplicity of JSON output.

Based on Boost Property Tree http://www.boost.org/doc/libs/1_60_0/doc/html/property_tree/parsers.html#property_tree.parsers.json_parser

"VOTABLE":

{

"<xmlattr>":

{

"version": "1.3",

"xmlns:xsi": "http:\/\/www.w3.org\/2001\/XMLSchema-instance",

"xmlns": "http:\/\/www.ivoa.net\/xml\/VOTable\/v1.3",

"xmlns:stc": "http:\/\/www.ivoa.net\/xml\/STC\/v1.30"

},

...

"RESOURCE":

{

"<xmlattr>":

{

"type": "results"

},

"INFO":

{

"<xmlattr>":

{

"name": "QUERY_STATUS",

"value": "OK"

}

},

"TABLE":

{

"FIELD":

{

"<xmlattr>":

{

"name": "cntr",

"datatype": "int"

}

},

...

"DATA":

{

"TABLEDATA":

[

[

"224910226",

"s",

...

]

]

}

}

}

}

}

+ Direct VOTable (XML) mapping

- Similarly to previous example, result is a more complex object. May lose some of the simplicity of JSON output.

• How to we find a compromise between the lightness that JSON provides and the richness the XML/VOTable can contribute?
• Should we think of mapping based on XML to JSON converters http://badgerfish.ning.com/, http://camel.apache.org/xmljson.html?
• Do we define a mimetype (application/x-votable+json?)?
• How many would use it? (Clients? Internal use?)
• Do we write an IVOA Note for it?
• Or advise/suggestion as part of the VOTable/TAP docs? "In the MOC standard document, we just suggested one way to serialize MOC in JSON. It is not in the standard part of the document, but just as an advise." <Pierre Fenrique>