Data Sheets and Model Cards

Astronomy datasets have been growing, and so are the attempts to use them wth a variety of machine learning techniques. While we would like to use all data, data fusion for diverse uneven or not-fully-matched datasets can be a challenge. Creating machine learning and artificial intelligence models for such datasets and follow-up validation can be challenging owing to lack of large labeled training datasets. To address this two related concepts that have emerged recently in data science are that of Data Sheets for data sets (Gebru et al. arXiv:1803.09010), and model cards for models (Mitchell et al., arXiv:1810.03993). This is just like each component in the electronics industry comes with a datasheet that describes operating characteristics, test results, recommended use etc., We recommend that for each astronomy dataset uniform and standardized datasheets that advertise similar meta-properties should be created, not just stating what “is” but also where each of the dataset could go, much like lego-blocks. This will enable data fusion, and also thwart mis-guided use of datasets. Similarly the models that we build will carry not just the usual provenance, but explicit characteristics displaying known biases and hence added caution when being used in certain ways. While this trend started in social fields where bias is explicit, it has been successfully applied in the Planetary Data System (PDS) setup for identifying key descriptors in an equally diverse dataspace (https://pds.nasa.gov/datastandards/documents/im/v1/index_1G00.html#10.31%C2%A0%C2%A0class_pds_observation_area).