| François Lanusse (remote) |
Make astronomical data AI-ready across surveys: Lessons-learned from the Multimodal Universe Project |
15+5 |
Deep Learning has seen a recent shift in paradigm, from training specialized models on dedicated datasets, to so-called Foundation Models, trained in a self-supervised manner on vast amounts of data and then adapted to solve specific tasks with state-of-the-art performance. This new paradigm has been exceptionally successful not only for large language models (LLMs) but in other domains such as vision models. However applications of this new approach in astrophysics are still very scarce, for reasons ranging from new architectures to the (surprising) lack of availability of suitable large scale datasets. In this talk, I will discuss our work on building the Multimodal Universe Dataset, the first attempt at building a framework for homogenizing data across many astronomical surveys specifically for AI training purposes. Besides discussing the technical solution we adopted, I will highlight some of the outstanding challenges to enable cross-matching across surveys and streaming data in a way that is compatible with large AI model training. Project page: https://github.com/MultimodalUniverse/MultimodalUniverse |
|
