Difference: SODA-1_0-Next (1 vs. 2)

Revision 22023-11-11 - FrancoisBonnarel

 
META TOPICPARENT name="IvoaDAL"

SODA1.0-Next page.

  • Previous material before May 2020 (most of it common with SIA-Next + DAP)
Old "DAL future" page : https://wiki.ivoa.net/twiki/bin/view/IVOA/DALFuture

IVOA note : http://www.ivoa.net/documents/Notes/RecentDALProtocolsFeedback/index.html

IVOA interop meetings presentation :

Paris : https://wiki.ivoa.net/internal/IVOA/InterOpMay2019DAL/SIA2-SODA-next.pdf

College Park : https://wiki.ivoa.net/internal/IVOA/InterOpNov2018DAL/HiPStoFits_Prototype_IVOA.pdf

Victoria : https://wiki.ivoa.net/internal/IVOA/InterOpMayy2018DAL/DAL-Feedback.pdf

  • Additional material since May 2020 (IVOA interop meetings and Running meetings - most of it common with SIA-Next + DAP)

We list presentations also look at Notes and discussions.

May 2020 virtual interop : https://wiki.ivoa.net/internal/IVOA/InterOpMay2020DAL/SIA2-SODA-nextPyVO-support.pdf

November 2020 virtual interop : https://wiki.ivoa.net/internal/IVOA/InterOpNov2020DAL/SIA2-SODA-next-status.pdf

January 2021 running meeting #7: https://wiki.ivoa.net/internal/IVOA/IvoaDAL_RunningMeetings/IVOA_DAL_RM7_etherpad.txt

February 2021 running meeting #8 : https://wiki.ivoa.net/internal/IVOA/IvoaDAL_RunningMeetings/IVOA_DAL_RM8_etherpad.txt

March 2021 running meeting #9 : https://wiki.ivoa.net/internal/IVOA/IvoaDAL_RunningMeetings/IVOA_DAL_RM9_etherpad.txt

May 2021 virtual interop : https://wiki.ivoa.net/internal/IVOA/InterOpMay2021DAL/SIA2-nextDataSetDiscov.pdf

November 2021 virtual interop : https://wiki.ivoa.net/internal/IVOA/InterOpNov2021DAL/minoc-soda-objectstore.pdf

April 2022 virtual interop : https://wiki.ivoa.net/internal/IVOA/InterOpApr2022DAL/SIA2SODAUpgradesExtendingSIA2toDataSetSAP.pdf

September 2022 running meeting #14 : https://wiki.ivoa.net/internal/IVOA/IvoaDAL_RunningMeetings/DAP-SODA-DALRunningmeeting.pdf

October 2022 virtual interop : https://wiki.ivoa.net/internal/IVOA/InterOpApr2022DAL/SIA2SODAUpgradesExtendingSIA2toDataSetSAP.pdf

March 2022 running meeting #15 : https://wiki.ivoa.net/internal/IVOA/IvoaDAL_RunningMeetings/DalRunning10-DL-DAP-SODA.pdf

May 2023 Bologna interop : https://wiki.ivoa.net/internal/IVOA/InterOpMay2023DAL/SIA-DAPSODA.pdf

SODA-1.0 feedback, issues and proposals

On GitHub: https://github.com/ivoa-std/SODA

  • Wrong VOTAble syntax in polarization example #1
ON January the 16th 2020 Alberto Micol discovered :

Page 19 of the SODA standard shows:

'Polarization states to be extracted.

My understanding of the VOTable standard is that the correct way should instead be

(see the sentence

"All three MIN, MAX and OPTION sub-elements store their value corresponding to the minimum, maximum, or ``special value'' in a value attribute."

In the paragraph at: http://www.ivoa.net/documents/VOTable/20191021/REC-VOTable-1.4-20191021.html#sec:values ):

Polarization states to be extracted.

Alberto is perfectly right !!!

  • Wrong example for BAND interval MIN and MAX values #2

This appeared in a discussion between Alberto Micol and Markus on the dal Mailing list

MIN and MAX cannot be single values as in the example It should be an array. But an array of what ?

An array of minimal values and an array of maximal values (Markus)

An array of minimal length interval and an array of maximal length interval (Alberto)

  • pixel cutouts (instead of world coordinates) on all axes are missing #3

Considered for version 1.0 but delayed to stay simple (due to syntax problems) Asked by CASDA , others Ranges can be given by SODA service descriptor or computed from ObsCore

Which syntax ?

  • No possibility to control output WCS by regridding/rebinning exists in SODA. #4

This was possible in SIA1.0 for the spatial axis (only) could be done by ADDing standard parameters to SODA for Spatial resolution : SPATRES Rotation : ROTA Sky Projection : PROJ or Alternatively by providing a WCS = (full text of WCS part of FITS header)

Skyview + CDS (SIA on top of Hips2FITS) have this functionality

Should we stabdardize it ? And If yes should we extend it to SIA2 in virtual mode (SIA issue #8) ? Or should we let it as a mixture of SIAP2 and SIAP1 parameters ?

  • It is not possible to query SODA services by MOC #5

MOC (as well as STMOC, and maybe STEMOC) has proven to be a very efficient way to describe the coverage of data and is used for discovery of data. It has been often discussed and also during last interop that it should be useful to query SODA services by MOC? It should probably be with a new parameter and not via POS considering that it is probably more useful to have STMOC or STEMOC, than spatial MOC alone. This is also valid for SIA of course (SIA Issue #9). Of course, the behaviour for SODA will be very different than the one with SIA. SODA-(ST)MOC will extract a subset matching the (ST)MOC while SIA-MOC will only select datasets intersecting with the MOC. I wonder if this new parameter should not be standardized by DALI

  • Managing multiple PARAMS queries responses #6

When several of the PARAM queries (POS, BAND, etc...) are multiple we are facing a cartesian product issue: how do we retrieve all these results. Seems to be easy in async mode which is built to manage multiple results? But in sync? MEF formats? archive files? VOTable containing pointers? What about using UPLOAD functionality (DALI, 3.4.5 UPLOAD) for this?

  • SODA spec doesn't tell us how to provide dataproduct_type transformation #7

First implementations already provide various flavours of cutouts from cubes. Some cutout interface can extract subcubes from a large cube. Some others could create spectra or images (dimension reduction). An idea could be to use the DPTYPE parameter designed for SIAP2 to force this product_type conversion. This has been discussed for TimeSeries there: http://www.ivoa.net/documents/Notes/TimeSeriesDiscoveryAndAccess/index.html


<--  
-->
Added:
>
>
META FILEATTACHMENT attachment="SODA-20231111.pdf" attr="" comment="Proposition of a new version of SODA" date="1699740309" name="SODA-20231111.pdf" path="SODA-20231111.pdf" size="704641" user="FrancoisBonnarel" version="1"
META FILEATTACHMENT attachment="DAP-SODA-status.pdf" attr="" comment="DAP and SODA new fatures : why ?" date="1699742853" name="DAP-SODA-status.pdf" path="DAP-SODA-status.pdf" size="490899" user="FrancoisBonnarel" version="1"
 

Revision 12023-05-31 - FrancoisBonnarel

 
META TOPICPARENT name="IvoaDAL"

SODA1.0-Next page.

  • Previous material before May 2020 (most of it common with SIA-Next + DAP)
Old "DAL future" page : https://wiki.ivoa.net/twiki/bin/view/IVOA/DALFuture

IVOA note : http://www.ivoa.net/documents/Notes/RecentDALProtocolsFeedback/index.html

IVOA interop meetings presentation :

Paris : https://wiki.ivoa.net/internal/IVOA/InterOpMay2019DAL/SIA2-SODA-next.pdf

College Park : https://wiki.ivoa.net/internal/IVOA/InterOpNov2018DAL/HiPStoFits_Prototype_IVOA.pdf

Victoria : https://wiki.ivoa.net/internal/IVOA/InterOpMayy2018DAL/DAL-Feedback.pdf

  • Additional material since May 2020 (IVOA interop meetings and Running meetings - most of it common with SIA-Next + DAP)

We list presentations also look at Notes and discussions.

May 2020 virtual interop : https://wiki.ivoa.net/internal/IVOA/InterOpMay2020DAL/SIA2-SODA-nextPyVO-support.pdf

November 2020 virtual interop : https://wiki.ivoa.net/internal/IVOA/InterOpNov2020DAL/SIA2-SODA-next-status.pdf

January 2021 running meeting #7: https://wiki.ivoa.net/internal/IVOA/IvoaDAL_RunningMeetings/IVOA_DAL_RM7_etherpad.txt

February 2021 running meeting #8 : https://wiki.ivoa.net/internal/IVOA/IvoaDAL_RunningMeetings/IVOA_DAL_RM8_etherpad.txt

March 2021 running meeting #9 : https://wiki.ivoa.net/internal/IVOA/IvoaDAL_RunningMeetings/IVOA_DAL_RM9_etherpad.txt

May 2021 virtual interop : https://wiki.ivoa.net/internal/IVOA/InterOpMay2021DAL/SIA2-nextDataSetDiscov.pdf

November 2021 virtual interop : https://wiki.ivoa.net/internal/IVOA/InterOpNov2021DAL/minoc-soda-objectstore.pdf

April 2022 virtual interop : https://wiki.ivoa.net/internal/IVOA/InterOpApr2022DAL/SIA2SODAUpgradesExtendingSIA2toDataSetSAP.pdf

September 2022 running meeting #14 : https://wiki.ivoa.net/internal/IVOA/IvoaDAL_RunningMeetings/DAP-SODA-DALRunningmeeting.pdf

October 2022 virtual interop : https://wiki.ivoa.net/internal/IVOA/InterOpApr2022DAL/SIA2SODAUpgradesExtendingSIA2toDataSetSAP.pdf

March 2022 running meeting #15 : https://wiki.ivoa.net/internal/IVOA/IvoaDAL_RunningMeetings/DalRunning10-DL-DAP-SODA.pdf

May 2023 Bologna interop : https://wiki.ivoa.net/internal/IVOA/InterOpMay2023DAL/SIA-DAPSODA.pdf

SODA-1.0 feedback, issues and proposals

On GitHub: https://github.com/ivoa-std/SODA

  • Wrong VOTAble syntax in polarization example #1
ON January the 16th 2020 Alberto Micol discovered :

Page 19 of the SODA standard shows:

'Polarization states to be extracted.

My understanding of the VOTable standard is that the correct way should instead be

(see the sentence

"All three MIN, MAX and OPTION sub-elements store their value corresponding to the minimum, maximum, or ``special value'' in a value attribute."

In the paragraph at: http://www.ivoa.net/documents/VOTable/20191021/REC-VOTable-1.4-20191021.html#sec:values ):

Polarization states to be extracted.

Alberto is perfectly right !!!

  • Wrong example for BAND interval MIN and MAX values #2

This appeared in a discussion between Alberto Micol and Markus on the dal Mailing list

MIN and MAX cannot be single values as in the example It should be an array. But an array of what ?

An array of minimal values and an array of maximal values (Markus)

An array of minimal length interval and an array of maximal length interval (Alberto)

  • pixel cutouts (instead of world coordinates) on all axes are missing #3

Considered for version 1.0 but delayed to stay simple (due to syntax problems) Asked by CASDA , others Ranges can be given by SODA service descriptor or computed from ObsCore

Which syntax ?

  • No possibility to control output WCS by regridding/rebinning exists in SODA. #4

This was possible in SIA1.0 for the spatial axis (only) could be done by ADDing standard parameters to SODA for Spatial resolution : SPATRES Rotation : ROTA Sky Projection : PROJ or Alternatively by providing a WCS = (full text of WCS part of FITS header)

Skyview + CDS (SIA on top of Hips2FITS) have this functionality

Should we stabdardize it ? And If yes should we extend it to SIA2 in virtual mode (SIA issue #8) ? Or should we let it as a mixture of SIAP2 and SIAP1 parameters ?

  • It is not possible to query SODA services by MOC #5

MOC (as well as STMOC, and maybe STEMOC) has proven to be a very efficient way to describe the coverage of data and is used for discovery of data. It has been often discussed and also during last interop that it should be useful to query SODA services by MOC? It should probably be with a new parameter and not via POS considering that it is probably more useful to have STMOC or STEMOC, than spatial MOC alone. This is also valid for SIA of course (SIA Issue #9). Of course, the behaviour for SODA will be very different than the one with SIA. SODA-(ST)MOC will extract a subset matching the (ST)MOC while SIA-MOC will only select datasets intersecting with the MOC. I wonder if this new parameter should not be standardized by DALI

  • Managing multiple PARAMS queries responses #6

When several of the PARAM queries (POS, BAND, etc...) are multiple we are facing a cartesian product issue: how do we retrieve all these results. Seems to be easy in async mode which is built to manage multiple results? But in sync? MEF formats? archive files? VOTable containing pointers? What about using UPLOAD functionality (DALI, 3.4.5 UPLOAD) for this?

  • SODA spec doesn't tell us how to provide dataproduct_type transformation #7

First implementations already provide various flavours of cutouts from cubes. Some cutout interface can extract subcubes from a large cube. Some others could create spectra or images (dimension reduction). An idea could be to use the DPTYPE parameter designed for SIAP2 to force this product_type conversion. This has been discussed for TimeSeries there: http://www.ivoa.net/documents/Notes/TimeSeriesDiscoveryAndAccess/index.html


<--  
-->
 
This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback