P3T meeting with Clients - Oct 16, 2024

Attendees

• Attendees:
  • Pat D, Gregory DF, Adrian D, Brian Major, Brian McLean, Dave M, Frossie E, Grgory M, Jesus S, Jose O, Joshua F, Marco M, Mark A, Markus D, Pierre F, Pierre L, Renaud S, Russ A, Sara B, Tamara C, Janet E
• Regrets :

Zoom

Highlights

• Great group attendence at 22
• P3T team presented Documents and prototype work completed so far
• Discussion with Apps and P3T was constructive and supportive with some concerns
• Concerns centered on ...
  • backward compatibility with client code
  • major version changes to standards in the VO in general (somewhat tangential)
  • It was noted that there is code in the VO with different versions of the standards deployed
• Actions:
  • Janet draft invite to Clients via Apps group after discussing at TCG -- DONE
  • Dave prepare ~2 slide for clients mtg with P3T prioriites as a reminder -- DONE
  • JE with P3T help: Prepare mtg page for Client mtg early with links to docs and perhaps a prototype(?!) -- DONE
  • ALL: Review the draft docs so we present them to the Clients after internal review
  • ALL: Look at each others prototypes to know we're on the right track here too before Client mtg

Meeting Agenda/Notes

• Overview/Reminder of what we're setting out to do - From Dave
  • https://github.com/Zarquan/jordanita/blob/main/presentations/20241016-IVOA-P3T/20241016-IVOA-P3T.pdf
  • Why are we doing this? *
    • Cost to projects - developer time
    • Complexity a barrier to adoption
    • Security - some patterns are not meeting modern security recommendations
  • What's in scope
    • Machine readable (OpenAPI)
    • Compatible with current
      • development frameworks
      • tools & IDEs
  • What's out of scope
    • Solving other problems
    • Want to isolate this work
• Pointer to Docs/Prototypes
  • Draft docs: (Russ)
    • Draft IVOA web service standards framework - https://sqr-091.lsst.io/
    • Draft IVOA SODA web service specification - https://sqr-093.lsst.io/
    • Draft IVOA JSON encoding - https://sqr-092.lsst.io/
    • Layered approach to specifications
    • Docs are just intended to be an example of the layering
  • Prototypes in GitHub
    • UWS Implementation (Josh)
      • OPENAPI Doc: https://github.com/ivoa/PTTT/pull/1.
      • Example implementation: https://github.com/spacetelescope/fastapi-uws
      • UWS spec direct link: https://github.com/jwfraustro/PTTT/blob/UWS-MAST/uws/openapi.yml
    • TAP-.2 (1.1 + user table extension) - Pat
      • OpenAPI spec: https://github.com/pdowler/PTTT/tree/main/tap
      • Using (referencing) libraries from openapi.yaml (uws, tap, vosi, …)
      • Tricky part was separating TAP/UWS responsibility
        • TAP describes the jobInfo
        • UWS the interactions
      • Async: Both the UWS and TAP API response codes must be described together
      • Sync: Even more redundancy/overlap than in async
      • Lint tools report conformance
      • Payload modelling not particularly attractive
        • But referencing specs is easier to do
      • Pat Question: is this easier to read/understand?
        • Mark T: yes, easy to read
    • Execution Broker (Dave)
      • OpenAPI component: https://github.com/Zarquan/Calycopis-broker/blob/main/openapi/ivoa/openapi-0.8.yaml
      • Added the execution planning model (payload) to openapi
        • Inheritance works really well
        • Able to describe complex string formats and validators
• Apps/P3T group feedback to Docs/Prototype work - Discussion
  • Russ: would a standardized encoding format be useful to the standardization process?
  • Mark T: How about backward compatibility?
    • (Note: the prototypes shown are not backwards compatible)
    • Russ: a new start with these tools
    • Mark T: we should prioritize backward compatibility over tooling support
    • Markus: experience with SIAP1 & SIAP2. 2 introduced in 2018, but 1 still more prevalent. Fragmentation in major versioning should be avoided. IVOA is a disparate group of implementers with different timelines and budgets.
    • Josh: can't avoid doing major version changes
    • Frossie: Would be good to know the understanding of effort to make the proposed backwards compatibility changes.
    • Pierre F: What are the details of what is not backwards compatible?
      • Russ: rough answer: error handling at a different layer (http rather than payload), issues around data types, xml translation to other protocols, etc. A collection of smallish, client-service interactions.
      • Dave: case sensitivity not supported out-of-box, form encoding not recommended because of security issues
    • Dave on major versioning: should solving this problem be a separate effort?
    • Markus: going this route would require active management to ensure the rollout of the deployment across the VO, in particular reaching out and actively communicating with the data centres with clear communication what they need to do and what it takes to do it.
    • Gregory D-P: the library implementation ecosystem isn't sufficient to write the suite of VO services.
  • Pierre L: appreciates the effort being made to having better described standard definitions.
  • Dave: Are there any other issues we haven't addressed?
  • Marco: What is the extent of the backwards compatibility changes we should make? There are options.
  • Pierre F: Would it be valuable to try to prototype a more simple protocol such as SIA? It would be good to have more specific details of what changes are necessary.
  • Summary
    • Backwards compatibility with client code the main concern
    • Major version changes to standards in the VO in general (somewhat tangential)
    • It was noted that there is code in the VO with different versions of the standards deployed
• Looking ahead to the Nov Interop
  • Schedule topic to organize program of session at Next meeting;; We're scheduled 2-3:30 on Friday, Nov 15
  • Marco suggested leaving plenty of time for discussion with audience.
  • JE suggestion for folks to consider ... talks 10-12 min for Proto/Doc talks; 10 min Janet/Brian and keep all question to end?? That leave ~30 min questions/discussion. Could make it a panel and include a few apps ppl???
    • 2 minute intro - Janet
    • Prototypes/Implementations -- What's the angle we want to present?? Do we try to answer a short set of questions??
      • Josh
      • Pat
      • Dave
    • Documents
      • Russ
    • Wrap-up/Discussion - Brian
• Next meeting:
  • 2 suggestions: Wed, Oct 23 @20UTC or Wed, Nov 06 @20UTC (JE traveling week of Oct 27)