PDL 1.0 Proposed Recommendation : Request For CommentsThe Community comments period is now finished. The TCG Review was running during 4 weeks from 5 February to 5 March 2014 The comments and answers of the previous period are available in an Archive part of this page to avoid any confusion with the new Community RFC. The very last document is available here : pdf: Last version : 18 May 2014 This document will act as RFC centre for the PDL 1.0 Proposed Recommendation http://www.ivoa.net/documents/PDL/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.
Implementation detailsThe two main implementations are:
TCG Review Period
Comments from the TCG members during the TCG Review period: 5 February to 5 March 2014
Application Working Group ( _Mark Taylor, Pierre Fernique)Some of my concerns from the earlier RFC period have been addressed. Following the comments from Franck et al., I am prepared to believe that PDL is a good fit to the kind of constraints that are required for theoretical services for which this standard is targetted. I am however a bit sceptical of sentences like:
As long as the scope of the current version of this standard is mainly recognised as applying to theoretical type services, Applications WG recommends approval. A few new errors have been introduced since the previous version:
These errors are now fixed --IVOA.AndreSchaaff
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)
3.3, third paragraph, I think radiant should be radians The big one: The document presents a data model for Service, but none of the diagrams that present the model use standard recognisable UML. This makes it very hard for readers to understand the data model. At a minimum, you would need to add a section that explains the diagram syntax and how UML concepts are expressed, but that still forces many readers to learn a new diagram language for this document. Not sure how to resolve this easily...
-- PatrickDowler - 2014-04-29 Approved. -- PatrickDowler - 2014-05-18 Data Model Working Group ( _Jesus Salgado, Omar Laurino)Approved -- OmarLaurino - 2014-05-18 Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec)Approved | ||||||||
Added: | ||||||||
> > | -- AndreSchaaff - 2014-05-18 | |||||||
Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Registry approves this document GretchenGreene May 18, 2014
Semantics Working Group ( _Norman Gray, Mireille Louys)The semantics working group has only relatively minor comments: Substance:
-- NormanGray - 2014-04-07
Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)As I already said, to my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. When I started to work on Theory at the IVOA (end of 2005), I asked to scientists what they would expect from VO-Theory. For a lot of scientists, that was not to have access to simulations but to be able to give online access to their codes. The interest to do it in a VO context was the hope to be able to pipe scientific codes together. As, for example, to post-process outputs of large MHD simulations (published in a database - so relying on what is today SimDM / SimDAL) with online codes to compute, for example line intensities that they could compare to observations. At this time, the CEA by Astrogrid partly answered the need. It allowed to easily put online a scientific code, but had not detail descriptions for interoperability between applications in a scientific meaning (units, type of data, semantics tags, ...). The CEA has never been standardized, maybe because the funding difficulties of Astrogrid. Today, PDL answers the need. With it, it is now possible to describe a scientific application in a standardized way. It may seems complicate to use. But, in fact, for simple applications, the description is also simple. The standard is just comprehensive enough to allow description of complex applications. I also noticed that in France, several data centers plan to use PDL to describe their applications and give access to them in a VO context. In Spain, people working in European projects about workflows, not directly related to VO, find it useful and integrated it in Taverna for example. I think that most comments done during the Hawaii InterOp have been taken into account in the revised version of the document. For me, we can approve it. -- FranckLePetit - 2014-03-06 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
Archive : Previous Community period for commentsPlease, don't modify the below part of this page.
IVOA Community Comments Period: 24 December 2013 - 21 January 2014
Comments from the IVOA Community during the RFC period:
PDL as it stands is a serialization of an underlying data model for service parameters (where one could add several types that have been identified as necessary in DAL discussions, e.g., intervals, but that's another matter). I suspect it would be fairly straightforward to formulate the DM itself in VO-DML. The disadvantage would be that the schema generated by VO-DML itself would be different from the hand-written one, so implementations would need an update. On the other hand, the DM is specified explicitely rather than being somewhat implied by the combination of XML Schema and prose, it's much clearer what to do on extension, and you get a VOTable serialization for free, which is really nice when you want to embed PDL in, say, responses of DAL services. -- MarkusDemleitner - 2014-01-14
Sorry for picking this up so late. -- OmarLaurino - 2014-04-23 N.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
|
PDL 1.0 Proposed Recommendation : Request For CommentsThe Community comments period is now finished. The TCG Review was running during 4 weeks from 5 February to 5 March 2014 The comments and answers of the previous period are available in an Archive part of this page to avoid any confusion with the new Community RFC. The very last document is available here : pdf: Last version : 18 May 2014 This document will act as RFC centre for the PDL 1.0 Proposed Recommendation http://www.ivoa.net/documents/PDL/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.
Implementation detailsThe two main implementations are:
TCG Review Period
Comments from the TCG members during the TCG Review period: 5 February to 5 March 2014
Application Working Group ( _Mark Taylor, Pierre Fernique)Some of my concerns from the earlier RFC period have been addressed. Following the comments from Franck et al., I am prepared to believe that PDL is a good fit to the kind of constraints that are required for theoretical services for which this standard is targetted. I am however a bit sceptical of sentences like:
As long as the scope of the current version of this standard is mainly recognised as applying to theoretical type services, Applications WG recommends approval. A few new errors have been introduced since the previous version:
These errors are now fixed --IVOA.AndreSchaaff
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)
3.3, third paragraph, I think radiant should be radians The big one: The document presents a data model for Service, but none of the diagrams that present the model use standard recognisable UML. This makes it very hard for readers to understand the data model. At a minimum, you would need to add a section that explains the diagram syntax and how UML concepts are expressed, but that still forces many readers to learn a new diagram language for this document. Not sure how to resolve this easily...
-- PatrickDowler - 2014-04-29 Approved. -- PatrickDowler - 2014-05-18 Data Model Working Group ( _Jesus Salgado, Omar Laurino) | ||||||||
Changed: | ||||||||
< < | Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec) | |||||||
> > | ||||||||
Added: | ||||||||
> > | Approved
-- OmarLaurino - 2014-05-18 Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec) | |||||||
Approved
Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Registry approves this document GretchenGreene May 18, 2014
Semantics Working Group ( _Norman Gray, Mireille Louys)The semantics working group has only relatively minor comments: Substance:
-- NormanGray - 2014-04-07
Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)As I already said, to my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. When I started to work on Theory at the IVOA (end of 2005), I asked to scientists what they would expect from VO-Theory. For a lot of scientists, that was not to have access to simulations but to be able to give online access to their codes. The interest to do it in a VO context was the hope to be able to pipe scientific codes together. As, for example, to post-process outputs of large MHD simulations (published in a database - so relying on what is today SimDM / SimDAL) with online codes to compute, for example line intensities that they could compare to observations. At this time, the CEA by Astrogrid partly answered the need. It allowed to easily put online a scientific code, but had not detail descriptions for interoperability between applications in a scientific meaning (units, type of data, semantics tags, ...). The CEA has never been standardized, maybe because the funding difficulties of Astrogrid. Today, PDL answers the need. With it, it is now possible to describe a scientific application in a standardized way. It may seems complicate to use. But, in fact, for simple applications, the description is also simple. The standard is just comprehensive enough to allow description of complex applications. I also noticed that in France, several data centers plan to use PDL to describe their applications and give access to them in a VO context. In Spain, people working in European projects about workflows, not directly related to VO, find it useful and integrated it in Taverna for example. I think that most comments done during the Hawaii InterOp have been taken into account in the revised version of the document. For me, we can approve it. -- FranckLePetit - 2014-03-06 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
Archive : Previous Community period for commentsPlease, don't modify the below part of this page.
IVOA Community Comments Period: 24 December 2013 - 21 January 2014
Comments from the IVOA Community during the RFC period:
PDL as it stands is a serialization of an underlying data model for service parameters (where one could add several types that have been identified as necessary in DAL discussions, e.g., intervals, but that's another matter). I suspect it would be fairly straightforward to formulate the DM itself in VO-DML. The disadvantage would be that the schema generated by VO-DML itself would be different from the hand-written one, so implementations would need an update. On the other hand, the DM is specified explicitely rather than being somewhat implied by the combination of XML Schema and prose, it's much clearer what to do on extension, and you get a VOTable serialization for free, which is really nice when you want to embed PDL in, say, responses of DAL services. -- MarkusDemleitner - 2014-01-14
Sorry for picking this up so late. -- OmarLaurino - 2014-04-23 N.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
|
PDL 1.0 Proposed Recommendation : Request For CommentsThe Community comments period is now finished. The TCG Review was running during 4 weeks from 5 February to 5 March 2014 The comments and answers of the previous period are available in an Archive part of this page to avoid any confusion with the new Community RFC. The very last document is available here : pdf: Last version : 18 May 2014 This document will act as RFC centre for the PDL 1.0 Proposed Recommendation http://www.ivoa.net/documents/PDL/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.
Implementation detailsThe two main implementations are:
TCG Review Period
Comments from the TCG members during the TCG Review period: 5 February to 5 March 2014
Application Working Group ( _Mark Taylor, Pierre Fernique)Some of my concerns from the earlier RFC period have been addressed. Following the comments from Franck et al., I am prepared to believe that PDL is a good fit to the kind of constraints that are required for theoretical services for which this standard is targetted. I am however a bit sceptical of sentences like:
As long as the scope of the current version of this standard is mainly recognised as applying to theoretical type services, Applications WG recommends approval. A few new errors have been introduced since the previous version:
These errors are now fixed --IVOA.AndreSchaaff
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)
3.3, third paragraph, I think radiant should be radians The big one: The document presents a data model for Service, but none of the diagrams that present the model use standard recognisable UML. This makes it very hard for readers to understand the data model. At a minimum, you would need to add a section that explains the diagram syntax and how UML concepts are expressed, but that still forces many readers to learn a new diagram language for this document. Not sure how to resolve this easily...
-- PatrickDowler - 2014-04-29 Approved. -- PatrickDowler - 2014-05-18 Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec)Approved
Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner) | ||||||||
Added: | ||||||||
> > | Registry approves this document GretchenGreene May 18, 2014 | |||||||
Semantics Working Group ( _Norman Gray, Mireille Louys)The semantics working group has only relatively minor comments: Substance:
-- NormanGray - 2014-04-07
Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)As I already said, to my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. When I started to work on Theory at the IVOA (end of 2005), I asked to scientists what they would expect from VO-Theory. For a lot of scientists, that was not to have access to simulations but to be able to give online access to their codes. The interest to do it in a VO context was the hope to be able to pipe scientific codes together. As, for example, to post-process outputs of large MHD simulations (published in a database - so relying on what is today SimDM / SimDAL) with online codes to compute, for example line intensities that they could compare to observations. At this time, the CEA by Astrogrid partly answered the need. It allowed to easily put online a scientific code, but had not detail descriptions for interoperability between applications in a scientific meaning (units, type of data, semantics tags, ...). The CEA has never been standardized, maybe because the funding difficulties of Astrogrid. Today, PDL answers the need. With it, it is now possible to describe a scientific application in a standardized way. It may seems complicate to use. But, in fact, for simple applications, the description is also simple. The standard is just comprehensive enough to allow description of complex applications. I also noticed that in France, several data centers plan to use PDL to describe their applications and give access to them in a VO context. In Spain, people working in European projects about workflows, not directly related to VO, find it useful and integrated it in Taverna for example. I think that most comments done during the Hawaii InterOp have been taken into account in the revised version of the document. For me, we can approve it. -- FranckLePetit - 2014-03-06 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
Archive : Previous Community period for commentsPlease, don't modify the below part of this page.
IVOA Community Comments Period: 24 December 2013 - 21 January 2014
Comments from the IVOA Community during the RFC period:
PDL as it stands is a serialization of an underlying data model for service parameters (where one could add several types that have been identified as necessary in DAL discussions, e.g., intervals, but that's another matter). I suspect it would be fairly straightforward to formulate the DM itself in VO-DML. The disadvantage would be that the schema generated by VO-DML itself would be different from the hand-written one, so implementations would need an update. On the other hand, the DM is specified explicitely rather than being somewhat implied by the combination of XML Schema and prose, it's much clearer what to do on extension, and you get a VOTable serialization for free, which is really nice when you want to embed PDL in, say, responses of DAL services. -- MarkusDemleitner - 2014-01-14
Sorry for picking this up so late. -- OmarLaurino - 2014-04-23 N.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
|
PDL 1.0 Proposed Recommendation : Request For CommentsThe Community comments period is now finished. The TCG Review was running during 4 weeks from 5 February to 5 March 2014 The comments and answers of the previous period are available in an Archive part of this page to avoid any confusion with the new Community RFC. The very last document is available here : pdf: Last version : 18 May 2014 This document will act as RFC centre for the PDL 1.0 Proposed Recommendation http://www.ivoa.net/documents/PDL/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.
Implementation detailsThe two main implementations are:
TCG Review Period
Comments from the TCG members during the TCG Review period: 5 February to 5 March 2014
Application Working Group ( _Mark Taylor, Pierre Fernique)Some of my concerns from the earlier RFC period have been addressed. Following the comments from Franck et al., I am prepared to believe that PDL is a good fit to the kind of constraints that are required for theoretical services for which this standard is targetted. I am however a bit sceptical of sentences like:
As long as the scope of the current version of this standard is mainly recognised as applying to theoretical type services, Applications WG recommends approval. A few new errors have been introduced since the previous version:
These errors are now fixed --IVOA.AndreSchaaff
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)
3.3, third paragraph, I think radiant should be radians The big one: The document presents a data model for Service, but none of the diagrams that present the model use standard recognisable UML. This makes it very hard for readers to understand the data model. At a minimum, you would need to add a section that explains the diagram syntax and how UML concepts are expressed, but that still forces many readers to learn a new diagram language for this document. Not sure how to resolve this easily...
-- PatrickDowler - 2014-04-29 Approved. -- PatrickDowler - 2014-05-18 Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec) | ||||||||
Added: | ||||||||
> > | Approved | |||||||
Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)The semantics working group has only relatively minor comments: Substance:
-- NormanGray - 2014-04-07
Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)As I already said, to my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. When I started to work on Theory at the IVOA (end of 2005), I asked to scientists what they would expect from VO-Theory. For a lot of scientists, that was not to have access to simulations but to be able to give online access to their codes. The interest to do it in a VO context was the hope to be able to pipe scientific codes together. As, for example, to post-process outputs of large MHD simulations (published in a database - so relying on what is today SimDM / SimDAL) with online codes to compute, for example line intensities that they could compare to observations. At this time, the CEA by Astrogrid partly answered the need. It allowed to easily put online a scientific code, but had not detail descriptions for interoperability between applications in a scientific meaning (units, type of data, semantics tags, ...). The CEA has never been standardized, maybe because the funding difficulties of Astrogrid. Today, PDL answers the need. With it, it is now possible to describe a scientific application in a standardized way. It may seems complicate to use. But, in fact, for simple applications, the description is also simple. The standard is just comprehensive enough to allow description of complex applications. I also noticed that in France, several data centers plan to use PDL to describe their applications and give access to them in a VO context. In Spain, people working in European projects about workflows, not directly related to VO, find it useful and integrated it in Taverna for example. I think that most comments done during the Hawaii InterOp have been taken into account in the revised version of the document. For me, we can approve it. -- FranckLePetit - 2014-03-06 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
Archive : Previous Community period for commentsPlease, don't modify the below part of this page.
IVOA Community Comments Period: 24 December 2013 - 21 January 2014
Comments from the IVOA Community during the RFC period:
PDL as it stands is a serialization of an underlying data model for service parameters (where one could add several types that have been identified as necessary in DAL discussions, e.g., intervals, but that's another matter). I suspect it would be fairly straightforward to formulate the DM itself in VO-DML. The disadvantage would be that the schema generated by VO-DML itself would be different from the hand-written one, so implementations would need an update. On the other hand, the DM is specified explicitely rather than being somewhat implied by the combination of XML Schema and prose, it's much clearer what to do on extension, and you get a VOTable serialization for free, which is really nice when you want to embed PDL in, say, responses of DAL services. -- MarkusDemleitner - 2014-01-14
Sorry for picking this up so late. -- OmarLaurino - 2014-04-23 N.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
|
PDL 1.0 Proposed Recommendation : Request For CommentsThe Community comments period is now finished. The TCG Review was running during 4 weeks from 5 February to 5 March 2014 The comments and answers of the previous period are available in an Archive part of this page to avoid any confusion with the new Community RFC. | ||||||||
Changed: | ||||||||
< < | The very last document is available here : pdf: Last version : 16 May 2014 | |||||||
> > | The very last document is available here : pdf: Last version : 18 May 2014 | |||||||
This document will act as RFC centre for the PDL 1.0 Proposed Recommendation http://www.ivoa.net/documents/PDL/
In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.
Implementation detailsThe two main implementations are:
TCG Review Period
Comments from the TCG members during the TCG Review period: 5 February to 5 March 2014
Application Working Group ( _Mark Taylor, Pierre Fernique)Some of my concerns from the earlier RFC period have been addressed. Following the comments from Franck et al., I am prepared to believe that PDL is a good fit to the kind of constraints that are required for theoretical services for which this standard is targetted. I am however a bit sceptical of sentences like:
As long as the scope of the current version of this standard is mainly recognised as applying to theoretical type services, Applications WG recommends approval. A few new errors have been introduced since the previous version:
These errors are now fixed --IVOA.AndreSchaaff
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)
3.3, third paragraph, I think radiant should be radians The big one: The document presents a data model for Service, but none of the diagrams that present the model use standard recognisable UML. This makes it very hard for readers to understand the data model. At a minimum, you would need to add a section that explains the diagram syntax and how UML concepts are expressed, but that still forces many readers to learn a new diagram language for this document. Not sure how to resolve this easily...
-- PatrickDowler - 2014-04-29 Approved. -- PatrickDowler - 2014-05-18 Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)The semantics working group has only relatively minor comments: Substance:
-- NormanGray - 2014-04-07
Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)As I already said, to my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. When I started to work on Theory at the IVOA (end of 2005), I asked to scientists what they would expect from VO-Theory. For a lot of scientists, that was not to have access to simulations but to be able to give online access to their codes. The interest to do it in a VO context was the hope to be able to pipe scientific codes together. As, for example, to post-process outputs of large MHD simulations (published in a database - so relying on what is today SimDM / SimDAL) with online codes to compute, for example line intensities that they could compare to observations. At this time, the CEA by Astrogrid partly answered the need. It allowed to easily put online a scientific code, but had not detail descriptions for interoperability between applications in a scientific meaning (units, type of data, semantics tags, ...). The CEA has never been standardized, maybe because the funding difficulties of Astrogrid. Today, PDL answers the need. With it, it is now possible to describe a scientific application in a standardized way. It may seems complicate to use. But, in fact, for simple applications, the description is also simple. The standard is just comprehensive enough to allow description of complex applications. I also noticed that in France, several data centers plan to use PDL to describe their applications and give access to them in a VO context. In Spain, people working in European projects about workflows, not directly related to VO, find it useful and integrated it in Taverna for example. I think that most comments done during the Hawaii InterOp have been taken into account in the revised version of the document. For me, we can approve it. -- FranckLePetit - 2014-03-06 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
Archive : Previous Community period for commentsPlease, don't modify the below part of this page.
IVOA Community Comments Period: 24 December 2013 - 21 January 2014
Comments from the IVOA Community during the RFC period:
PDL as it stands is a serialization of an underlying data model for service parameters (where one could add several types that have been identified as necessary in DAL discussions, e.g., intervals, but that's another matter). I suspect it would be fairly straightforward to formulate the DM itself in VO-DML. The disadvantage would be that the schema generated by VO-DML itself would be different from the hand-written one, so implementations would need an update. On the other hand, the DM is specified explicitely rather than being somewhat implied by the combination of XML Schema and prose, it's much clearer what to do on extension, and you get a VOTable serialization for free, which is really nice when you want to embed PDL in, say, responses of DAL services. -- MarkusDemleitner - 2014-01-14
Sorry for picking this up so late. -- OmarLaurino - 2014-04-23 N.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) | ||||||||
Deleted: | ||||||||
< < |
| |||||||
|
PDL 1.0 Proposed Recommendation : Request For CommentsThe Community comments period is now finished. The TCG Review was running during 4 weeks from 5 February to 5 March 2014 The comments and answers of the previous period are available in an Archive part of this page to avoid any confusion with the new Community RFC. The very last document is available here : pdf: Last version : 16 May 2014 This document will act as RFC centre for the PDL 1.0 Proposed Recommendation http://www.ivoa.net/documents/PDL/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.
Implementation detailsThe two main implementations are:
TCG Review Period
Comments from the TCG members during the TCG Review period: 5 February to 5 March 2014
Application Working Group ( _Mark Taylor, Pierre Fernique)Some of my concerns from the earlier RFC period have been addressed. Following the comments from Franck et al., I am prepared to believe that PDL is a good fit to the kind of constraints that are required for theoretical services for which this standard is targetted. I am however a bit sceptical of sentences like:
As long as the scope of the current version of this standard is mainly recognised as applying to theoretical type services, Applications WG recommends approval. A few new errors have been introduced since the previous version:
These errors are now fixed --IVOA.AndreSchaaff
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)
3.3, third paragraph, I think radiant should be radians The big one: The document presents a data model for Service, but none of the diagrams that present the model use standard recognisable UML. This makes it very hard for readers to understand the data model. At a minimum, you would need to add a section that explains the diagram syntax and how UML concepts are expressed, but that still forces many readers to learn a new diagram language for this document. Not sure how to resolve this easily...
-- PatrickDowler - 2014-04-29 Approved. -- PatrickDowler - 2014-05-18 Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)The semantics working group has only relatively minor comments: Substance:
-- NormanGray - 2014-04-07
Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)As I already said, to my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. When I started to work on Theory at the IVOA (end of 2005), I asked to scientists what they would expect from VO-Theory. For a lot of scientists, that was not to have access to simulations but to be able to give online access to their codes. The interest to do it in a VO context was the hope to be able to pipe scientific codes together. As, for example, to post-process outputs of large MHD simulations (published in a database - so relying on what is today SimDM / SimDAL) with online codes to compute, for example line intensities that they could compare to observations. At this time, the CEA by Astrogrid partly answered the need. It allowed to easily put online a scientific code, but had not detail descriptions for interoperability between applications in a scientific meaning (units, type of data, semantics tags, ...). The CEA has never been standardized, maybe because the funding difficulties of Astrogrid. Today, PDL answers the need. With it, it is now possible to describe a scientific application in a standardized way. It may seems complicate to use. But, in fact, for simple applications, the description is also simple. The standard is just comprehensive enough to allow description of complex applications. I also noticed that in France, several data centers plan to use PDL to describe their applications and give access to them in a VO context. In Spain, people working in European projects about workflows, not directly related to VO, find it useful and integrated it in Taverna for example. I think that most comments done during the Hawaii InterOp have been taken into account in the revised version of the document. For me, we can approve it. -- FranckLePetit - 2014-03-06 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
Archive : Previous Community period for commentsPlease, don't modify the below part of this page.
IVOA Community Comments Period: 24 December 2013 - 21 January 2014
Comments from the IVOA Community during the RFC period:
PDL as it stands is a serialization of an underlying data model for service parameters (where one could add several types that have been identified as necessary in DAL discussions, e.g., intervals, but that's another matter). I suspect it would be fairly straightforward to formulate the DM itself in VO-DML. The disadvantage would be that the schema generated by VO-DML itself would be different from the hand-written one, so implementations would need an update. On the other hand, the DM is specified explicitely rather than being somewhat implied by the combination of XML Schema and prose, it's much clearer what to do on extension, and you get a VOTable serialization for free, which is really nice when you want to embed PDL in, say, responses of DAL services. -- MarkusDemleitner - 2014-01-14
Sorry for picking this up so late. -- OmarLaurino - 2014-04-23 N.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) | ||||||||
Added: | ||||||||
> > |
| |||||||
| ||||||||
Added: | ||||||||
> > |
| |||||||
PDL 1.0 Proposed Recommendation : Request For CommentsThe Community comments period is now finished. The TCG Review was running during 4 weeks from 5 February to 5 March 2014 The comments and answers of the previous period are available in an Archive part of this page to avoid any confusion with the new Community RFC. The very last document is available here : pdf: Last version : 16 May 2014 This document will act as RFC centre for the PDL 1.0 Proposed Recommendation http://www.ivoa.net/documents/PDL/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.
Implementation detailsThe two main implementations are:
TCG Review Period
Comments from the TCG members during the TCG Review period: 5 February to 5 March 2014
Application Working Group ( _Mark Taylor, Pierre Fernique)Some of my concerns from the earlier RFC period have been addressed. Following the comments from Franck et al., I am prepared to believe that PDL is a good fit to the kind of constraints that are required for theoretical services for which this standard is targetted. I am however a bit sceptical of sentences like:
As long as the scope of the current version of this standard is mainly recognised as applying to theoretical type services, Applications WG recommends approval. A few new errors have been introduced since the previous version:
| ||||||||
Changed: | ||||||||
< < | -- MarkTaylor - 2014-02-07 | |||||||
> > | -- MarkTaylor - 2014-02-07 These errors are now fixed --IVOA.AndreSchaaff | |||||||
Deleted: | ||||||||
< < | These errors are now fixed --IVOA.AndreSchaaff | |||||||
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)
3.3, third paragraph, I think radiant should be radians The big one: The document presents a data model for Service, but none of the diagrams that present the model use standard recognisable UML. This makes it very hard for readers to understand the data model. At a minimum, you would need to add a section that explains the diagram syntax and how UML concepts are expressed, but that still forces many readers to learn a new diagram language for this document. Not sure how to resolve this easily... | ||||||||
Changed: | ||||||||
< < | -- PatrickDowler - 2014-04-29 | |||||||
> > | -- PatrickDowler - 2014-04-29 The comments are taken into account into the last document excepted the last one concerning the use of UML diagrams, we propose to take it into account for a future revision of the document as the current diagrams are human readable. We agree that more standardised ones would be better --IVOA.AndreSchaaff | |||||||
Deleted: | ||||||||
< < | The comments are taken into account into the last document excepted the last one concerning the use of UML diagrams, we propose to take it into account for a future revision of the document as the current diagrams are human readable. We agree that more standardised ones would be better --IVOA.AndreSchaaff | |||||||
Added: | ||||||||
> > | Approved.
-- PatrickDowler - 2014-05-18 | |||||||
Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)The semantics working group has only relatively minor comments: Substance:
| ||||||||
Changed: | ||||||||
< < | -- NormanGray - 2014-04-07 | |||||||
> > | -- NormanGray - 2014-04-07 The comments are taken into account into the last document --IVOA.AndreSchaaff | |||||||
Deleted: | ||||||||
< < | The comments are taken into account into the last document --IVOA.AndreSchaaff | |||||||
Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)As I already said, to my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. When I started to work on Theory at the IVOA (end of 2005), I asked to scientists what they would expect from VO-Theory. For a lot of scientists, that was not to have access to simulations but to be able to give online access to their codes. The interest to do it in a VO context was the hope to be able to pipe scientific codes together. As, for example, to post-process outputs of large MHD simulations (published in a database - so relying on what is today SimDM / SimDAL) with online codes to compute, for example line intensities that they could compare to observations. At this time, the CEA by Astrogrid partly answered the need. It allowed to easily put online a scientific code, but had not detail descriptions for interoperability between applications in a scientific meaning (units, type of data, semantics tags, ...). The CEA has never been standardized, maybe because the funding difficulties of Astrogrid. Today, PDL answers the need. With it, it is now possible to describe a scientific application in a standardized way. It may seems complicate to use. But, in fact, for simple applications, the description is also simple. The standard is just comprehensive enough to allow description of complex applications. I also noticed that in France, several data centers plan to use PDL to describe their applications and give access to them in a VO context. In Spain, people working in European projects about workflows, not directly related to VO, find it useful and integrated it in Taverna for example. I think that most comments done during the Hawaii InterOp have been taken into account in the revised version of the document. For me, we can approve it. -- FranckLePetit - 2014-03-06 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
Archive : Previous Community period for commentsPlease, don't modify the below part of this page.
IVOA Community Comments Period: 24 December 2013 - 21 January 2014
Comments from the IVOA Community during the RFC period: | ||||||||
Changed: | ||||||||
< < | PDL as it stands is a serialization of an underlying data model for service parameters (where one could add several types that have been identified as necessary in DAL discussions, e.g., intervals, but that's another matter). I suspect it would be fairly straightforward to formulate the DM itself in VO-DML. The disadvantage would be that the schema generated by VO-DML itself would be different from the hand-written one, so implementations would need an update. On the other hand, the DM is specified explicitely rather than being somewhat implied by the combination of XML Schema and prose, it's much clearer what to do on extension, and you get a VOTable serialization for free, which is really nice when you want to embed PDL in, say, responses of DAL services. -- MarkusDemleitner - 2014-01-14 | |||||||
> > | PDL as it stands is a serialization of an underlying data model for service parameters (where one could add several types that have been identified as necessary in DAL discussions, e.g., intervals, but that's another matter). I suspect it would be fairly straightforward to formulate the DM itself in VO-DML. The disadvantage would be that the schema generated by VO-DML itself would be different from the hand-written one, so implementations would need an update. On the other hand, the DM is specified explicitely rather than being somewhat implied by the combination of XML Schema and prose, it's much clearer what to do on extension, and you get a VOTable serialization for free, which is really nice when you want to embed PDL in, say, responses of DAL services. -- MarkusDemleitner - 2014-01-14 Answer : Yes PDL can be seen as a serialization of an underlying data model. By the way intervals are actually handled in PDL in a very powerful way (using the ValueInRange object, see paragraph 9.6.6 of the Proposed Recommendation document): one could express not only that a parameter is into a ‘simple’ numerical range, but defining complex intervals, something where INF and SUP limits are defined as functions and/or expressions involving numerical constants and/or the other parameters. From a practical point of view PDL could be expressed with VO-DML but this would imply to totally rewrite the PR document and to change all the existing implementations (and redeploy the service, with related clients, based on the actual implementations). In any case VO-DML would have an impact on other standards and PDL would be just one case. Several implementations are based on the PDL grammar (including some production services). These implementations took 2/3 years of work and in some cases are really complex (e.g. the automatic generator of checking algorithm and the generator of the dynamic interface are more or less compilers of PDL, written in Java). The cost for adapting these implementations would be prohibitive today. -- AndreSchaaff and Carlo Maria Zwolf | |||||||
Deleted: | ||||||||
< < | Answer : Yes PDL can be seen as a serialization of an underlying data model. By the way intervals are actually handled in PDL in a very powerful way (using the ValueInRange object, see paragraph 9.6.6 of the Proposed Recommendation document): one could express not only that a parameter is into a ‘simple’ numerical range, but defining complex intervals, something where INF and SUP limits are defined as functions and/or expressions involving numerical constants and/or the other parameters. From a practical point of view PDL could be expressed with VO-DML but this would imply to totally rewrite the PR document and to change all the existing implementations (and redeploy the service, with related clients, based on the actual implementations). In any case VO-DML would have an impact on other standards and PDL would be just one case. Several implementations are based on the PDL grammar (including some production services). These implementations took 2/3 years of work and in some cases are really complex (e.g. the automatic generator of checking algorithm and the generator of the dynamic interface are more or less compilers of PDL, written in Java). The cost for adapting these implementations would be prohibitive today. -- AndreSchaaff and Carlo Maria Zwolf | |||||||
Sorry for picking this up so late. I agree with Markus that PDL explicitly defines a Data Model and so such DM could be suitable to be expressed in VODML. This doesn't have much to do with the document itself or the existing implementations, though. At least not in my opinion. What Markus suggests may have some advantages: - Serialization Agnosticism: PDL defines an XML schema, so a valid representation of a PDL instance is an XML document. With VODML one could have different serialization (e.g. VOTable) that could be unambiguously mapped to the native XML, and so interoperable among different serializations. - Model reuse: PDL could be reused in other models if needed. This is to the advantage of the community more than of PDL itself. So, the exercise of coming up with a data model description as a VODML file would not require any changes to the current implementations, but would broaden the scope and impact of PDL itself, allowing it to be reused by other models, by applications wanting to standardize the description of the operations they perform on a file according to the user's input, and by services that can embed PDL instances in their responses, even though the responses do not comply with the PDL XSD. This can be done independently of the REC process, in my opinion, for two reasons: i) this was not a DM WG effort, even though it effectively defines a Data Model. ii) PDL was around well before VODML and it would be unfair to stops its REC process because of it. Let me also stress that VODML and PDL share a lot in common and that, although different in scope, they both overcome the same class of interoperability issues in a rigorous, and similar, way. More collaboration between the WGs could probably have benefitted both projects, and there is still time to initiate such collaboration. -- OmarLaurino - 2014-04-23 N.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
|
PDL 1.0 Proposed Recommendation : Request For CommentsThe Community comments period is now finished. The TCG Review was running during 4 weeks from 5 February to 5 March 2014 The comments and answers of the previous period are available in an Archive part of this page to avoid any confusion with the new Community RFC. The very last document is available here : pdf: Last version : 16 May 2014 This document will act as RFC centre for the PDL 1.0 Proposed Recommendation http://www.ivoa.net/documents/PDL/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.
Implementation detailsThe two main implementations are:
TCG Review Period
Comments from the TCG members during the TCG Review period: 5 February to 5 March 2014
Application Working Group ( _Mark Taylor, Pierre Fernique)Some of my concerns from the earlier RFC period have been addressed. Following the comments from Franck et al., I am prepared to believe that PDL is a good fit to the kind of constraints that are required for theoretical services for which this standard is targetted. I am however a bit sceptical of sentences like:
As long as the scope of the current version of this standard is mainly recognised as applying to theoretical type services, Applications WG recommends approval. A few new errors have been introduced since the previous version:
| ||||||||
Changed: | ||||||||
< < | -- MarkTaylor - 2014-02-07 These errors are now fixed --IVOA.AndreSchaaff | |||||||
> > | -- MarkTaylor - 2014-02-07 | |||||||
Added: | ||||||||
> > | These errors are now fixed --IVOA.AndreSchaaff | |||||||
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)
3.3, third paragraph, I think radiant should be radians The big one: The document presents a data model for Service, but none of the diagrams that present the model use standard recognisable UML. This makes it very hard for readers to understand the data model. At a minimum, you would need to add a section that explains the diagram syntax and how UML concepts are expressed, but that still forces many readers to learn a new diagram language for this document. Not sure how to resolve this easily... | ||||||||
Changed: | ||||||||
< < | -- PatrickDowler - 2014-04-29 The comments are taken into account into the last document excepted the last one concerning the use of UML diagrams, we propose to take it into account for a future revision of the document as the current diagrams are human readable. We agree that more standardised ones would be better --IVOA.AndreSchaaff | |||||||
> > | -- PatrickDowler - 2014-04-29
The comments are taken into account into the last document excepted the last one concerning the use of UML diagrams, we propose to take it into account for a future revision of the document as the current diagrams are human readable. We agree that more standardised ones would be better --IVOA.AndreSchaaff | |||||||
Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)The semantics working group has only relatively minor comments: Substance:
| ||||||||
Changed: | ||||||||
< < | -- NormanGray - 2014-04-07 The comments are taken into account into the last document --IVOA.AndreSchaaff | |||||||
> > | -- NormanGray - 2014-04-07
The comments are taken into account into the last document --IVOA.AndreSchaaff | |||||||
Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)As I already said, to my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. When I started to work on Theory at the IVOA (end of 2005), I asked to scientists what they would expect from VO-Theory. For a lot of scientists, that was not to have access to simulations but to be able to give online access to their codes. The interest to do it in a VO context was the hope to be able to pipe scientific codes together. As, for example, to post-process outputs of large MHD simulations (published in a database - so relying on what is today SimDM / SimDAL) with online codes to compute, for example line intensities that they could compare to observations. At this time, the CEA by Astrogrid partly answered the need. It allowed to easily put online a scientific code, but had not detail descriptions for interoperability between applications in a scientific meaning (units, type of data, semantics tags, ...). The CEA has never been standardized, maybe because the funding difficulties of Astrogrid. Today, PDL answers the need. With it, it is now possible to describe a scientific application in a standardized way. It may seems complicate to use. But, in fact, for simple applications, the description is also simple. The standard is just comprehensive enough to allow description of complex applications. I also noticed that in France, several data centers plan to use PDL to describe their applications and give access to them in a VO context. In Spain, people working in European projects about workflows, not directly related to VO, find it useful and integrated it in Taverna for example. I think that most comments done during the Hawaii InterOp have been taken into account in the revised version of the document. For me, we can approve it. -- FranckLePetit - 2014-03-06 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
Archive : Previous Community period for commentsPlease, don't modify the below part of this page.
IVOA Community Comments Period: 24 December 2013 - 21 January 2014
Comments from the IVOA Community during the RFC period:PDL as it stands is a serialization of an underlying data model for service parameters (where one could add several types that have been identified as necessary in DAL discussions, e.g., intervals, but that's another matter). I suspect it would be fairly straightforward to formulate the DM itself in VO-DML. The disadvantage would be that the schema generated by VO-DML itself would be different from the hand-written one, so implementations would need an update. On the other hand, the DM is specified explicitely rather than being somewhat implied by the combination of XML Schema and prose, it's much clearer what to do on extension, and you get a VOTable serialization for free, which is really nice when you want to embed PDL in, say, responses of DAL services. -- MarkusDemleitner - 2014-01-14 | ||||||||
Added: | ||||||||
> > | Answer : Yes PDL can be seen as a serialization of an underlying data model. By the way intervals are actually handled in PDL in a very powerful way (using the ValueInRange object, see paragraph 9.6.6 of the Proposed Recommendation document): one could express not only that a parameter is into a ‘simple’ numerical range, but defining complex intervals, something where INF and SUP limits are defined as functions and/or expressions involving numerical constants and/or the other parameters. From a practical point of view PDL could be expressed with VO-DML but this would imply to totally rewrite the PR document and to change all the existing implementations (and redeploy the service, with related clients, based on the actual implementations). In any case VO-DML would have an impact on other standards and PDL would be just one case. Several implementations are based on the PDL grammar (including some production services). These implementations took 2/3 years of work and in some cases are really complex (e.g. the automatic generator of checking algorithm and the generator of the dynamic interface are more or less compilers of PDL, written in Java). The cost for adapting these implementations would be prohibitive today. -- AndreSchaaff and Carlo Maria Zwolf | |||||||
Deleted: | ||||||||
< < | Answer : Yes PDL can be seen as a serialization of an underlying data model. By the way intervals are actually handled in PDL in a very powerful way (using the ValueInRange object, see paragraph 9.6.6 of the Proposed Recommendation document): one could express not only that a parameter is into a ‘simple’ numerical range, but defining complex intervals, something where INF and SUP limits are defined as functions and/or expressions involving numerical constants and/or the other parameters. From a practical point of view PDL could be expressed with VO-DML but this would imply to totally rewrite the PR document and to change all the existing implementations (and redeploy the service, with related clients, based on the actual implementations). In any case VO-DML would have an impact on other standards and PDL would be just one case. Several implementations are based on the PDL grammar (including some production services). These implementations took 2/3 years of work and in some cases are really complex (e.g. the automatic generator of checking algorithm and the generator of the dynamic interface are more or less compilers of PDL, written in Java). The cost for adapting these implementations would be prohibitive today. -- AndreSchaaff and Carlo Maria Zwolf | |||||||
Sorry for picking this up so late. I agree with Markus that PDL explicitly defines a Data Model and so such DM could be suitable to be expressed in VODML. This doesn't have much to do with the document itself or the existing implementations, though. At least not in my opinion. What Markus suggests may have some advantages: - Serialization Agnosticism: PDL defines an XML schema, so a valid representation of a PDL instance is an XML document. With VODML one could have different serialization (e.g. VOTable) that could be unambiguously mapped to the native XML, and so interoperable among different serializations. - Model reuse: PDL could be reused in other models if needed. This is to the advantage of the community more than of PDL itself. So, the exercise of coming up with a data model description as a VODML file would not require any changes to the current implementations, but would broaden the scope and impact of PDL itself, allowing it to be reused by other models, by applications wanting to standardize the description of the operations they perform on a file according to the user's input, and by services that can embed PDL instances in their responses, even though the responses do not comply with the PDL XSD. This can be done independently of the REC process, in my opinion, for two reasons: i) this was not a DM WG effort, even though it effectively defines a Data Model. ii) PDL was around well before VODML and it would be unfair to stops its REC process because of it. Let me also stress that VODML and PDL share a lot in common and that, although different in scope, they both overcome the same class of interoperability issues in a rigorous, and similar, way. More collaboration between the WGs could probably have benefitted both projects, and there is still time to initiate such collaboration. -- OmarLaurino - 2014-04-23 N.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) | ||||||||
Deleted: | ||||||||
< < | ||||||||
|
PDL 1.0 Proposed Recommendation : Request For CommentsThe Community comments period is now finished. The TCG Review was running during 4 weeks from 5 February to 5 March 2014 The comments and answers of the previous period are available in an Archive part of this page to avoid any confusion with the new Community RFC. | ||||||||
Added: | ||||||||
> > | The very last document is available here : pdf: Last version : 16 May 2014 | |||||||
This document will act as RFC centre for the PDL 1.0 Proposed Recommendation http://www.ivoa.net/documents/PDL/
In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.
Implementation detailsThe two main implementations are:
TCG Review Period
Comments from the TCG members during the TCG Review period: 5 February to 5 March 2014
Application Working Group ( _Mark Taylor, Pierre Fernique)Some of my concerns from the earlier RFC period have been addressed. Following the comments from Franck et al., I am prepared to believe that PDL is a good fit to the kind of constraints that are required for theoretical services for which this standard is targetted. I am however a bit sceptical of sentences like:
As long as the scope of the current version of this standard is mainly recognised as applying to theoretical type services, Applications WG recommends approval. A few new errors have been introduced since the previous version:
These errors are now fixed --IVOA.AndreSchaaff
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)
3.3, third paragraph, I think radiant should be radians The big one: The document presents a data model for Service, but none of the diagrams that present the model use standard recognisable UML. This makes it very hard for readers to understand the data model. At a minimum, you would need to add a section that explains the diagram syntax and how UML concepts are expressed, but that still forces many readers to learn a new diagram language for this document. Not sure how to resolve this easily... -- PatrickDowler - 2014-04-29 | ||||||||
Added: | ||||||||
> > | The comments are taken into account into the last document excepted the last one concerning the use of UML diagrams, we propose to take it into account for a future revision of the document as the current diagrams are human readable. We agree that more standardised ones would be better --IVOA.AndreSchaaff | |||||||
Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)The semantics working group has only relatively minor comments: Substance:
-- NormanGray - 2014-04-07 | ||||||||
Added: | ||||||||
> > | The comments are taken into account into the last document --IVOA.AndreSchaaff | |||||||
Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)As I already said, to my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. When I started to work on Theory at the IVOA (end of 2005), I asked to scientists what they would expect from VO-Theory. For a lot of scientists, that was not to have access to simulations but to be able to give online access to their codes. The interest to do it in a VO context was the hope to be able to pipe scientific codes together. As, for example, to post-process outputs of large MHD simulations (published in a database - so relying on what is today SimDM / SimDAL) with online codes to compute, for example line intensities that they could compare to observations. At this time, the CEA by Astrogrid partly answered the need. It allowed to easily put online a scientific code, but had not detail descriptions for interoperability between applications in a scientific meaning (units, type of data, semantics tags, ...). The CEA has never been standardized, maybe because the funding difficulties of Astrogrid. Today, PDL answers the need. With it, it is now possible to describe a scientific application in a standardized way. It may seems complicate to use. But, in fact, for simple applications, the description is also simple. The standard is just comprehensive enough to allow description of complex applications. I also noticed that in France, several data centers plan to use PDL to describe their applications and give access to them in a VO context. In Spain, people working in European projects about workflows, not directly related to VO, find it useful and integrated it in Taverna for example. I think that most comments done during the Hawaii InterOp have been taken into account in the revised version of the document. For me, we can approve it. -- FranckLePetit - 2014-03-06 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
Archive : Previous Community period for commentsPlease, don't modify the below part of this page.
IVOA Community Comments Period: 24 December 2013 - 21 January 2014
Comments from the IVOA Community during the RFC period:PDL as it stands is a serialization of an underlying data model for service parameters (where one could add several types that have been identified as necessary in DAL discussions, e.g., intervals, but that's another matter). I suspect it would be fairly straightforward to formulate the DM itself in VO-DML. The disadvantage would be that the schema generated by VO-DML itself would be different from the hand-written one, so implementations would need an update. On the other hand, the DM is specified explicitely rather than being somewhat implied by the combination of XML Schema and prose, it's much clearer what to do on extension, and you get a VOTable serialization for free, which is really nice when you want to embed PDL in, say, responses of DAL services. -- MarkusDemleitner - 2014-01-14 Answer : Yes PDL can be seen as a serialization of an underlying data model. By the way intervals are actually handled in PDL in a very powerful way (using the ValueInRange object, see paragraph 9.6.6 of the Proposed Recommendation document): one could express not only that a parameter is into a ‘simple’ numerical range, but defining complex intervals, something where INF and SUP limits are defined as functions and/or expressions involving numerical constants and/or the other parameters. From a practical point of view PDL could be expressed with VO-DML but this would imply to totally rewrite the PR document and to change all the existing implementations (and redeploy the service, with related clients, based on the actual implementations). In any case VO-DML would have an impact on other standards and PDL would be just one case. Several implementations are based on the PDL grammar (including some production services). These implementations took 2/3 years of work and in some cases are really complex (e.g. the automatic generator of checking algorithm and the generator of the dynamic interface are more or less compilers of PDL, written in Java). The cost for adapting these implementations would be prohibitive today. -- AndreSchaaff and Carlo Maria Zwolf
Sorry for picking this up so late. -- OmarLaurino - 2014-04-23 N.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) | ||||||||
Changed: | ||||||||
< < |
| |||||||
> > | ||||||||
|
PDL 1.0 Proposed Recommendation : Request For Comments | ||||||||
Changed: | ||||||||
< < | The Community comments period is now finished. The TCG Review is running during 4 weeks from 5 February to 5 March 2014 | |||||||
> > | The Community comments period is now finished. The TCG Review was running during 4 weeks from 5 February to 5 March 2014 | |||||||
The comments and answers of the previous period are available in an Archive part of this page to avoid any confusion with the new Community RFC.
This document will act as RFC centre for the PDL 1.0 Proposed Recommendation http://www.ivoa.net/documents/PDL/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.
Implementation detailsThe two main implementations are:
| ||||||||
Deleted: | ||||||||
< < | IVOA Community Comments Period: 24 December 2013 - 21 January 2014
Comments from the IVOA Community during the RFC period:PDL as it stands is a serialization of an underlying data model for service parameters (where one could add several types that have been identified as necessary in DAL discussions, e.g., intervals, but that's another matter). I suspect it would be fairly straightforward to formulate the DM itself in VO-DML. The disadvantage would be that the schema generated by VO-DML itself would be different from the hand-written one, so implementations would need an update. On the other hand, the DM is specified explicitely rather than being somewhat implied by the combination of XML Schema and prose, it's much clearer what to do on extension, and you get a VOTable serialization for free, which is really nice when you want to embed PDL in, say, responses of DAL services. -- MarkusDemleitner - 2014-01-14 Answer : Yes PDL can be seen as a serialization of an underlying data model. By the way intervals are actually handled in PDL in a very powerful way (using the ValueInRange object, see paragraph 9.6.6 of the Proposed Recommendation document): one could express not only that a parameter is into a ‘simple’ numerical range, but defining complex intervals, something where INF and SUP limits are defined as functions and/or expressions involving numerical constants and/or the other parameters. From a practical point of view PDL could be expressed with VO-DML but this would imply to totally rewrite the PR document and to change all the existing implementations (and redeploy the service, with related clients, based on the actual implementations). In any case VO-DML would have an impact on other standards and PDL would be just one case. Several implementations are based on the PDL grammar (including some production services). These implementations took 2/3 years of work and in some cases are really complex (e.g. the automatic generator of checking algorithm and the generator of the dynamic interface are more or less compilers of PDL, written in Java). The cost for adapting these implementations would be prohibitive today. -- AndreSchaaff and Carlo Maria Zwolf
Sorry for picking this up so late. -- OmarLaurino - 2014-04-23 | |||||||
TCG Review Period
Comments from the TCG members during the TCG Review period: 5 February to 5 March 2014
Application Working Group ( _Mark Taylor, Pierre Fernique)Some of my concerns from the earlier RFC period have been addressed. Following the comments from Franck et al., I am prepared to believe that PDL is a good fit to the kind of constraints that are required for theoretical services for which this standard is targetted. I am however a bit sceptical of sentences like:
As long as the scope of the current version of this standard is mainly recognised as applying to theoretical type services, Applications WG recommends approval. A few new errors have been introduced since the previous version:
These errors are now fixed --IVOA.AndreSchaaff
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)
3.3, third paragraph, I think radiant should be radians The big one: The document presents a data model for Service, but none of the diagrams that present the model use standard recognisable UML. This makes it very hard for readers to understand the data model. At a minimum, you would need to add a section that explains the diagram syntax and how UML concepts are expressed, but that still forces many readers to learn a new diagram language for this document. Not sure how to resolve this easily... -- PatrickDowler - 2014-04-29 Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)The semantics working group has only relatively minor comments: Substance:
-- NormanGray - 2014-04-07
Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)As I already said, to my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. When I started to work on Theory at the IVOA (end of 2005), I asked to scientists what they would expect from VO-Theory. For a lot of scientists, that was not to have access to simulations but to be able to give online access to their codes. The interest to do it in a VO context was the hope to be able to pipe scientific codes together. As, for example, to post-process outputs of large MHD simulations (published in a database - so relying on what is today SimDM / SimDAL) with online codes to compute, for example line intensities that they could compare to observations. At this time, the CEA by Astrogrid partly answered the need. It allowed to easily put online a scientific code, but had not detail descriptions for interoperability between applications in a scientific meaning (units, type of data, semantics tags, ...). The CEA has never been standardized, maybe because the funding difficulties of Astrogrid. Today, PDL answers the need. With it, it is now possible to describe a scientific application in a standardized way. It may seems complicate to use. But, in fact, for simple applications, the description is also simple. The standard is just comprehensive enough to allow description of complex applications. I also noticed that in France, several data centers plan to use PDL to describe their applications and give access to them in a VO context. In Spain, people working in European projects about workflows, not directly related to VO, find it useful and integrated it in Taverna for example. I think that most comments done during the Hawaii InterOp have been taken into account in the revised version of the document. For me, we can approve it. -- FranckLePetit - 2014-03-06 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
Archive : Previous Community period for commentsPlease, don't modify the below part of this page. | ||||||||
Added: | ||||||||
> > |
IVOA Community Comments Period: 24 December 2013 - 21 January 2014
Comments from the IVOA Community during the RFC period:PDL as it stands is a serialization of an underlying data model for service parameters (where one could add several types that have been identified as necessary in DAL discussions, e.g., intervals, but that's another matter). I suspect it would be fairly straightforward to formulate the DM itself in VO-DML. The disadvantage would be that the schema generated by VO-DML itself would be different from the hand-written one, so implementations would need an update. On the other hand, the DM is specified explicitely rather than being somewhat implied by the combination of XML Schema and prose, it's much clearer what to do on extension, and you get a VOTable serialization for free, which is really nice when you want to embed PDL in, say, responses of DAL services. -- MarkusDemleitner - 2014-01-14 Answer : Yes PDL can be seen as a serialization of an underlying data model. By the way intervals are actually handled in PDL in a very powerful way (using the ValueInRange object, see paragraph 9.6.6 of the Proposed Recommendation document): one could express not only that a parameter is into a ‘simple’ numerical range, but defining complex intervals, something where INF and SUP limits are defined as functions and/or expressions involving numerical constants and/or the other parameters. From a practical point of view PDL could be expressed with VO-DML but this would imply to totally rewrite the PR document and to change all the existing implementations (and redeploy the service, with related clients, based on the actual implementations). In any case VO-DML would have an impact on other standards and PDL would be just one case. Several implementations are based on the PDL grammar (including some production services). These implementations took 2/3 years of work and in some cases are really complex (e.g. the automatic generator of checking algorithm and the generator of the dynamic interface are more or less compilers of PDL, written in Java). The cost for adapting these implementations would be prohibitive today. -- AndreSchaaff and Carlo Maria Zwolf
Sorry for picking this up so late. -- OmarLaurino - 2014-04-23 | |||||||
N.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
|
PDL 1.0 Proposed Recommendation : Request For CommentsThe Community comments period is now finished. The TCG Review is running during 4 weeks from 5 February to 5 March 2014 The comments and answers of the previous period are available in an Archive part of this page to avoid any confusion with the new Community RFC. This document will act as RFC centre for the PDL 1.0 Proposed Recommendation http://www.ivoa.net/documents/PDL/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.
Implementation detailsThe two main implementations are:
IVOA Community Comments Period: 24 December 2013 - 21 January 2014
Comments from the IVOA Community during the RFC period:PDL as it stands is a serialization of an underlying data model for service parameters (where one could add several types that have been identified as necessary in DAL discussions, e.g., intervals, but that's another matter). I suspect it would be fairly straightforward to formulate the DM itself in VO-DML. The disadvantage would be that the schema generated by VO-DML itself would be different from the hand-written one, so implementations would need an update. On the other hand, the DM is specified explicitely rather than being somewhat implied by the combination of XML Schema and prose, it's much clearer what to do on extension, and you get a VOTable serialization for free, which is really nice when you want to embed PDL in, say, responses of DAL services. -- MarkusDemleitner - 2014-01-14 Answer : Yes PDL can be seen as a serialization of an underlying data model. By the way intervals are actually handled in PDL in a very powerful way (using the ValueInRange object, see paragraph 9.6.6 of the Proposed Recommendation document): one could express not only that a parameter is into a ‘simple’ numerical range, but defining complex intervals, something where INF and SUP limits are defined as functions and/or expressions involving numerical constants and/or the other parameters. From a practical point of view PDL could be expressed with VO-DML but this would imply to totally rewrite the PR document and to change all the existing implementations (and redeploy the service, with related clients, based on the actual implementations). In any case VO-DML would have an impact on other standards and PDL would be just one case. Several implementations are based on the PDL grammar (including some production services). These implementations took 2/3 years of work and in some cases are really complex (e.g. the automatic generator of checking algorithm and the generator of the dynamic interface are more or less compilers of PDL, written in Java). The cost for adapting these implementations would be prohibitive today. -- AndreSchaaff and Carlo Maria Zwolf
Sorry for picking this up so late. -- OmarLaurino - 2014-04-23 TCG Review Period
Comments from the TCG members during the TCG Review period: 5 February to 5 March 2014
Application Working Group ( _Mark Taylor, Pierre Fernique)Some of my concerns from the earlier RFC period have been addressed. Following the comments from Franck et al., I am prepared to believe that PDL is a good fit to the kind of constraints that are required for theoretical services for which this standard is targetted. I am however a bit sceptical of sentences like:
As long as the scope of the current version of this standard is mainly recognised as applying to theoretical type services, Applications WG recommends approval. A few new errors have been introduced since the previous version:
These errors are now fixed --IVOA.AndreSchaaff
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)
3.3, third paragraph, I think radiant should be radians The big one: The document presents a data model for Service, but none of the diagrams that present the model use standard recognisable UML. This makes it very hard for readers to understand the data model. At a minimum, you would need to add a section that explains the diagram syntax and how UML concepts are expressed, but that still forces many readers to learn a new diagram language for this document. Not sure how to resolve this easily... -- PatrickDowler - 2014-04-29 Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)The semantics working group has only relatively minor comments: Substance:
-- NormanGray - 2014-04-07
Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)As I already said, to my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. When I started to work on Theory at the IVOA (end of 2005), I asked to scientists what they would expect from VO-Theory. For a lot of scientists, that was not to have access to simulations but to be able to give online access to their codes. The interest to do it in a VO context was the hope to be able to pipe scientific codes together. As, for example, to post-process outputs of large MHD simulations (published in a database - so relying on what is today SimDM / SimDAL) with online codes to compute, for example line intensities that they could compare to observations. At this time, the CEA by Astrogrid partly answered the need. It allowed to easily put online a scientific code, but had not detail descriptions for interoperability between applications in a scientific meaning (units, type of data, semantics tags, ...). The CEA has never been standardized, maybe because the funding difficulties of Astrogrid. Today, PDL answers the need. With it, it is now possible to describe a scientific application in a standardized way. It may seems complicate to use. But, in fact, for simple applications, the description is also simple. The standard is just comprehensive enough to allow description of complex applications. I also noticed that in France, several data centers plan to use PDL to describe their applications and give access to them in a VO context. In Spain, people working in European projects about workflows, not directly related to VO, find it useful and integrated it in Taverna for example. I think that most comments done during the Hawaii InterOp have been taken into account in the revised version of the document. For me, we can approve it. -- FranckLePetit - 2014-03-06 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
Archive : Previous Community period for commentsPlease, don't modify the below part of this page.N.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) | ||||||||
Added: | ||||||||
> > |
| |||||||
PDL 1.0 Proposed Recommendation : Request For CommentsThe Community comments period is now finished. The TCG Review is running during 4 weeks from 5 February to 5 March 2014 The comments and answers of the previous period are available in an Archive part of this page to avoid any confusion with the new Community RFC. This document will act as RFC centre for the PDL 1.0 Proposed Recommendation http://www.ivoa.net/documents/PDL/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.
Implementation detailsThe two main implementations are:
IVOA Community Comments Period: 24 December 2013 - 21 January 2014
Comments from the IVOA Community during the RFC period:PDL as it stands is a serialization of an underlying data model for service parameters (where one could add several types that have been identified as necessary in DAL discussions, e.g., intervals, but that's another matter). I suspect it would be fairly straightforward to formulate the DM itself in VO-DML. The disadvantage would be that the schema generated by VO-DML itself would be different from the hand-written one, so implementations would need an update. On the other hand, the DM is specified explicitely rather than being somewhat implied by the combination of XML Schema and prose, it's much clearer what to do on extension, and you get a VOTable serialization for free, which is really nice when you want to embed PDL in, say, responses of DAL services. -- MarkusDemleitner - 2014-01-14 Answer : Yes PDL can be seen as a serialization of an underlying data model. By the way intervals are actually handled in PDL in a very powerful way (using the ValueInRange object, see paragraph 9.6.6 of the Proposed Recommendation document): one could express not only that a parameter is into a ‘simple’ numerical range, but defining complex intervals, something where INF and SUP limits are defined as functions and/or expressions involving numerical constants and/or the other parameters. From a practical point of view PDL could be expressed with VO-DML but this would imply to totally rewrite the PR document and to change all the existing implementations (and redeploy the service, with related clients, based on the actual implementations). In any case VO-DML would have an impact on other standards and PDL would be just one case. Several implementations are based on the PDL grammar (including some production services). These implementations took 2/3 years of work and in some cases are really complex (e.g. the automatic generator of checking algorithm and the generator of the dynamic interface are more or less compilers of PDL, written in Java). The cost for adapting these implementations would be prohibitive today. -- AndreSchaaff and Carlo Maria Zwolf
Sorry for picking this up so late. -- OmarLaurino - 2014-04-23 TCG Review Period
Comments from the TCG members during the TCG Review period: 5 February to 5 March 2014
Application Working Group ( _Mark Taylor, Pierre Fernique)Some of my concerns from the earlier RFC period have been addressed. Following the comments from Franck et al., I am prepared to believe that PDL is a good fit to the kind of constraints that are required for theoretical services for which this standard is targetted. I am however a bit sceptical of sentences like:
As long as the scope of the current version of this standard is mainly recognised as applying to theoretical type services, Applications WG recommends approval. A few new errors have been introduced since the previous version:
These errors are now fixed --IVOA.AndreSchaaff
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel) | ||||||||
Added: | ||||||||
> > |
3.3, third paragraph, I think radiant should be radians 3.3, fourth paragraph, one would normally say "Astronomy and Astrophysics" not Astronomic 6, section name is ParameterReference while actual class is ParameterRef, use latter as heading 7, Dates: there is a lot of compelxity in expressing Date(time) values in astronomy. Even with an ISO8601 string, you still need to say something about timezones, reference positions, and all that fun STC stuff. The easy way out for services is to restrict to one of each and then forbid specify them in the value. I think the nature of PDL is such that you have to deal with this and not just let services pick their own conventions or rules. 9.1, section name doesn't match class being discussed (also the Figure 6 caption doesnt' match the class name) 9.2, section name (AtomicConstant expression) doesn't match class being discussed (also the Figure 7 caption says AtomicParameter expression) 9.3, section name (parenthesisContent expression should have capital P (also the Figure 8 caption needs a space before ParenthesisContent) 9.4 to 9.7 the section names use "object" when I think it means "class"; in any case that word should be capitalised. In fixing the section headers in 9.1 to 9.3, the same style/structure should be applied (be it Object or Class)... if it was me I would use just the class name in the section headings. Same applies to 10.1 through 10.6.8 Figure 17 caption doesn't mention When It took me a while to guess that Sup and Inf in Figure 25 meant Superior and Inferior. Could the words be spelled out? How about Upper and Lower? The big one: The document presents a data model for Service, but none of the diagrams that present the model use standard recognisable UML. This makes it very hard for readers to understand the data model. At a minimum, you would need to add a section that explains the diagram syntax and how UML concepts are expressed, but that still forces many readers to learn a new diagram language for this document. Not sure how to resolve this easily... -- PatrickDowler - 2014-04-29 | |||||||
Data Model Working Group ( _Jesus Salgado, Omar Laurino) | ||||||||
Changed: | ||||||||
< < | Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_) | |||||||
> > | Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec) | |||||||
Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)The semantics working group has only relatively minor comments: Substance:
-- NormanGray - 2014-04-07
Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)As I already said, to my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. When I started to work on Theory at the IVOA (end of 2005), I asked to scientists what they would expect from VO-Theory. For a lot of scientists, that was not to have access to simulations but to be able to give online access to their codes. The interest to do it in a VO context was the hope to be able to pipe scientific codes together. As, for example, to post-process outputs of large MHD simulations (published in a database - so relying on what is today SimDM / SimDAL) with online codes to compute, for example line intensities that they could compare to observations. At this time, the CEA by Astrogrid partly answered the need. It allowed to easily put online a scientific code, but had not detail descriptions for interoperability between applications in a scientific meaning (units, type of data, semantics tags, ...). The CEA has never been standardized, maybe because the funding difficulties of Astrogrid. Today, PDL answers the need. With it, it is now possible to describe a scientific application in a standardized way. It may seems complicate to use. But, in fact, for simple applications, the description is also simple. The standard is just comprehensive enough to allow description of complex applications. I also noticed that in France, several data centers plan to use PDL to describe their applications and give access to them in a VO context. In Spain, people working in European projects about workflows, not directly related to VO, find it useful and integrated it in Taverna for example. I think that most comments done during the Hawaii InterOp have been taken into account in the revised version of the document. For me, we can approve it. -- FranckLePetit - 2014-03-06 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
Archive : Previous Community period for commentsPlease, don't modify the below part of this page.N.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
PDL 1.0 Proposed Recommendation : Request For CommentsThe Community comments period is now finished. The TCG Review is running during 4 weeks from 5 February to 5 March 2014 The comments and answers of the previous period are available in an Archive part of this page to avoid any confusion with the new Community RFC. This document will act as RFC centre for the PDL 1.0 Proposed Recommendation http://www.ivoa.net/documents/PDL/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.
Implementation detailsThe two main implementations are:
IVOA Community Comments Period: 24 December 2013 - 21 January 2014
Comments from the IVOA Community during the RFC period:PDL as it stands is a serialization of an underlying data model for service parameters (where one could add several types that have been identified as necessary in DAL discussions, e.g., intervals, but that's another matter). I suspect it would be fairly straightforward to formulate the DM itself in VO-DML. The disadvantage would be that the schema generated by VO-DML itself would be different from the hand-written one, so implementations would need an update. On the other hand, the DM is specified explicitely rather than being somewhat implied by the combination of XML Schema and prose, it's much clearer what to do on extension, and you get a VOTable serialization for free, which is really nice when you want to embed PDL in, say, responses of DAL services. -- MarkusDemleitner - 2014-01-14 Answer : Yes PDL can be seen as a serialization of an underlying data model. By the way intervals are actually handled in PDL in a very powerful way (using the ValueInRange object, see paragraph 9.6.6 of the Proposed Recommendation document): one could express not only that a parameter is into a ‘simple’ numerical range, but defining complex intervals, something where INF and SUP limits are defined as functions and/or expressions involving numerical constants and/or the other parameters. From a practical point of view PDL could be expressed with VO-DML but this would imply to totally rewrite the PR document and to change all the existing implementations (and redeploy the service, with related clients, based on the actual implementations). In any case VO-DML would have an impact on other standards and PDL would be just one case. Several implementations are based on the PDL grammar (including some production services). These implementations took 2/3 years of work and in some cases are really complex (e.g. the automatic generator of checking algorithm and the generator of the dynamic interface are more or less compilers of PDL, written in Java). The cost for adapting these implementations would be prohibitive today. -- AndreSchaaff and Carlo Maria Zwolf | ||||||||
Added: | ||||||||
> > | Sorry for picking this up so late. I agree with Markus that PDL explicitly defines a Data Model and so such DM could be suitable to be expressed in VODML. This doesn't have much to do with the document itself or the existing implementations, though. At least not in my opinion. What Markus suggests may have some advantages: - Serialization Agnosticism: PDL defines an XML schema, so a valid representation of a PDL instance is an XML document. With VODML one could have different serialization (e.g. VOTable) that could be unambiguously mapped to the native XML, and so interoperable among different serializations. - Model reuse: PDL could be reused in other models if needed. This is to the advantage of the community more than of PDL itself. So, the exercise of coming up with a data model description as a VODML file would not require any changes to the current implementations, but would broaden the scope and impact of PDL itself, allowing it to be reused by other models, by applications wanting to standardize the description of the operations they perform on a file according to the user's input, and by services that can embed PDL instances in their responses, even though the responses do not comply with the PDL XSD. This can be done independently of the REC process, in my opinion, for two reasons: i) this was not a DM WG effort, even though it effectively defines a Data Model. ii) PDL was around well before VODML and it would be unfair to stops its REC process because of it. Let me also stress that VODML and PDL share a lot in common and that, although different in scope, they both overcome the same class of interoperability issues in a rigorous, and similar, way. More collaboration between the WGs could probably have benefitted both projects, and there is still time to initiate such collaboration. -- OmarLaurino - 2014-04-23 | |||||||
TCG Review Period
Comments from the TCG members during the TCG Review period: 5 February to 5 March 2014
Application Working Group ( _Mark Taylor, Pierre Fernique)Some of my concerns from the earlier RFC period have been addressed. Following the comments from Franck et al., I am prepared to believe that PDL is a good fit to the kind of constraints that are required for theoretical services for which this standard is targetted. I am however a bit sceptical of sentences like:
As long as the scope of the current version of this standard is mainly recognised as applying to theoretical type services, Applications WG recommends approval. A few new errors have been introduced since the previous version:
These errors are now fixed --IVOA.AndreSchaaff
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys) | ||||||||
Deleted: | ||||||||
< < | ||||||||
The semantics working group has only relatively minor comments:
Substance: | ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
Deleted: | ||||||||
< < | rejected'?
| |||||||
| ||||||||
Deleted: | ||||||||
< < | ||||||||
Layout: | ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
| ||||||||
Deleted: | ||||||||
< < | ||||||||
The Semantics Group approves this document in principle, after some suitable resolution of the points above. | ||||||||
Deleted: | ||||||||
< < | ||||||||
-- NormanGray - 2014-04-07
Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)As I already said, to my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. When I started to work on Theory at the IVOA (end of 2005), I asked to scientists what they would expect from VO-Theory. For a lot of scientists, that was not to have access to simulations but to be able to give online access to their codes. The interest to do it in a VO context was the hope to be able to pipe scientific codes together. As, for example, to post-process outputs of large MHD simulations (published in a database - so relying on what is today SimDM / SimDAL) with online codes to compute, for example line intensities that they could compare to observations. At this time, the CEA by Astrogrid partly answered the need. It allowed to easily put online a scientific code, but had not detail descriptions for interoperability between applications in a scientific meaning (units, type of data, semantics tags, ...). The CEA has never been standardized, maybe because the funding difficulties of Astrogrid. Today, PDL answers the need. With it, it is now possible to describe a scientific application in a standardized way. It may seems complicate to use. But, in fact, for simple applications, the description is also simple. The standard is just comprehensive enough to allow description of complex applications. I also noticed that in France, several data centers plan to use PDL to describe their applications and give access to them in a VO context. In Spain, people working in European projects about workflows, not directly related to VO, find it useful and integrated it in Taverna for example. I think that most comments done during the Hawaii InterOp have been taken into account in the revised version of the document. For me, we can approve it. -- FranckLePetit - 2014-03-06 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
Archive : Previous Community period for commentsPlease, don't modify the below part of this page.N.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
Deleted: | ||||||||
< < | ||||||||
PDL 1.0 Proposed Recommendation : Request For CommentsThe Community comments period is now finished. The TCG Review is running during 4 weeks from 5 February to 5 March 2014 The comments and answers of the previous period are available in an Archive part of this page to avoid any confusion with the new Community RFC. This document will act as RFC centre for the PDL 1.0 Proposed Recommendation http://www.ivoa.net/documents/PDL/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.
Implementation detailsThe two main implementations are:
IVOA Community Comments Period: 24 December 2013 - 21 January 2014
Comments from the IVOA Community during the RFC period:PDL as it stands is a serialization of an underlying data model for service parameters (where one could add several types that have been identified as necessary in DAL discussions, e.g., intervals, but that's another matter). I suspect it would be fairly straightforward to formulate the DM itself in VO-DML. The disadvantage would be that the schema generated by VO-DML itself would be different from the hand-written one, so implementations would need an update. On the other hand, the DM is specified explicitely rather than being somewhat implied by the combination of XML Schema and prose, it's much clearer what to do on extension, and you get a VOTable serialization for free, which is really nice when you want to embed PDL in, say, responses of DAL services. -- MarkusDemleitner - 2014-01-14 Answer : Yes PDL can be seen as a serialization of an underlying data model. By the way intervals are actually handled in PDL in a very powerful way (using the ValueInRange object, see paragraph 9.6.6 of the Proposed Recommendation document): one could express not only that a parameter is into a ‘simple’ numerical range, but defining complex intervals, something where INF and SUP limits are defined as functions and/or expressions involving numerical constants and/or the other parameters. From a practical point of view PDL could be expressed with VO-DML but this would imply to totally rewrite the PR document and to change all the existing implementations (and redeploy the service, with related clients, based on the actual implementations). In any case VO-DML would have an impact on other standards and PDL would be just one case. Several implementations are based on the PDL grammar (including some production services). These implementations took 2/3 years of work and in some cases are really complex (e.g. the automatic generator of checking algorithm and the generator of the dynamic interface are more or less compilers of PDL, written in Java). The cost for adapting these implementations would be prohibitive today. -- AndreSchaaff and Carlo Maria Zwolf
TCG Review Period
Comments from the TCG members during the TCG Review period: 5 February to 5 March 2014
Application Working Group ( _Mark Taylor, Pierre Fernique)Some of my concerns from the earlier RFC period have been addressed. Following the comments from Franck et al., I am prepared to believe that PDL is a good fit to the kind of constraints that are required for theoretical services for which this standard is targetted. I am however a bit sceptical of sentences like:
As long as the scope of the current version of this standard is mainly recognised as applying to theoretical type services, Applications WG recommends approval. A few new errors have been introduced since the previous version:
These errors are now fixed --IVOA.AndreSchaaff
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys) | ||||||||
Added: | ||||||||
> > |
The semantics working group has only relatively minor comments: Substance:
Layout:
The Semantics Group approves this document in principle, after some suitable resolution of the points above.
-- NormanGray - 2014-04-07 | |||||||
Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)As I already said, to my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. When I started to work on Theory at the IVOA (end of 2005), I asked to scientists what they would expect from VO-Theory. For a lot of scientists, that was not to have access to simulations but to be able to give online access to their codes. The interest to do it in a VO context was the hope to be able to pipe scientific codes together. As, for example, to post-process outputs of large MHD simulations (published in a database - so relying on what is today SimDM / SimDAL) with online codes to compute, for example line intensities that they could compare to observations. At this time, the CEA by Astrogrid partly answered the need. It allowed to easily put online a scientific code, but had not detail descriptions for interoperability between applications in a scientific meaning (units, type of data, semantics tags, ...). The CEA has never been standardized, maybe because the funding difficulties of Astrogrid. Today, PDL answers the need. With it, it is now possible to describe a scientific application in a standardized way. It may seems complicate to use. But, in fact, for simple applications, the description is also simple. The standard is just comprehensive enough to allow description of complex applications. I also noticed that in France, several data centers plan to use PDL to describe their applications and give access to them in a VO context. In Spain, people working in European projects about workflows, not directly related to VO, find it useful and integrated it in Taverna for example. I think that most comments done during the Hawaii InterOp have been taken into account in the revised version of the document. For me, we can approve it. -- FranckLePetit - 2014-03-06 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
Archive : Previous Community period for commentsPlease, don't modify the below part of this page.N.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
Deleted: | ||||||||
< < | Please go to this page | |||||||
Deleted: | ||||||||
< < | ||||||||
Added: | ||||||||
> > | Please go to this page
| |||||||
PDL 1.0 Proposed Recommendation : Request For CommentsThe Community comments period is now finished. The TCG Review is running during 4 weeks from 5 February to 5 March 2014 The comments and answers of the previous period are available in an Archive part of this page to avoid any confusion with the new Community RFC. This document will act as RFC centre for the PDL 1.0 Proposed Recommendation http://www.ivoa.net/documents/PDL/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.
Implementation detailsThe two main implementations are:
IVOA Community Comments Period: 24 December 2013 - 21 January 2014
Comments from the IVOA Community during the RFC period:PDL as it stands is a serialization of an underlying data model for service parameters (where one could add several types that have been identified as necessary in DAL discussions, e.g., intervals, but that's another matter). I suspect it would be fairly straightforward to formulate the DM itself in VO-DML. The disadvantage would be that the schema generated by VO-DML itself would be different from the hand-written one, so implementations would need an update. On the other hand, the DM is specified explicitely rather than being somewhat implied by the combination of XML Schema and prose, it's much clearer what to do on extension, and you get a VOTable serialization for free, which is really nice when you want to embed PDL in, say, responses of DAL services. -- MarkusDemleitner - 2014-01-14 Answer : Yes PDL can be seen as a serialization of an underlying data model. By the way intervals are actually handled in PDL in a very powerful way (using the ValueInRange object, see paragraph 9.6.6 of the Proposed Recommendation document): one could express not only that a parameter is into a ‘simple’ numerical range, but defining complex intervals, something where INF and SUP limits are defined as functions and/or expressions involving numerical constants and/or the other parameters. From a practical point of view PDL could be expressed with VO-DML but this would imply to totally rewrite the PR document and to change all the existing implementations (and redeploy the service, with related clients, based on the actual implementations). In any case VO-DML would have an impact on other standards and PDL would be just one case. Several implementations are based on the PDL grammar (including some production services). These implementations took 2/3 years of work and in some cases are really complex (e.g. the automatic generator of checking algorithm and the generator of the dynamic interface are more or less compilers of PDL, written in Java). The cost for adapting these implementations would be prohibitive today. -- AndreSchaaff and Carlo Maria Zwolf
TCG Review Period
Comments from the TCG members during the TCG Review period: 5 February to 5 March 2014
Application Working Group ( _Mark Taylor, Pierre Fernique)Some of my concerns from the earlier RFC period have been addressed. Following the comments from Franck et al., I am prepared to believe that PDL is a good fit to the kind of constraints that are required for theoretical services for which this standard is targetted. I am however a bit sceptical of sentences like:
As long as the scope of the current version of this standard is mainly recognised as applying to theoretical type services, Applications WG recommends approval. A few new errors have been introduced since the previous version:
| ||||||||
Changed: | ||||||||
< < | -- MarkTaylor - 2014-02-07 | |||||||
> > | -- MarkTaylor - 2014-02-07 These errors are now fixed --IVOA.AndreSchaaff | |||||||
Deleted: | ||||||||
< < | These errors are now fixed --IVOA.AndreSchaaff | |||||||
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner) | ||||||||
Added: | ||||||||
> > |
As I already said, to my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics.
When I started to work on Theory at the IVOA (end of 2005), I asked to scientists what they would expect from VO-Theory. For a lot of scientists, that was not to have access to simulations but to be able to give online access to their codes. The interest to do it in a VO context was the hope to be able to pipe scientific codes together. As, for example, to post-process outputs of large MHD simulations (published in a database - so relying on what is today SimDM / SimDAL) with online codes to compute, for example line intensities that they could compare to observations. At this time, the CEA by Astrogrid partly answered the need. It allowed to easily put online a scientific code, but had not detail descriptions for interoperability between applications in a scientific meaning (units, type of data, semantics tags, ...). The CEA has never been standardized, maybe because the funding difficulties of Astrogrid. Today, PDL answers the need. With it, it is now possible to describe a scientific application in a standardized way. It may seems complicate to use. But, in fact, for simple applications, the description is also simple. The standard is just comprehensive enough to allow description of complex applications. I also noticed that in France, several data centers plan to use PDL to describe their applications and give access to them in a VO context. In Spain, people working in European projects about workflows, not directly related to VO, find it useful and integrated it in Taverna for example. I think that most comments done during the Hawaii InterOp have been taken into account in the revised version of the document. For me, we can approve it. -- FranckLePetit - 2014-03-06 | |||||||
Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
Archive : Previous Community period for commentsPlease, don't modify the below part of this page.N.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
PDL 1.0 Proposed Recommendation : Request For CommentsThe Community comments period is now finished. The TCG Review is running during 4 weeks from 5 February to 5 March 2014 The comments and answers of the previous period are available in an Archive part of this page to avoid any confusion with the new Community RFC. This document will act as RFC centre for the PDL 1.0 Proposed Recommendation http://www.ivoa.net/documents/PDL/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.
Implementation detailsThe two main implementations are:
IVOA Community Comments Period: 24 December 2013 - 21 January 2014
Comments from the IVOA Community during the RFC period:PDL as it stands is a serialization of an underlying data model for service parameters (where one could add several types that have been identified as necessary in DAL discussions, e.g., intervals, but that's another matter). I suspect it would be fairly straightforward to formulate the DM itself in VO-DML. The disadvantage would be that the schema generated by VO-DML itself would be different from the hand-written one, so implementations would need an update. On the other hand, the DM is specified explicitely rather than being somewhat implied by the combination of XML Schema and prose, it's much clearer what to do on extension, and you get a VOTable serialization for free, which is really nice when you want to embed PDL in, say, responses of DAL services. -- MarkusDemleitner - 2014-01-14 Answer : Yes PDL can be seen as a serialization of an underlying data model. By the way intervals are actually handled in PDL in a very powerful way (using the ValueInRange object, see paragraph 9.6.6 of the Proposed Recommendation document): one could express not only that a parameter is into a ‘simple’ numerical range, but defining complex intervals, something where INF and SUP limits are defined as functions and/or expressions involving numerical constants and/or the other parameters. From a practical point of view PDL could be expressed with VO-DML but this would imply to totally rewrite the PR document and to change all the existing implementations (and redeploy the service, with related clients, based on the actual implementations). In any case VO-DML would have an impact on other standards and PDL would be just one case. Several implementations are based on the PDL grammar (including some production services). These implementations took 2/3 years of work and in some cases are really complex (e.g. the automatic generator of checking algorithm and the generator of the dynamic interface are more or less compilers of PDL, written in Java). The cost for adapting these implementations would be prohibitive today. -- AndreSchaaff and Carlo Maria Zwolf
TCG Review Period
Comments from the TCG members during the TCG Review period: 5 February to 5 March 2014
Application Working Group ( _Mark Taylor, Pierre Fernique)Some of my concerns from the earlier RFC period have been addressed. Following the comments from Franck et al., I am prepared to believe that PDL is a good fit to the kind of constraints that are required for theoretical services for which this standard is targetted. I am however a bit sceptical of sentences like:
| ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
Deleted: | ||||||||
< < | ||||||||
For instance data access services might typically require non-trivial constraints on input/output parameters of type table, image or cube, which would require significant changes.
As long as the scope of the current version of this standard is mainly recognised as applying to theoretical type services, Applications WG recommends approval. A few new errors have been introduced since the previous version:
| ||||||||
Changed: | ||||||||
< < |
-- MarkTaylor - 2014-02-07 | |||||||
> > |
These errors are now fixed --IVOA.AndreSchaaff | |||||||
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
Archive : Previous Community period for commentsPlease, don't modify the below part of this page.N.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
PDL 1.0 Proposed Recommendation : Request For CommentsThe Community comments period is now finished. The TCG Review is running during 4 weeks from 5 February to 5 March 2014 The comments and answers of the previous period are available in an Archive part of this page to avoid any confusion with the new Community RFC. This document will act as RFC centre for the PDL 1.0 Proposed Recommendation http://www.ivoa.net/documents/PDL/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.
Implementation detailsThe two main implementations are:
IVOA Community Comments Period: 24 December 2013 - 21 January 2014
Comments from the IVOA Community during the RFC period:PDL as it stands is a serialization of an underlying data model for service parameters (where one could add several types that have been identified as necessary in DAL discussions, e.g., intervals, but that's another matter). I suspect it would be fairly straightforward to formulate the DM itself in VO-DML. The disadvantage would be that the schema generated by VO-DML itself would be different from the hand-written one, so implementations would need an update. On the other hand, the DM is specified explicitely rather than being somewhat implied by the combination of XML Schema and prose, it's much clearer what to do on extension, and you get a VOTable serialization for free, which is really nice when you want to embed PDL in, say, responses of DAL services. -- MarkusDemleitner - 2014-01-14 Answer : Yes PDL can be seen as a serialization of an underlying data model. By the way intervals are actually handled in PDL in a very powerful way (using the ValueInRange object, see paragraph 9.6.6 of the Proposed Recommendation document): one could express not only that a parameter is into a ‘simple’ numerical range, but defining complex intervals, something where INF and SUP limits are defined as functions and/or expressions involving numerical constants and/or the other parameters. From a practical point of view PDL could be expressed with VO-DML but this would imply to totally rewrite the PR document and to change all the existing implementations (and redeploy the service, with related clients, based on the actual implementations). In any case VO-DML would have an impact on other standards and PDL would be just one case. Several implementations are based on the PDL grammar (including some production services). These implementations took 2/3 years of work and in some cases are really complex (e.g. the automatic generator of checking algorithm and the generator of the dynamic interface are more or less compilers of PDL, written in Java). The cost for adapting these implementations would be prohibitive today. -- AndreSchaaff and Carlo Maria Zwolf
TCG Review Period
Comments from the TCG members during the TCG Review period: 5 February to 5 March 2014
Application Working Group ( _Mark Taylor, Pierre Fernique) | ||||||||
Added: | ||||||||
> > |
Some of my concerns from the earlier RFC period have been addressed.
Following the comments from Franck et al., I am prepared to believe that PDL is a good fit to the kind of constraints that are required for theoretical services for which this standard is targetted. I am however a bit sceptical of sentences like:
For instance data access services might typically require non-trivial constraints on input/output parameters of type table, image or cube, which would require significant changes. As long as the scope of the current version of this standard is mainly recognised as applying to theoretical type services, Applications WG recommends approval. A few new errors have been introduced since the previous version:
-- MarkTaylor - 2014-02-07 | |||||||
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
Archive : Previous Community period for commentsPlease, don't modify the below part of this page.N.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
PDL 1.0 Proposed Recommendation : Request For Comments | ||||||||
Changed: | ||||||||
< < | The Community comments period is now finished. The TCG Review is running during 4 weeks from 5 February to 5 March 2014 | |||||||
> > | The Community comments period is now finished. The TCG Review is running during 4 weeks from 5 February to 5 March 2014 | |||||||
The comments and answers of the previous period are available in an Archive part of this page to avoid any confusion with the new Community RFC.
This document will act as RFC centre for the PDL 1.0 Proposed Recommendation http://www.ivoa.net/documents/PDL/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.
Implementation detailsThe two main implementations are:
IVOA Community Comments Period: 24 December 2013 - 21 January 2014
Comments from the IVOA Community during the RFC period:PDL as it stands is a serialization of an underlying data model for service parameters (where one could add several types that have been identified as necessary in DAL discussions, e.g., intervals, but that's another matter). I suspect it would be fairly straightforward to formulate the DM itself in VO-DML. The disadvantage would be that the schema generated by VO-DML itself would be different from the hand-written one, so implementations would need an update. On the other hand, the DM is specified explicitely rather than being somewhat implied by the combination of XML Schema and prose, it's much clearer what to do on extension, and you get a VOTable serialization for free, which is really nice when you want to embed PDL in, say, responses of DAL services. -- MarkusDemleitner - 2014-01-14 Answer : Yes PDL can be seen as a serialization of an underlying data model. By the way intervals are actually handled in PDL in a very powerful way (using the ValueInRange object, see paragraph 9.6.6 of the Proposed Recommendation document): one could express not only that a parameter is into a ‘simple’ numerical range, but defining complex intervals, something where INF and SUP limits are defined as functions and/or expressions involving numerical constants and/or the other parameters. From a practical point of view PDL could be expressed with VO-DML but this would imply to totally rewrite the PR document and to change all the existing implementations (and redeploy the service, with related clients, based on the actual implementations). In any case VO-DML would have an impact on other standards and PDL would be just one case. Several implementations are based on the PDL grammar (including some production services). These implementations took 2/3 years of work and in some cases are really complex (e.g. the automatic generator of checking algorithm and the generator of the dynamic interface are more or less compilers of PDL, written in Java). The cost for adapting these implementations would be prohibitive today. -- AndreSchaaff and Carlo Maria Zwolf
TCG Review Period
Comments from the TCG members during the TCG Review period: 5 February to 5 March 2014
Application Working Group ( _Mark Taylor, Pierre Fernique)Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
Archive : Previous Community period for commentsPlease, don't modify the below part of this page.N.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
PDL 1.0 Proposed Recommendation : Request For Comments | ||||||||
Changed: | ||||||||
< < | Following the last TCG telecon it has been decided to start a new Community RFC process. The document has been cleaned and updated. | |||||||
> > | The Community comments period is now finished. The TCG Review is running during 4 weeks from 5 February to 5 March 2014 | |||||||
Deleted: | ||||||||
< < | ||||||||
Changed: | ||||||||
< < | The comments and answers of the previous periods are available in an Archive part of this page to avoid any confusion with the new Community RFC. | |||||||
> > | The comments and answers of the previous period are available in an Archive part of this page to avoid any confusion with the new Community RFC. | |||||||
This document will act as RFC centre for the PDL 1.0 Proposed Recommendation http://www.ivoa.net/documents/PDL/
In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.
Implementation detailsThe two main implementations are:
| ||||||||
Changed: | ||||||||
< < | IVOA Community Comments Period: 24 December 2013 - 21 January 2014 | |||||||
> > | IVOA Community Comments Period: 24 December 2013 - 21 January 2014 | |||||||
Changed: | ||||||||
< < | Comments from the IVOA Community during the RFC period: | |||||||
> > | Comments from the IVOA Community during the RFC period: | |||||||
PDL as it stands is a serialization of an underlying data model for service parameters (where one could add several types that have been identified as necessary in DAL discussions, e.g., intervals, but that's another matter). I suspect it would be fairly straightforward to formulate the DM itself in VO-DML. The disadvantage would be that the schema generated by VO-DML itself would be different from the hand-written one, so implementations would need an update. On the other hand, the DM is specified explicitely rather than being somewhat implied by the combination of XML Schema and prose, it's much clearer what to do on extension, and you get a VOTable serialization for free, which is really nice when you want to embed PDL in, say, responses of DAL services. -- MarkusDemleitner - 2014-01-14 | ||||||||
Changed: | ||||||||
< < | Answer : | |||||||
> > | Answer : Yes PDL can be seen as a serialization of an underlying data model. By the way intervals are actually handled in PDL in a very powerful way (using the ValueInRange object, see paragraph 9.6.6 of the Proposed Recommendation document): one could express not only that a parameter is into a ‘simple’ numerical range, but defining complex intervals, something where INF and SUP limits are defined as functions and/or expressions involving numerical constants and/or the other parameters. From a practical point of view PDL could be expressed with VO-DML but this would imply to totally rewrite the PR document and to change all the existing implementations (and redeploy the service, with related clients, based on the actual implementations). In any case VO-DML would have an impact on other standards and PDL would be just one case. Several implementations are based on the PDL grammar (including some production services). These implementations took 2/3 years of work and in some cases are really complex (e.g. the automatic generator of checking algorithm and the generator of the dynamic interface are more or less compilers of PDL, written in Java). The cost for adapting these implementations would be prohibitive today. -- AndreSchaaff and Carlo Maria Zwolf | |||||||
Deleted: | ||||||||
< < | Yes PDL can be seen as a serialization of an underlying data model. By the way intervals are actually handled in PDL in a very powerful way (using the ValueInRange object, see paragraph 9.6.6 of the Proposed Recommendation document): one could express not only that a parameter is into a ‘simple’ numerical range, but defining complex intervals, something where INF and SUP limits are defined as functions and/or expressions involving numerical constants and/or the other parameters. From a practical point of view PDL could be expressed with VO-DML but this would imply to totally rewrite the PR document and to change all the existing implementations (and redeploy the service, with related clients, based on the actual implementations). In any case VO-DML would have an impact on other standards and PDL would be just one case. Several implementations are based on the PDL grammar (including some production services). These implementations took 2/3 years of work and in some cases are really complex (e.g. the automatic generator of checking algorithm and the generator of the dynamic interface are more or less compilers of PDL, written in Java). The cost for adapting these implementations would be prohibitive today. -- AndreSchaaff and Carlo Maria Zwolf | |||||||
Changed: | ||||||||
< < | TCG Review Period | |||||||
> > | TCG Review Period | |||||||
Changed: | ||||||||
< < | Comments from TCG members during the TCG Review period: date depending of the Community comments | |||||||
> > | Comments from the TCG members during the TCG Review period: 5 February to 5 March 2014 | |||||||
Application Working Group ( _Mark Taylor, Pierre Fernique)Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) | ||||||||
Changed: | ||||||||
< < | Archive : Previous Review period | |||||||
> > | Archive : Previous Community period for comments | |||||||
Please, don't modify the below part of this page.N.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
PDL 1.0 Proposed Recommendation : Request For CommentsFollowing the last TCG telecon it has been decided to start a new Community RFC process. The document has been cleaned and updated. The comments and answers of the previous periods are available in an Archive part of this page to avoid any confusion with the new Community RFC. This document will act as RFC centre for the PDL 1.0 Proposed Recommendation http://www.ivoa.net/documents/PDL/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.
Implementation detailsThe two main implementations are:
IVOA Community Comments Period: 24 December 2013 - 21 January 2014 | ||||||||
Deleted: | ||||||||
< < | PDL as it stands is a serialization of an underlying data model for service parameters (where one could add several types that have been identified as necessary in DAL discussions, e.g., intervals, but that's another matter). I suspect it would be fairly straightforward to formulate the DM itself in VO-DML. The disadvantage would be that the schema generated by VO-DML itself would be different from the hand-written one, so implementations would need an update. On the other hand, the DM is specified explicitely rather than being somewhat implied by the combination of XML Schema and prose, it's much clearer what to do on extension, and you get a VOTable serialization for free, which is really nice when you want to embed PDL in, say, responses of DAL services. -- MarkusDemleitner - 2014-01-14 | |||||||
Comments from the IVOA Community during the RFC period: | ||||||||
Changed: | ||||||||
< < | ... | |||||||
> > | PDL as it stands is a serialization of an underlying data model for service parameters (where one could add several types that have been identified as necessary in DAL discussions, e.g., intervals, but that's another matter). I suspect it would be fairly straightforward to formulate the DM itself in VO-DML. The disadvantage would be that the schema generated by VO-DML itself would be different from the hand-written one, so implementations would need an update. On the other hand, the DM is specified explicitely rather than being somewhat implied by the combination of XML Schema and prose, it's much clearer what to do on extension, and you get a VOTable serialization for free, which is really nice when you want to embed PDL in, say, responses of DAL services. -- MarkusDemleitner - 2014-01-14 | |||||||
Added: | ||||||||
> > | Answer : Yes PDL can be seen as a serialization of an underlying data model. By the way intervals are actually handled in PDL in a very powerful way (using the ValueInRange object, see paragraph 9.6.6 of the Proposed Recommendation document): one could express not only that a parameter is into a ‘simple’ numerical range, but defining complex intervals, something where INF and SUP limits are defined as functions and/or expressions involving numerical constants and/or the other parameters. From a practical point of view PDL could be expressed with VO-DML but this would imply to totally rewrite the PR document and to change all the existing implementations (and redeploy the service, with related clients, based on the actual implementations). In any case VO-DML would have an impact on other standards and PDL would be just one case. Several implementations are based on the PDL grammar (including some production services). These implementations took 2/3 years of work and in some cases are really complex (e.g. the automatic generator of checking algorithm and the generator of the dynamic interface are more or less compilers of PDL, written in Java). The cost for adapting these implementations would be prohibitive today. -- AndreSchaaff and Carlo Maria Zwolf | |||||||
TCG Review Period
Comments from TCG members during the TCG Review period: date depending of the Community comments
Application Working Group ( _Mark Taylor, Pierre Fernique)Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
Archive : Previous Review periodPlease, don't modify the below part of this page.N.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
Changed: | ||||||||
< < | PDL GWS RFC | |||||||
> > | PDL 1.0 Proposed Recommendation : Request For Comments | |||||||
Following the last TCG telecon it has been decided to start a new Community RFC process. The document has been cleaned and updated.
The comments and answers of the previous periods are available in an Archive part of this page to avoid any confusion with the new Community RFC. | ||||||||
Changed: | ||||||||
< < | This document will act as RFC centre for the PDL GWS http://www.ivoa.net/documents/PDL/ | |||||||
> > | This document will act as RFC centre for the PDL 1.0 Proposed Recommendation http://www.ivoa.net/documents/PDL/ | |||||||
In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.
Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.
Implementation detailsThe two main implementations are:
IVOA Community Comments Period: 24 December 2013 - 21 January 2014 | ||||||||
Changed: | ||||||||
< < | PDL as it stands is a serialization of an underlying data model for | |||||||
> > | PDL as it stands is a serialization of an underlying data model for service parameters (where one could add several types that have been identified as necessary in DAL discussions, e.g., intervals, but that's another matter). I suspect it would be fairly straightforward to formulate the DM itself in VO-DML. The disadvantage would be that the schema generated by VO-DML itself would be different from the hand-written one, so implementations would need an update. On the other hand, the DM is specified explicitely rather than being somewhat implied by the combination of XML Schema and prose, it's much clearer what to do on extension, and you get a VOTable serialization for free, which is really nice when you want to embed PDL in, say, responses of DAL services. -- MarkusDemleitner - 2014-01-14 | |||||||
Deleted: | ||||||||
< < | service parameters (where one could add several types that have been identified as necessary in DAL discussions, e.g., intervals, but that's another matter). I suspect it would be fairly straightforward to formulate the DM itself in VO-DML. The disadvantage would be that the schema generated by VO-DML itself would be different from the hand-written one, so implementations would need an update. On the other hand, the DM is specified explicitely rather than being somewhat implied by the combination of XML Schema and prose, it's much clearer what to do on extension, and you get a VOTable serialization for free, which is really nice when you want to embed PDL in, say, responses of DAL services. -- MarkusDemleitner - 2014-01-14 | |||||||
Comments from the IVOA Community during the RFC period:...
TCG Review Period
Comments from TCG members during the TCG Review period: date depending of the Community comments
Application Working Group ( _Mark Taylor, Pierre Fernique)Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
Archive : Previous Review periodPlease, don't modify the below part of this page.N.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
PDL GWS RFCFollowing the last TCG telecon it has been decided to start a new Community RFC process. The document has been cleaned and updated. The comments and answers of the previous periods are available in an Archive part of this page to avoid any confusion with the new Community RFC. This document will act as RFC centre for the PDL GWS http://www.ivoa.net/documents/PDL/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.
Implementation detailsThe two main implementations are:
IVOA Community Comments Period: 24 December 2013 - 21 January 2014 | ||||||||
Added: | ||||||||
> > | PDL as it stands is a serialization of an underlying data model for service parameters (where one could add several types that have been identified as necessary in DAL discussions, e.g., intervals, but that's another matter). I suspect it would be fairly straightforward to formulate the DM itself in VO-DML. The disadvantage would be that the schema generated by VO-DML itself would be different from the hand-written one, so implementations would need an update. On the other hand, the DM is specified explicitely rather than being somewhat implied by the combination of XML Schema and prose, it's much clearer what to do on extension, and you get a VOTable serialization for free, which is really nice when you want to embed PDL in, say, responses of DAL services. -- MarkusDemleitner - 2014-01-14 | |||||||
Comments from the IVOA Community during the RFC period:...
TCG Review Period
Comments from TCG members during the TCG Review period: date depending of the Community comments
Application Working Group ( _Mark Taylor, Pierre Fernique)Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
Archive : Previous Review periodPlease, don't modify the below part of this page.N.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
PDL GWS RFC | ||||||||
Changed: | ||||||||
< < | Following the comments during the last Review period, it has been decided to extend it RFC process. The document has been cleaned and updated. We are waiting on its publication.The Community Review will restart juste after and for 4 weeks. | |||||||
> > | Following the last TCG telecon it has been decided to start a new Community RFC process. The document has been cleaned and updated. | |||||||
Changed: | ||||||||
< < | The comments and answers of the previous Reviews are available in an Archive part of this page to avoid any confusion with the new Review period. | |||||||
> > | The comments and answers of the previous periods are available in an Archive part of this page to avoid any confusion with the new Community RFC. | |||||||
Changed: | ||||||||
< < | This document will act as RFC centre for the PDL GWS http://www.ivoa.net/documents/PDL/ (it has been proposed for publication, it will be available soon) | |||||||
> > | This document will act as RFC centre for the PDL GWS http://www.ivoa.net/documents/PDL/ | |||||||
In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.
Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.
Implementation detailsThe two main implementations are:
| ||||||||
Changed: | ||||||||
< < | IVOA Community Review Period: 20 December 2013 - 17 January 2014 | |||||||
> > | IVOA Community Comments Period: 24 December 2013 - 21 January 2014 | |||||||
Comments from the IVOA Community during the RFC period: | ||||||||
Added: | ||||||||
> > | ... | |||||||
TCG Review Period | ||||||||
Changed: | ||||||||
< < | Comments from TCG members during the TCG Review period: date depending of the Community Review | |||||||
> > | Comments from TCG members during the TCG Review period: date depending of the Community comments | |||||||
Application Working Group ( _Mark Taylor, Pierre Fernique)Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) | ||||||||
Changed: | ||||||||
< < | Previous Review period | |||||||
> > | Archive : Previous Review period | |||||||
Added: | ||||||||
> > | Please, don't modify the below part of this page. | |||||||
N.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
PDL GWS RFCFollowing the comments during the last Review period, it has been decided to extend it RFC process. The document has been cleaned and updated. We are waiting on its publication.The Community Review will restart juste after and for 4 weeks. The comments and answers of the previous Reviews are available in an Archive part of this page to avoid any confusion with the new Review period. | ||||||||
Changed: | ||||||||
< < | This document will act as RFC centre for the PDL GWS http://www.ivoa.net/documents/PDL/ has been proposed for publication, it will be available soon | |||||||
> > | This document will act as RFC centre for the PDL GWS http://www.ivoa.net/documents/PDL/ (it has been proposed for publication, it will be available soon) | |||||||
In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.
Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.
Implementation detailsThe two main implementations are:
IVOA Community Review Period: 20 December 2013 - 17 January 2014
Comments from the IVOA Community during the RFC period:
TCG Review Period
Comments from TCG members during the TCG Review period: date depending of the Community Review
Application Working Group ( _Mark Taylor, Pierre Fernique)Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
Previous Review periodN.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
PDL GWS RFC | ||||||||
Changed: | ||||||||
< < | Following the comments during the last Review period, it has been decided to extend the Review. The document is under cleaning and updating and the Community Review will restart the 20 December 2013 for 4 weeks. | |||||||
> > | Following the comments during the last Review period, it has been decided to extend it RFC process. The document has been cleaned and updated. We are waiting on its publication.The Community Review will restart juste after and for 4 weeks. | |||||||
The comments and answers of the previous Reviews are available in an Archive part of this page to avoid any confusion with the new Review period. | ||||||||
Changed: | ||||||||
< < | This document will act as RFC centre for the PDL GWS http://www.ivoa.net/documents/PDL/ will be updated on Friday 20 December 2013 | |||||||
> > | This document will act as RFC centre for the PDL GWS http://www.ivoa.net/documents/PDL/ has been proposed for publication, it will be available soon | |||||||
In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.
Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.
Implementation detailsThe two main implementations are:
IVOA Community Review Period: 20 December 2013 - 17 January 2014
Comments from the IVOA Community during the RFC period:
TCG Review Period
Comments from TCG members during the TCG Review period: date depending of the Community Review
Application Working Group ( _Mark Taylor, Pierre Fernique)Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
Previous Review periodN.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
PDL GWS RFCFollowing the comments during the last Review period, it has been decided to extend the Review. The document is under cleaning and updating and the Community Review will restart the 20 December 2013 for 4 weeks. The comments and answers of the previous Reviews are available in an Archive part of this page to avoid any confusion with the new Review period. This document will act as RFC centre for the PDL GWS http://www.ivoa.net/documents/PDL/ will be updated on Friday 20 December 2013 In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document.
Implementation details | ||||||||
Changed: | ||||||||
< < |
| |||||||
> > | The two main implementations are:
| |||||||
Added: | ||||||||
> > |
| |||||||
IVOA Community Review Period: 20 December 2013 - 17 January 2014
Comments from the IVOA Community during the RFC period:
TCG Review Period
Comments from TCG members during the TCG Review period: date depending of the Community Review
Application Working Group ( _Mark Taylor, Pierre Fernique)Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
Previous Review periodN.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
PDL GWS RFC | ||||||||
Changed: | ||||||||
< < | N.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version | |||||||
> > | Following the comments during the last Review period, it has been decided to extend the Review. The document is under cleaning and updating and the Community Review will restart the 20 December 2013 for 4 weeks. | |||||||
Added: | ||||||||
> > | ||||||||
Changed: | ||||||||
< < | This document will act as RFC centre for the PDL GWS http://www.ivoa.net/documents/PDL/ | |||||||
> > | The comments and answers of the previous Reviews are available in an Archive part of this page to avoid any confusion with the new Review period. | |||||||
Added: | ||||||||
> > | This document will act as RFC centre for the PDL GWS http://www.ivoa.net/documents/PDL/ will be updated on Friday 20 December 2013 | |||||||
In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.
Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document. | ||||||||
Added: | ||||||||
> > | ||||||||
Implementation details
| ||||||||
Added: | ||||||||
> > |
IVOA Community Review Period: 20 December 2013 - 17 January 2014
Comments from the IVOA Community during the RFC period:
TCG Review Period
Comments from TCG members during the TCG Review period: date depending of the Community Review
Application Working Group ( _Mark Taylor, Pierre Fernique)Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick)
Previous Review periodN.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version | |||||||
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)
Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
PDL GWS RFCN.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version This document will act as RFC centre for the PDL GWS http://www.ivoa.net/documents/PDL/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document. Implementation details
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique) | ||||||||
Changed: | ||||||||
< < | Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. | |||||||
> > | Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf | |||||||
Deleted: | ||||||||
< < | Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf | |||||||
General Comments
It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
| ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
-- MarkTaylor - 2013-12-10
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino) | ||||||||
Changed: | ||||||||
< < | Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec) | |||||||
> > | Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec_) | |||||||
Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
PDL GWS RFCN.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version This document will act as RFC centre for the PDL GWS http://www.ivoa.net/documents/PDL/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document. Implementation details
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique) | ||||||||
Added: | ||||||||
> > | Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. Answers in red following a checking point per point by A. Schaaff / C.M. Zwölf | |||||||
General Comments
It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. | ||||||||
Deleted: | ||||||||
< < | Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. | |||||||
Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
--- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension> | ||||||||
Changed: | ||||||||
< < | which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a SingleParameter but was added in a subclass). | |||||||
> > | which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a SingleParameter but was added in a subclass). ---in a primitive version of PDL this was not so ‘boilerplate’. Then it appeared that the dimension of a vector parameter must be an expression. Consider for example a service computing polynomial interpolations. A first parameter could be the degree of the polynomial and the second parameter a vector containing the set of points. For this simple case one has to express the constraint that the degree of the polynomial must be equal to the size of the vector. This kind of argument explains why the dimension is an expression. Of course in a simple case this could be complex, just for saying that you have a scalar parameter. | |||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
PDL GWS RFCN.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version This document will act as RFC centre for the PDL GWS http://www.ivoa.net/documents/PDL/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document. Implementation details
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. | ||||||||
Added: | ||||||||
> > | Global remark: The need for this a-priori verification (and guideline for user) tools comes from services exposing theoretical simulation. If the parameters are decomposed with the finest granularity, PDL is a good tool for performing à priori verification, notifying errors to user before submitting jobs to a server system (cf. the Paris-Durham shock service, presented at the Heidelberg interop during the Application session). Comments will be taken into account and minor corrections or modifications will be done in this first version, others should be pushed to a next release. | |||||||
Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements: | ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
In order to cope with requirements along those lines PDL would have to become considerably more complicated, and it's not clear to me that such complication is justified by the usefulness of having validation done in a service description. I would have thought a more profitable approach would be to have a much simpler form of parameter description (less detail, especially in numerical constraints, than the existing PDL draft) and encourage services to issue helpful error messages when invalid parameter sets are presented. | ||||||||
Added: | ||||||||
> > | --- the need for numerical constraints comes from theoretical services, where often users need help for filling the set of parameters | |||||||
The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance.
However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>
which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)The new document has been completed accordingly to the remarks done at the Hawaii InterOp. PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. | ||||||||
Changed: | ||||||||
< < | So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully | |||||||
> > | So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully | |||||||
-- FranckLePetit - 2013-12-15
Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
PDL GWS RFCN.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version This document will act as RFC centre for the PDL GWS http://www.ivoa.net/documents/PDL/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document. Implementation details
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)General Comments It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. | ||||||||
Changed: | ||||||||
< < | However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. | |||||||
> > | However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. | |||||||
Changed: | ||||||||
< < | Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." | |||||||
> > | Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements: | |||||||
Deleted: | ||||||||
< < | but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements: | |||||||
| ||||||||
Added: | ||||||||
> > | In order to cope with requirements along those lines PDL would have to become considerably more complicated, and it's not clear to me that such complication is justified by the usefulness of having validation done in a service description. I would have thought a more profitable approach would be to have a much simpler form of parameter description (less detail, especially in numerical constraints, than the existing PDL draft) and encourage services to issue helpful error messages when invalid parameter sets are presented. | |||||||
Changed: | ||||||||
< < | In order to cope with requirements along those lines PDL would have to become considerably more complicated, and it's not clear to me that such complication is justified by the usefulness of having validation done in a service description. I would have thought a more profitable approach would be to have a much simpler form of parameter description (less detail, especially in numerical constraints, than the existing PDL draft) and encourage services to issue helpful error messages when invalid parameter sets are presented. | |||||||
> > | The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. | |||||||
Deleted: | ||||||||
< < | The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. | |||||||
However, I do have some more detailed comments which I would like the authors to address.
Detailed comments | ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension> | ||||||||
Deleted: | ||||||||
< < | which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a SingleParameter but was added in a subclass). | |||||||
Changed: | ||||||||
< < |
| |||||||
> > | which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a SingleParameter but was added in a subclass). | |||||||
Added: | ||||||||
> > |
| |||||||
| ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
| ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
| ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
Deleted: | ||||||||
< < | ||||||||
-- MarkTaylor - 2013-12-10 | ||||||||
Deleted: | ||||||||
< < | ||||||||
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner) | ||||||||
Added: | ||||||||
> > |
The new document has been completed accordingly to the remarks done at the Hawaii InterOp.
PDL should be useful in short term to provide a frame to publish online applications. One of the benefit to have such a frame is that applications will be able to communicate (computer - computer communication). Precise description of parameters of an application taking into account relationships between parameters is something that does not exist up to now, and that is very useful to scientists wishing to publish their applications. PDL description may appear complex but that is just because it is quite complete. For simple applications, their XML description is also simple. So the complexity of the standard should not be a blocking point. Maybe an implementation note with a few examples could help also. Concerning Theory, we have an interest in this standard to allow scientists to give access to their numerical models. A few years ago, Astrogrid developed the CEA (Common Execution Architecture) that fullfiled this need. Unfortunatly, CEA should have been standardized but that has never been done, and, all these efforts have been lost. PDL goes further than the CEA. So concerning Theory I.G. I would say we approve PDL as an IVOA standard (taking into account all Mark's remarks who read the document very carefully -- FranckLePetit - 2013-12-15 | |||||||
Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
PDL GWS RFCN.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version This document will act as RFC centre for the PDL GWS http://www.ivoa.net/documents/PDL/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document. Implementation details
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration
Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique) | ||||||||
Added: | ||||||||
> > |
General Comments
It is not obvious to me that PDL in its current form solves a pressing need. A standard providing some machine-readable characterisation of the inputs and outputs of services does seem like a good idea. However, the motivation of attempting to encode detailed constraints on the presented values of groups of variables is less obvious to me. It seems unrealistic to expect a generic standard to do a good job of performing a complete pre-submission test of whether a set of query parameters provides valid inputs to a given service - realistically only the service itself is going to be able to do that. Section 13.1 says "[PDL is] ... flexible and powerful enough for meeting description requirements coming with the most complex scientific codes." but it's not clear to me that this is the case. The constraints part of PDL, which does indeed have considerable complexity, has the feel to me of something put together to handle a small number of specific use cases (like eq. 16 and 17) but that will not be able to handle other ones. For instance, as far as I can see it is not able to answer the following, I would have thought quite common, requirements:
In order to cope with requirements along those lines PDL would have to become considerably more complicated, and it's not clear to me that such complication is justified by the usefulness of having validation done in a service description. I would have thought a more profitable approach would be to have a much simpler form of parameter description (less detail, especially in numerical constraints, than the existing PDL draft) and encourage services to issue helpful error messages when invalid parameter sets are presented. The above comments constitute my personal misgivings about the usefulness of this standard. However, the current draft seems to have support from some other IVOA members who are better placed than me to assess its usefulness (I don't have experience with workflows). So if the consensus within the GWS WG or other IVOA members is that this does provide a useful part of the standards jigsaw, Applications WG will not block its acceptance. However, I do have some more detailed comments which I would like the authors to address. Detailed comments
<Dimension xsi:type="AtomicConstantExpression" ConstantType="integer"> <Constant>1</Constant> </Dimension>which seems unnecessarily verbose. It would simplify PDL documents if this element was optional, and dimension assumed equal to 1 when absent. (Or perhaps, that it was not part of a SingleParameter but was added in a subclass).
-- MarkTaylor - 2013-12-10 | |||||||
Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino) | ||||||||
Changed: | ||||||||
< < | Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec) | |||||||
> > | Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec) | |||||||
Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys) | ||||||||
Changed: | ||||||||
< < | Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova) | |||||||
> > | Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova) | |||||||
Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
PDL GWS RFCN.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version This document will act as RFC centre for the PDL GWS http://www.ivoa.net/documents/PDL/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document. Implementation details
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
TCG Review Period was postponed after the Hawaii interop and the Community comments integration | ||||||||
Changed: | ||||||||
< < | Comments from TCG members during the TCG Review period: 15 November 2013 - 12 December 2013 | |||||||
> > | Comments from TCG members during the TCG Review period: 15 November 2013 - 13 December 2013 | |||||||
WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard.
IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
PDL GWS RFCN.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version This document will act as RFC centre for the PDL GWS http://www.ivoa.net/documents/PDL/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document. Implementation details
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
| ||||||||
Changed: | ||||||||
< < | TCG Review Period: postponed after the Hawaii interop and the Community comments integration | |||||||
> > | TCG Review Period was postponed after the Hawaii interop and the Community comments integration | |||||||
Comments from TCG members during the TCG Review period: 15 November 2013 - 12 December 2013WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
PDL GWS RFC | ||||||||
Changed: | ||||||||
< < | This document will act as RFC centre for the PDL GWS http://www.ivoa.net/documents/PDL/20130718/ | |||||||
> > | N.B. : comments from the Community review and remarks from the Hawaii interop GWS session have been integrated in the last version | |||||||
Added: | ||||||||
> > | This document will act as RFC centre for the PDL GWS http://www.ivoa.net/documents/PDL/ | |||||||
In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment.
Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document. Implementation details
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. the document will be cleaned -- AndreSchaaff - 2013-09-22 The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 I think we need to get some more people to read through carefully, especially some XML experts. a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22
| ||||||||
Changed: | ||||||||
< < | PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: It beginning of document it explains that PDL is made for describing parameters, physical quantities (via UCD), concept (via SKOS) and constrain (via PDL grammar). It specifies already exist Jobs Description Language (WSDL or WADL) that describe how service work and parameters basically. However, PDL describes completely the parameters. then on page 9: PDL is a VO standard for describing how to access data or driving online codes, exposed through the VO. PDL is not such a standard, UWS is the standard for REST service and as a service it can be described using Jobs Description Language. It is quite important that PDL is presented only as a grammar to describe parameter, not as all UWS and page 34....the PDL server, embedding the veri... It seems quite strange to have a PDL server, PDL can be used on server and client side, but we should have a PDL server as PDL is a grammar A small point, but we should not confuse a grammar with a job system | |||||||
> > | PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: It beginning of document it explains that PDL is made for describing parameters, physical quantities (via UCD), concept (via SKOS) and constrain (via PDL grammar). It specifies already exist Jobs Description Language (WSDL or WADL) that describe how service work and parameters basically. However, PDL describes completely the parameters. then on page 9: PDL is a VO standard for describing how to access data or driving online codes, exposed through the VO. PDL is not such a standard, UWS is the standard for REST service and as a service it can be described using Jobs Description Language. It is quite important that PDL is presented only as a grammar to describe parameter, not as all UWS and page 34....the PDL server, embedding the veri... It seems quite strange to have a PDL server, PDL can be used on server and client side, but we should have a PDL server as PDL is a grammar A small point, but we should not confuse a grammar with a job system | |||||||
the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22
Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much. To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
| ||||||||
Changed: | ||||||||
< < | TCG Review Period: postponed after the Hawaii interop | |||||||
> > | TCG Review Period: postponed after the Hawaii interop and the Community comments integration | |||||||
Changed: | ||||||||
< < | Comments from TCG members during the TCG Review period: | |||||||
> > | Comments from TCG members during the TCG Review period: 15 November 2013 - 12 December 2013 | |||||||
WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard.
IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
PDL GWS RFCThis document will act as RFC centre for the PDL GWS http://www.ivoa.net/documents/PDL/20130718/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document. Implementation details
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. | ||||||||
Added: | ||||||||
> > | the document will be cleaned -- AndreSchaaff - 2013-09-22 | |||||||
The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? | ||||||||
Added: | ||||||||
> > | the next update of the document will be clearer for theses differents points -- AndreSchaaff - 2013-09-22 | |||||||
I think we need to get some more people to read through carefully, especially some XML experts. | ||||||||
Added: | ||||||||
> > | a talk and a discussion is planned during the Hawaii interop sessions -- AndreSchaaff - 2013-09-22 | |||||||
Changed: | ||||||||
< < | PierreLeSidenar GretchenGreene (9/11/2013) | |||||||
> > | PierreLeSidenar GretchenGreene (9/11/2013) There is two line in the text with confusion: It beginning of document it explains that PDL is made for describing parameters, physical quantities (via UCD), concept (via SKOS) and constrain (via PDL grammar). It specifies already exist Jobs Description Language (WSDL or WADL) that describe how service work and parameters basically. However, PDL describes completely the parameters. then on page 9: PDL is a VO standard for describing how to access data or driving online codes, exposed through the VO. PDL is not such a standard, UWS is the standard for REST service and as a service it can be described using Jobs Description Language. It is quite important that PDL is presented only as a grammar to describe parameter, not as all UWS and page 34....the PDL server, embedding the veri... It seems quite strange to have a PDL server, PDL can be used on server and client side, but we should have a PDL server as PDL is a grammar A small point, but we should not confuse a grammar with a job system | |||||||
Changed: | ||||||||
< < | There is two line in the text with confusion: It beginning of document it explains that PDL is made for describing parameters, physical quantities (via UCD), concept (via SKOS) and constrain (via PDL grammar). It specifies already exist Jobs Description Language (WSDL or WADL) that describe how service work and parameters basically. However, PDL describes completely the parameters. then on page 9: PDL is a VO standard for describing how to access data or driving online codes, exposed through the VO. PDL is not such a standard, UWS is the standard for REST service and as a service it can be described using Jobs Description Language. It is quite important that PDL is presented only as a grammar to describe parameter, not as all UWS and page 34....the PDL server, embedding the veri... It seems quite strange to have a PDL server, PDL can be used on server and client side, but we should have a PDL server as PDL is a grammar A small point, but we should not confuse a grammar with a job system | |||||||
> > | the use of UWS will be refurbished in the next update of the document -- AndreSchaaff - 2013-09-22 | |||||||
Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much.
To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
| ||||||||
Changed: | ||||||||
< < | TCG Review Period: 18 September 2013 - 15 October 2013 | |||||||
> > | TCG Review Period: postponed after the Hawaii interop | |||||||
Added: | ||||||||
> > | ||||||||
Comments from TCG members during the TCG Review period:WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
PDL GWS RFCThis document will act as RFC centre for the PDL GWS http://www.ivoa.net/documents/PDL/20130718/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document. Implementation details
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? I think we need to get some more people to read through carefully, especially some XML experts. | ||||||||
Changed: | ||||||||
< < | ||||||||
> > | ||||||||
PierreLeSidenar GretchenGreene (9/11/2013) | ||||||||
Changed: | ||||||||
< < | There is two line in the text with confusion: It beginning of document it explains that PDL is made for describing parameters, physical quantities (via UCD), concept (via SKOS) and constrain (via PDL grammar). It specifies already exist Jobs Description Language (WSDL or WADL) that describe how service work and parameters basically. However, PDL describes completely the parameters. then on page 9: PDL is a VO standard for describing how to access data or driving online codes, exposed through the VO. PDL is not such a standard, UWS is the standard for REST service and as a service it can be described using Jobs Description Language. It is quite important that PDL is presented only as a grammar to describe parameter, not as all UWS and page 34....the PDL server, embedding the veri... It seems quite strange to have a PDL server, PDL can be used on server and client side, but we should have a PDL server as PDL is a grammar A small point, but we should not confuse a grammar with a job system | |||||||
> > | There is two line in the text with confusion: It beginning of document it explains that PDL is made for describing parameters, physical quantities (via UCD), concept (via SKOS) and constrain (via PDL grammar). It specifies already exist Jobs Description Language (WSDL or WADL) that describe how service work and parameters basically. However, PDL describes completely the parameters. then on page 9: PDL is a VO standard for describing how to access data or driving online codes, exposed through the VO. PDL is not such a standard, UWS is the standard for REST service and as a service it can be described using Jobs Description Language. It is quite important that PDL is presented only as a grammar to describe parameter, not as all UWS and page 34....the PDL server, embedding the veri... It seems quite strange to have a PDL server, PDL can be used on server and client side, but we should have a PDL server as PDL is a grammar A small point, but we should not confuse a grammar with a job system | |||||||
Added: | ||||||||
> > |
Franck Le Petit (19/10/2013): As part of a (minor) author of this standard, I cannot say much.
To my point of view, this standard fills a missing piece in the IVOA standards for interoperable services in astrophysics. Up to now, we have nothing to get a "real" interoperability (in a computer - computer meaning) between services. Indeed, PDL goes further than previous standards because it allows to describe properly input and ouputs of services associating to each quantities semantic meaning, units and, if required, relationship between parameters. So it has everything to put a bit of "intelligence" in the communication between services. This should be a useful standard. For Theory I.G., I see immediatly some several use cases. Any online application (online simulation code for example) can be described in details thanks to PDL. One of the difficulty often mentionned by the community to provide online codes is that, those ones are often complex to use and users may do mistake with online simulation codes. They could use them outside of their validity domain. The description of parameters with PDL allows to constrain those ones in validity domains, and so PDL answers this fear of the theorist community. Describing precisly inputs and outputs of services with relationships between parameters, should also be a major step forward for workflows. Workflows engines have been a subject of dicussions in the community for a long time but very few of them are really used. Proper description of services with PDL should help to have more efficient interoperable services in workflows engines.
| |||||||
TCG Review Period: 18 September 2013 - 15 October 2013Comments from TCG members during the TCG Review period:WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
PDL GWS RFCThis document will act as RFC centre for the PDL GWS http://www.ivoa.net/documents/PDL/20130718/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document. Implementation details
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions? In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? | ||||||||
Changed: | ||||||||
< < | I think we need to get some more people to read through carefully, especially some XML experts. | |||||||
> > | I think we need to get some more people to read through carefully, especially some XML experts. | |||||||
Added: | ||||||||
> > | PierreLeSidenar GretchenGreene (9/11/2013)
There is two line in the text with confusion: | |||||||
TCG Review Period: 18 September 2013 - 15 October 2013Comments from TCG members during the TCG Review period:WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) |
PDL GWS RFCThis document will act as RFC centre for the PDL GWS http://www.ivoa.net/documents/PDL/20130718/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document. Implementation details
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period: | ||||||||
Added: | ||||||||
> > | BobHanisch (9/10/2013): Wow, this is pretty complicated! Do we have a significant demand for these capabilities, particularly for expressing complex dependencies amongst parameters? Could there be a simple core with extensions?
In any case, the document itself needs a good editorial clean-up. There are lots of places where the English is not quite right. The word "unique" is used a lot in the document, but the scope is not clear (and I think is not the same in all cases). Sometimes it seems to refer to the entirety of the VO (e.g., ServiceID) and at other times (mostly, I think) it refers to within a particular PDL instance. Yes? I think we need to get some more people to read through carefully, especially some XML experts. | |||||||
TCG Review Period: 18 September 2013 - 15 October 2013Comments from TCG members during the TCG Review period:WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham) | ||||||||
Changed: | ||||||||
< < | Applications Working Group (_Mark Taylor, Pierre Fernique)Data Access Layer Working Group (Patrick Dowler, Francois Bonnarel)Data Model Working Group (_Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec)Registry Working Group (_Gretchen Greene, Pierre Le Sidaner)Semantics Working Group (_Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova) | |||||||
> > | Applications Working Group ( _Mark Taylor, Pierre Fernique)Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova) | |||||||
Education Interest Group ( _Massimo Ramella, Sudhanshu Barway ) | ||||||||
Changed: | ||||||||
< < | Knowledge Discovery in Databases Interest Group (George Djorgovski )Theory Interest Group (_Franck Le Petit, Rick Wagner)Time Domain Interest Group (_John Swinbank, Mike Fitzpatrick) | |||||||
> > | Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) | |||||||
PDL GWS RFCThis document will act as RFC centre for the PDL GWS http://www.ivoa.net/documents/PDL/20130718/ In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document. Implementation details
| ||||||||
Added: | ||||||||
> > |
| |||||||
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:
TCG Review Period: 18 September 2013 - 15 October 2013Comments from TCG members during the TCG Review period:WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham) | ||||||||
Changed: | ||||||||
< < | Applications Working Group ( _Mark Taylor, Pierre Fernique)Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group ( Andre Schaaff, Andreas Wicenec)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group ( Alberto Accomazzi, Francoise Genova) | |||||||
> > | Applications Working Group (_Mark Taylor, Pierre Fernique)Data Access Layer Working Group (Patrick Dowler, Francois Bonnarel)Data Model Working Group (_Jesus Salgado, Omar Laurino)Grid & Web Services Working Group (Andre Schaaff, Andreas Wicenec)Registry Working Group (_Gretchen Greene, Pierre Le Sidaner)Semantics Working Group (_Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group (Alberto Accomazzi, Francoise Genova) | |||||||
Education Interest Group ( _Massimo Ramella, Sudhanshu Barway ) | ||||||||
Changed: | ||||||||
< < | Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) | |||||||
> > | Knowledge Discovery in Databases Interest Group (George Djorgovski )Theory Interest Group (_Franck Le Petit, Rick Wagner)Time Domain Interest Group (_John Swinbank, Mike Fitzpatrick) | |||||||
Changed: | ||||||||
< < |
Recent changes in PDL1RFC web:
• Web Statistics • Web Home • Web Top Menu • Web Preferences • Web Notify • Web Left Bar • Web Topic List • Web Search Advanced • Web Search • Web Index • Web Changes • Web Atom • Web Create New Topic • Web Rss • more... | |||||||
> > | PDL GWS RFC | |||||||
Deleted: | ||||||||
< < | Welcome to the IVOA/PDL1RFC web | |||||||
Changed: | ||||||||
< < | Replace this text with a short description on how this web is used. | |||||||
> > | This document will act as RFC centre for the PDL GWS http://www.ivoa.net/documents/PDL/20130718/ | |||||||
Changed: | ||||||||
< < |
| |||||||
> > | In order to add a comment to the document, please edit this page and add your comment to the list below in the format used for the example (include your WikiName so authors can contact you for further information). When the author(s) of the document have considered the comment, they will provide a response after the comment. | |||||||
Deleted: | ||||||||
< < |
| |||||||
Changed: | ||||||||
< < | IVOA/PDL1RFC Web Utilities | |||||||
> > | Discussion about any of the comments or responses should be conducted on the grid@ivoa.net. However, please be sure to enter your initial comments here for full consideration in any future revisions of this document. | |||||||
Added: | ||||||||
> > | Implementation details | |||||||
Changed: | ||||||||
< < |
| |||||||
> > |
IVOA Community Review Period: 20 August 2013 - 16 September 2013Comments from the IVOA Community during the RFC period:
TCG Review Period: 18 September 2013 - 15 October 2013Comments from TCG members during the TCG Review period: | |||||||
Added: | ||||||||
> > |
WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard.
IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. WG chairs or vice chairs must read the Document, provide comments if any and formally indicate if they approve or not the Standard. IG chairs or vice chairs are also encouraged to do the same, although their inputs are not compulsory. TCG Chair & Vice Chair ( _Séverin Gaudet, Matthew Graham)Applications Working Group ( _Mark Taylor, Pierre Fernique)Data Access Layer Working Group ( Patrick Dowler, Francois Bonnarel)Data Model Working Group ( _Jesus Salgado, Omar Laurino)Grid & Web Services Working Group ( Andre Schaaff, Andreas Wicenec)Registry Working Group ( _Gretchen Greene, Pierre Le Sidaner)Semantics Working Group ( _Norman Gray, Mireille Louys)Data Curation & Preservation Interest Group ( Alberto Accomazzi, Francoise Genova)Education Interest Group ( _Massimo Ramella, Sudhanshu Barway )Knowledge Discovery in Databases Interest Group ( George Djorgovski )Theory Interest Group ( _Franck Le Petit, Rick Wagner)Time Domain Interest Group ( _John Swinbank, Mike Fitzpatrick) | |||||||
Added: | ||||||||
> > |
Recent changes in PDL1RFC web:
• Web Statistics • Web Home • Web Top Menu • Web Preferences • Web Notify • Web Left Bar • Web Topic List • Web Search Advanced • Web Search • Web Index • Web Changes • Web Atom • Web Create New Topic • Web Rss • more... | |||||||
Welcome to the IVOA/PDL1RFC web | ||||||||
Changed: | ||||||||
< < | Available Information | |||||||
> > | Replace this text with a short description on how this web is used. | |||||||
Added: | ||||||||
> > |
| |||||||
| ||||||||
Changed: | ||||||||
< < |
| |||||||
> > | ||||||||
Deleted: | ||||||||
< < | ||||||||
IVOA/PDL1RFC Web Utilities | ||||||||
Deleted: | ||||||||
< < | ||||||||
Added: | ||||||||
> > |
| |||||||
Welcome to the IVOA/PDL1RFC web
Available Information
IVOA/PDL1RFC Web Utilities |