IVOA Knowledge Discovery Interest Group

Charter

Knowledge discovery is the task of processing and analyzing data-sets with the aim of extracting new knowledge. This area spans widely across multiple disciplines, including visualization, remote data exploration, machine learning techniques, statistical methods, workflow orchestration, and polymorphic data access. To support the process of discovery, the KD-IG interacts closely with the other working/interest groups and feeds back requirements of the scientific community.

The activities of the KD-IG include the following items with a strong emphasis on the first two points:

1. Participating in the definition of new data preservation and exchange formats with respect to support machine learning algorithms.
2. Introducing uncertainties and probabilistic description to VO standards and services.
   
   
3. Presenting and collecting best practice examples of scientific data analytics in astronomy.
4. Defining requirements for implementing and adding machine learning capabilities to services.
5. Coordinating and unifying the access to data visualization functionalities.
6. Discussing the aspect of data provenance with respect to data used to derive/train models.
7. Introducing proper statistical scoring and evaluation methods as services.
8. Contributing to the discussion on scripting and orchestrating the scientific discovery workflow.
9. Supporting the development of dedicated knowledge discovery applications.

<-- 
We will develop and test scalable data mining algorithms and the accompanying new standards for VO interfaces and protocols, so that these algorithms can be discovered and used transparently within VO science workflows or in standalone data exploration applications. Therefore the activities of the KDD-IG will be:
 
 
 Support the definition of an ontology of the KDD tasks required by the astronomical community. This ontology will be used to define programming and documentation standards. 
 
 Make an inventory of existing methods relevant for astrophysical applications (more than 100 new KDD models and methods appear every month on specialized journals). 
 
 Identify reference data sets to be used for comparing, debugging and testing methods and tools. 
 
 Foster the implementation, using available VO standards and methods, of general purpose data exploration and data mining methods which will allow the general user to seamlessly exploit the complex data repositories offered by the VO. 
 
 Provide/receive feedbacks to/from the WGs in order to improve the usability of VO tools and standards. 
 
 Provide/receive from the community information to improve both the usability and the potentialities of Data Mining tools under the VO. 
 
 Define and pursue specific science cases which will be used to showcase the VO capabilities to the community. 
 
More important than anything else, we wish to use this IG as an arena where different groups can share experiences and plan future developments.
 
 
 -->

Motivations

KD-IG Meetings

Other Interesting Meetings for KD-IG

<-- 
Next tasks
 
 
 
 
 Definition of a taxonomy of Data Mining models. This taxonomy will contribute to the Standard Vocabulary of the Semantics WG2. 
 
 Definition of the requirements which a Data Mining model needs to match in order to be imported under the VObs standards. 
 
 Inventory of existing Data Mining models of relevant astrophysical interest. 
 
 Definition of standard template data sets for Data Mining models test and debugging. 
 
 Definition of standard data sets to be used as bases of knowledge for debugging and test of supervised methods. 
 
 Definition of procedures to extract and validate robust bases of knowledge from the VObs data archives using the VObs ontology. 
 
 Study of the scalability of Data Mining models under different computing infrastructures (definition of best benchmarks). 
 
-->

Related Topics

1. Dictionary of Data Mining terms;
2. Census of Data Mining and Machine Learning tools and methods of astronomical interest;
3. Template datasets for algorithm benchmarking;
4. A user guide for Knowledge Discovery in Databases in Astronomy;
5. Knowledge Discovery in Databases and VO standards;
6. Specific fields of applications of KDD in astronomical research;

Members

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

IVOA Knowledge Discovery Interest Group

Charter

Knowledge discovery is the task of processing and analyzing data-sets with the aim of extracting new knowledge. This area spans widely across multiple disciplines, including visualization, remote data exploration, machine learning techniques, statistical methods, workflow orchestration, and polymorphic data access. To support the process of discovery, the KD-IG interacts closely with the other working/interest groups and feeds back requirements of the scientific community.

The activities of the KD-IG include the following items with a strong emphasis on the first two points:

1. Participating in the definition of new data preservation and exchange formats with respect to support machine learning algorithms.
2. Introducing uncertainties and probabilistic description to VO standards and services.
   
   
3. Presenting and collecting best practice examples of scientific data analytics in astronomy.
4. Defining requirements for implementing and adding machine learning capabilities to services.
5. Coordinating and unifying the access to data visualization functionalities.
6. Discussing the aspect of data provenance with respect to data used to derive/train models.
7. Introducing proper statistical scoring and evaluation methods as services.
8. Contributing to the discussion on scripting and orchestrating the scientific discovery workflow.
9. Supporting the development of dedicated knowledge discovery applications.

<-- 
We will develop and test scalable data mining algorithms and the accompanying new standards for VO interfaces and protocols, so that these algorithms can be discovered and used transparently within VO science workflows or in standalone data exploration applications. Therefore the activities of the KDD-IG will be:
 
 
 Support the definition of an ontology of the KDD tasks required by the astronomical community. This ontology will be used to define programming and documentation standards. 
 
 Make an inventory of existing methods relevant for astrophysical applications (more than 100 new KDD models and methods appear every month on specialized journals). 
 
 Identify reference data sets to be used for comparing, debugging and testing methods and tools. 
 
 Foster the implementation, using available VO standards and methods, of general purpose data exploration and data mining methods which will allow the general user to seamlessly exploit the complex data repositories offered by the VO. 
 
 Provide/receive feedbacks to/from the WGs in order to improve the usability of VO tools and standards. 
 
 Provide/receive from the community information to improve both the usability and the potentialities of Data Mining tools under the VO. 
 
 Define and pursue specific science cases which will be used to showcase the VO capabilities to the community. 
 
More important than anything else, we wish to use this IG as an arena where different groups can share experiences and plan future developments.
 
 
 -->

Motivations

During the Strasbourg InterOp Meeting it emerged the need for an Interest Group on Data Mining (KD-IG) as an indispensable step to bridge the Virtual Observatory Infrastructure with the expected VO science. In fact, "...Data mining, or KDD, is the semi-automatic discovery of patterns, associations, changes, anomalies, and statistically significant structures and events in data. In other words, traditional data analysis is assumption driven as a hypothesis is formed and validated against the data. Data mining, in contrast, is discovery driven as the patterns are automatically extracted from data....”

<-- 
As such, Data Mining (DM) can be considered as the “frontier” of VO enabled science since it represents the only way to capture and reveal the scientific knowledge (patterns, trends, correlations, etc.) hidden behind the complexity of Massive Data Sets.
 -->

Data Mining is a rapidly evolving set of methodologies which needs to be imported under the VO umbrella and not just another application. As such, DM cannot be just a tool or a suite of tools offered by a group of developers to a “passive community”. Data Mining involves a large number of researchers across many domains. The astronomical community, which has only recently entered the Massive Data Sets era, makes use of just a handful of methods and tools which very often are far from optimal. The synergy of different expertise present in the IVOA makes it the ideal arena for exploring new and more modern approaches.

KD-IG requires a strong and continuous interaction with the scientific community which, besides testing the proposed solutions, methods, and tools, will also provide feedback and inputs aiming at extending the scientific capabilities of the VO.

The KD-IG will interfaces to many other IVOA working and interest groups: Applications, Semantics, Time Domain, Data Models, Grid & Web Services, and Resource Registry. This cross- discipline nature is also a primary reason to create a specific IG. Data Mining, in fact, addresses sophisticated and extreme modes of usage which require a careful orchestration and fine tuning of standards, methods, and tools provided by the other IVOA WGs and IGs. Typical examples are the automatic extraction of bases of knowledge from VO archives using VO ontologies; the transparent access to large computational facilities regardless the computational paradigm; the automated switching from asynchronous to synchronous mode of data access; and the extreme usage of workflows and advanced visualization methods. Furthermore, effective KD requires the possibility for an inexperienced user to contribute, or at least seamlessly use under the VO infrastructure, his/her own KD routines and methods. This situation puts strong requirements on security issues and opens new problems for ticketing and scheduling. In other words, the KD-IG will provide feedback to the solutions implemented by the WG’s and, by posing new operational problems, will stimulate the development and adoption of new solutions and standards.

We also wish to stress that, in ultimate analysis, the goal of the KD-IG is to allow the VO to produce new scientific knowledge publishable in astronomical journals. On the one end its activities will contribute to demonstrate to the community the power and necessity of federated access to the vast VO universe of data and, on the other, KD-IG will illustrate the power and performance of data mining algorithms to facilitate and accelerate astronomical discovery within this data universe.

KD-IG Meetings

Other Interesting Meetings for KD-IG

<-- 
Next tasks
 
 
 
 
 Definition of a taxonomy of Data Mining models. This taxonomy will contribute to the Standard Vocabulary of the Semantics WG2. 
 
 Definition of the requirements which a Data Mining model needs to match in order to be imported under the VObs standards. 
 
 Inventory of existing Data Mining models of relevant astrophysical interest. 
 
 Definition of standard template data sets for Data Mining models test and debugging. 
 
 Definition of standard data sets to be used as bases of knowledge for debugging and test of supervised methods. 
 
 Definition of procedures to extract and validate robust bases of knowledge from the VObs data archives using the VObs ontology. 
 
 Study of the scalability of Data Mining models under different computing infrastructures (definition of best benchmarks). 
 
-->

Related Topics

In the past hot topics had been identified. These are following the priorities emerged during the first KD-IG meeting held at the IVOA.InterOpMay2010KDD in Victoria, and singled out by the Chair in his welcome message to the members.

Please follow the links and edit the specific pages:

1. Dictionary of Data Mining terms;
2. Census of Data Mining and Machine Learning tools and methods of astronomical interest;
3. Template datasets for algorithm benchmarking;
4. A user guide for Knowledge Discovery in Databases in Astronomy;
5. Knowledge Discovery in Databases and VO standards;
6. Specific fields of applications of KDD in astronomical research;

Members

Chair: RaffaeleDAbrusco

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

IVOA Knowledge Discovery Interest Group

Charter

Knowledge discovery is the task of processing and analyzing data-sets with the aim of extracting new knowledge. This area spans widely across multiple disciplines, including visualization, remote data exploration, machine learning techniques, statistical methods, workflow orchestration, and polymorphic data access. To support the process of discovery, the KD-IG interacts closely with the other working/interest groups and feeds back requirements of the scientific community.

The activities of the KD-IG include the following items with a strong emphasis on the first two points:

1. Participating in the definition of new data preservation and exchange formats with respect to support machine learning algorithms.
2. Introducing uncertainties and probabilistic description to VO standards and services.
   
   
3. Presenting and collecting best practice examples of scientific data analytics in astronomy.
4. Defining requirements for implementing and adding machine learning capabilities to services.
5. Coordinating and unifying the access to data visualization functionalities.
6. Discussing the aspect of data provenance with respect to data used to derive/train models.
7. Introducing proper statistical scoring and evaluation methods as services.
8. Contributing to the discussion on scripting and orchestrating the scientific discovery workflow.
9. Supporting the development of dedicated knowledge discovery applications.

<-- 
We will develop and test scalable data mining algorithms and the accompanying new standards for VO interfaces and protocols, so that these algorithms can be discovered and used transparently within VO science workflows or in standalone data exploration applications. Therefore the activities of the KDD-IG will be:
 
 
 Support the definition of an ontology of the KDD tasks required by the astronomical community. This ontology will be used to define programming and documentation standards. 
 
 Make an inventory of existing methods relevant for astrophysical applications (more than 100 new KDD models and methods appear every month on specialized journals). 
 
 Identify reference data sets to be used for comparing, debugging and testing methods and tools. 
 
 Foster the implementation, using available VO standards and methods, of general purpose data exploration and data mining methods which will allow the general user to seamlessly exploit the complex data repositories offered by the VO. 
 
 Provide/receive feedbacks to/from the WGs in order to improve the usability of VO tools and standards. 
 
 Provide/receive from the community information to improve both the usability and the potentialities of Data Mining tools under the VO. 
 
 Define and pursue specific science cases which will be used to showcase the VO capabilities to the community. 
 
More important than anything else, we wish to use this IG as an arena where different groups can share experiences and plan future developments.
 
 
 -->

Motivations

During the Strasbourg InterOp Meeting it emerged the need for an Interest Group on Data Mining (KD-IG) as an indispensable step to bridge the Virtual Observatory Infrastructure with the expected VO science. In fact, "...Data mining, or KDD, is the semi-automatic discovery of patterns, associations, changes, anomalies, and statistically significant structures and events in data. In other words, traditional data analysis is assumption driven as a hypothesis is formed and validated against the data. Data mining, in contrast, is discovery driven as the patterns are automatically extracted from data....”

<-- 
As such, Data Mining (DM) can be considered as the “frontier” of VO enabled science since it represents the only way to capture and reveal the scientific knowledge (patterns, trends, correlations, etc.) hidden behind the complexity of Massive Data Sets.
 -->

Data Mining is a rapidly evolving set of methodologies which needs to be imported under the VO umbrella and not just another application. As such, DM cannot be just a tool or a suite of tools offered by a group of developers to a “passive community”. Data Mining involves a large number of researchers across many domains. The astronomical community, which has only recently entered the Massive Data Sets era, makes use of just a handful of methods and tools which very often are far from optimal. The synergy of different expertise present in the IVOA makes it the ideal arena for exploring new and more modern approaches.

KD-IG requires a strong and continuous interaction with the scientific community which, besides testing the proposed solutions, methods, and tools, will also provide feedback and inputs aiming at extending the scientific capabilities of the VO.

The KD-IG will interfaces to many other IVOA working and interest groups: Applications, Semantics, Time Domain, Data Models, Grid & Web Services, and Resource Registry. This cross- discipline nature is also a primary reason to create a specific IG. Data Mining, in fact, addresses sophisticated and extreme modes of usage which require a careful orchestration and fine tuning of standards, methods, and tools provided by the other IVOA WGs and IGs. Typical examples are the automatic extraction of bases of knowledge from VO archives using VO ontologies; the transparent access to large computational facilities regardless the computational paradigm; the automated switching from asynchronous to synchronous mode of data access; and the extreme usage of workflows and advanced visualization methods. Furthermore, effective KD requires the possibility for an inexperienced user to contribute, or at least seamlessly use under the VO infrastructure, his/her own KD routines and methods. This situation puts strong requirements on security issues and opens new problems for ticketing and scheduling. In other words, the KD-IG will provide feedback to the solutions implemented by the WG’s and, by posing new operational problems, will stimulate the development and adoption of new solutions and standards.

We also wish to stress that, in ultimate analysis, the goal of the KD-IG is to allow the VO to produce new scientific knowledge publishable in astronomical journals. On the one end its activities will contribute to demonstrate to the community the power and necessity of federated access to the vast VO universe of data and, on the other, KD-IG will illustrate the power and performance of data mining algorithms to facilitate and accelerate astronomical discovery within this data universe.

KD-IG Meetings

Other Interesting Meetings for KD-IG

<-- 
Next tasks
 
 
 
 
 Definition of a taxonomy of Data Mining models. This taxonomy will contribute to the Standard Vocabulary of the Semantics WG2. 
 
 Definition of the requirements which a Data Mining model needs to match in order to be imported under the VObs standards. 
 
 Inventory of existing Data Mining models of relevant astrophysical interest. 
 
 Definition of standard template data sets for Data Mining models test and debugging. 
 
 Definition of standard data sets to be used as bases of knowledge for debugging and test of supervised methods. 
 
 Definition of procedures to extract and validate robust bases of knowledge from the VObs data archives using the VObs ontology. 
 
 Study of the scalability of Data Mining models under different computing infrastructures (definition of best benchmarks). 
 
-->

Related Topics

In the past hot topics had been identified. These are following the priorities emerged during the first KD-IG meeting held at the IVOA.InterOpMay2010KDD in Victoria, and singled out by the Chair in his welcome message to the members.

Please follow the links and edit the specific pages:

1. Dictionary of Data Mining terms;
2. Census of Data Mining and Machine Learning tools and methods of astronomical interest;
3. Template datasets for algorithm benchmarking;
4. A user guide for Knowledge Discovery in Databases in Astronomy;
5. Knowledge Discovery in Databases and VO standards;
6. Specific fields of applications of KDD in astronomical research;

Members

Chair: RaffaeleDAbrusco

Vice Chair: n/a

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

IVOA Knowledge Discovery Interest Group

Charter

Knowledge discovery is the task of processing and analyzing data-sets with the aim of extracting new knowledge. This area spans widely across multiple disciplines, including visualization, remote data exploration, machine learning techniques, statistical methods, workflow orchestration, and polymorphic data access. To support the process of discovery, the KD-IG interacts closely with the other working/interest groups and feeds back requirements of the scientific community.

The activities of the KD-IG include the following items with a strong emphasis on the first two points:

1. Participating in the definition of new data preservation and exchange formats with respect to support machine learning algorithms.
2. Introducing uncertainties and probabilistic description to VO standards and services.
   
   
3. Presenting and collecting best practice examples of scientific data analytics in astronomy.
4. Defining requirements for implementing and adding machine learning capabilities to services.
5. Coordinating and unifying the access to data visualization functionalities.
6. Discussing the aspect of data provenance with respect to data used to derive/train models.
7. Introducing proper statistical scoring and evaluation methods as services.
8. Contributing to the discussion on scripting and orchestrating the scientific discovery workflow.
9. Supporting the development of dedicated knowledge discovery applications.

<-- 
We will develop and test scalable data mining algorithms and the accompanying new standards for VO interfaces and protocols, so that these algorithms can be discovered and used transparently within VO science workflows or in standalone data exploration applications. Therefore the activities of the KDD-IG will be:
 
 
 Support the definition of an ontology of the KDD tasks required by the astronomical community. This ontology will be used to define programming and documentation standards. 
 
 Make an inventory of existing methods relevant for astrophysical applications (more than 100 new KDD models and methods appear every month on specialized journals). 
 
 Identify reference data sets to be used for comparing, debugging and testing methods and tools. 
 
 Foster the implementation, using available VO standards and methods, of general purpose data exploration and data mining methods which will allow the general user to seamlessly exploit the complex data repositories offered by the VO. 
 
 Provide/receive feedbacks to/from the WGs in order to improve the usability of VO tools and standards. 
 
 Provide/receive from the community information to improve both the usability and the potentialities of Data Mining tools under the VO. 
 
 Define and pursue specific science cases which will be used to showcase the VO capabilities to the community. 
 
More important than anything else, we wish to use this IG as an arena where different groups can share experiences and plan future developments.
 
 
 -->

Motivations

During the Strasbourg InterOp Meeting it emerged the need for an Interest Group on Data Mining (KD-IG) as an indispensable step to bridge the Virtual Observatory Infrastructure with the expected VO science. In fact, "...Data mining, or KDD, is the semi-automatic discovery of patterns, associations, changes, anomalies, and statistically significant structures and events in data. In other words, traditional data analysis is assumption driven as a hypothesis is formed and validated against the data. Data mining, in contrast, is discovery driven as the patterns are automatically extracted from data....”

<-- 
As such, Data Mining (DM) can be considered as the “frontier” of VO enabled science since it represents the only way to capture and reveal the scientific knowledge (patterns, trends, correlations, etc.) hidden behind the complexity of Massive Data Sets.
 -->

Data Mining is a rapidly evolving set of methodologies which needs to be imported under the VO umbrella and not just another application. As such, DM cannot be just a tool or a suite of tools offered by a group of developers to a “passive community”. Data Mining involves a large number of researchers across many domains. The astronomical community, which has only recently entered the Massive Data Sets era, makes use of just a handful of methods and tools which very often are far from optimal. The synergy of different expertise present in the IVOA makes it the ideal arena for exploring new and more modern approaches.

KD-IG requires a strong and continuous interaction with the scientific community which, besides testing the proposed solutions, methods, and tools, will also provide feedback and inputs aiming at extending the scientific capabilities of the VO.

The KD-IG will interfaces to many other IVOA working and interest groups: Applications, Semantics, Time Domain, Data Models, Grid & Web Services, and Resource Registry. This cross- discipline nature is also a primary reason to create a specific IG. Data Mining, in fact, addresses sophisticated and extreme modes of usage which require a careful orchestration and fine tuning of standards, methods, and tools provided by the other IVOA WGs and IGs. Typical examples are the automatic extraction of bases of knowledge from VO archives using VO ontologies; the transparent access to large computational facilities regardless the computational paradigm; the automated switching from asynchronous to synchronous mode of data access; and the extreme usage of workflows and advanced visualization methods. Furthermore, effective KD requires the possibility for an inexperienced user to contribute, or at least seamlessly use under the VO infrastructure, his/her own KD routines and methods. This situation puts strong requirements on security issues and opens new problems for ticketing and scheduling. In other words, the KD-IG will provide feedback to the solutions implemented by the WG’s and, by posing new operational problems, will stimulate the development and adoption of new solutions and standards.

We also wish to stress that, in ultimate analysis, the goal of the KD-IG is to allow the VO to produce new scientific knowledge publishable in astronomical journals. On the one end its activities will contribute to demonstrate to the community the power and necessity of federated access to the vast VO universe of data and, on the other, KD-IG will illustrate the power and performance of data mining algorithms to facilitate and accelerate astronomical discovery within this data universe.

KD-IG Meetings

Other Interesting Meetings for KD-IG

<-- 
Next tasks
 
 
 
 
 Definition of a taxonomy of Data Mining models. This taxonomy will contribute to the Standard Vocabulary of the Semantics WG2. 
 
 Definition of the requirements which a Data Mining model needs to match in order to be imported under the VObs standards. 
 
 Inventory of existing Data Mining models of relevant astrophysical interest. 
 
 Definition of standard template data sets for Data Mining models test and debugging. 
 
 Definition of standard data sets to be used as bases of knowledge for debugging and test of supervised methods. 
 
 Definition of procedures to extract and validate robust bases of knowledge from the VObs data archives using the VObs ontology. 
 
 Study of the scalability of Data Mining models under different computing infrastructures (definition of best benchmarks). 
 
-->

Related Topics

In the past hot topics had been identified. These are following the priorities emerged during the first KD-IG meeting held at the IVOA.InterOpMay2010KDD in Victoria, and singled out by the Chair in his welcome message to the members.

Please follow the links and edit the specific pages:

1. Dictionary of Data Mining terms;
2. Census of Data Mining and Machine Learning tools and methods of astronomical interest;
3. Template datasets for algorithm benchmarking;
4. A user guide for Knowledge Discovery in Databases in Astronomy;
5. Knowledge Discovery in Databases and VO standards;
6. Specific fields of applications of KDD in astronomical research;

Members

Chair: RaffaeleDAbrusco

Vice Chair: n/a

Task Force Members:

RaffaeleDAbrusco
Rick Ebert
MatthewGraham
Steve Groom
Franck Le Petit
Kenny Lo
Jiri Nadvornik
Kai Lars Polsterer
PetrSkoda
HerveWozniak

Passive Members:

Sheelu Abraham
Lauretta Auvil
NickBall
ThomasBoch
Sen Bodhisattva
KirkBorne
MaxBrescia
Robert Brunner
Darren Davis
ReinaldoDeCarvalho
SebastienDerriere
GiuseppeLongo
CiroDonalek
Paul Eglitis
Pepi Fabbiano
PierreFernique
Peter Freeman
Fabian Gieseke
Alyssa Goodman
Alexander Gray
Paul Green
BobHanisch
Sheth Kartik
AjitKembhavi
OmarLaurino
Ann Lee
JeffLusted
AshishMahabal
BobMann
William March
Joseph Mazzarella
SabineMcConnell
Fionn Murtagh
NinanSajeethPhilip
Rebecca Nugent
PaoloPadovani
FabioPasian
Misha Pesenson
Manuel Luis Sarro Baro
RiccardoSmareglia
Antonino Staiano
RoyWilliams
Chris Stubbs
NicholasWalton
Yongheng Zhao

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

IVOA Knowledge Discovery Interest Group

Charter

1. Participating in the definition of new data preservation and exchange formats with respect to support machine learning algorithms.
2. Introducing uncertainties and probabilistic description to VO standards and services.
   
   
3. Presenting and collecting best practice examples of scientific data analytics in astronomy.
4. Defining requirements for implementing and adding machine learning capabilities to services.
5. Coordinating and unifying the access to data visualization functionalities.
6. Discussing the aspect of data provenance with respect to data used to derive/train models.
7. Introducing proper statistical scoring and evaluation methods as services.
8. Contributing to the discussion on scripting and orchestrating the scientific discovery workflow.
9. Supporting the development of dedicated knowledge discovery applications.

<-- 
We will develop and test scalable data mining algorithms and the accompanying new standards for VO interfaces and protocols, so that these algorithms can be discovered and used transparently within VO science workflows or in standalone data exploration applications. Therefore the activities of the KDD-IG will be:
 
 
 Support the definition of an ontology of the KDD tasks required by the astronomical community. This ontology will be used to define programming and documentation standards. 
 
 Make an inventory of existing methods relevant for astrophysical applications (more than 100 new KDD models and methods appear every month on specialized journals). 
 
 Identify reference data sets to be used for comparing, debugging and testing methods and tools. 
 
 Foster the implementation, using available VO standards and methods, of general purpose data exploration and data mining methods which will allow the general user to seamlessly exploit the complex data repositories offered by the VO. 
 
 Provide/receive feedbacks to/from the WGs in order to improve the usability of VO tools and standards. 
 
 Provide/receive from the community information to improve both the usability and the potentialities of Data Mining tools under the VO. 
 
 Define and pursue specific science cases which will be used to showcase the VO capabilities to the community. 
 
More important than anything else, we wish to use this IG as an arena where different groups can share experiences and plan future developments.
 
 
 -->

Motivations

During the Strasbourg InterOp Meeting it emerged the need for an Interest Group on Data Mining (KD-IG) as an indispensable step to bridge the Virtual Observatory Infrastructure with the expected VO science. In fact, "...Data mining, or KDD, is the semi-automatic discovery of patterns, associations, changes, anomalies, and statistically significant structures and events in data. In other words, traditional data analysis is assumption driven as a hypothesis is formed and validated against the data. Data mining, in contrast, is discovery driven as the patterns are automatically extracted from data....”

<-- 
As such, Data Mining (DM) can be considered as the “frontier” of VO enabled science since it represents the only way to capture and reveal the scientific knowledge (patterns, trends, correlations, etc.) hidden behind the complexity of Massive Data Sets.
 -->

Data Mining is a rapidly evolving set of methodologies which needs to be imported under the VO umbrella and not just another application. As such, DM cannot be just a tool or a suite of tools offered by a group of developers to a “passive community”. Data Mining involves a large number of researchers across many domains. The astronomical community, which has only recently entered the Massive Data Sets era, makes use of just a handful of methods and tools which very often are far from optimal. The synergy of different expertise present in the IVOA makes it the ideal arena for exploring new and more modern approaches.

KD-IG requires a strong and continuous interaction with the scientific community which, besides testing the proposed solutions, methods, and tools, will also provide feedback and inputs aiming at extending the scientific capabilities of the VO.

The KD-IG will interfaces to many other IVOA working and interest groups: Applications, Semantics, Time Domain, Data Models, Grid & Web Services, and Resource Registry. This cross- discipline nature is also a primary reason to create a specific IG. Data Mining, in fact, addresses sophisticated and extreme modes of usage which require a careful orchestration and fine tuning of standards, methods, and tools provided by the other IVOA WGs and IGs. Typical examples are the automatic extraction of bases of knowledge from VO archives using VO ontologies; the transparent access to large computational facilities regardless the computational paradigm; the automated switching from asynchronous to synchronous mode of data access; and the extreme usage of workflows and advanced visualization methods. Furthermore, effective KD requires the possibility for an inexperienced user to contribute, or at least seamlessly use under the VO infrastructure, his/her own KD routines and methods. This situation puts strong requirements on security issues and opens new problems for ticketing and scheduling. In other words, the KD-IG will provide feedback to the solutions implemented by the WG’s and, by posing new operational problems, will stimulate the development and adoption of new solutions and standards.

We also wish to stress that, in ultimate analysis, the goal of the KD-IG is to allow the VO to produce new scientific knowledge publishable in astronomical journals. On the one end its activities will contribute to demonstrate to the community the power and necessity of federated access to the vast VO universe of data and, on the other, KD-IG will illustrate the power and performance of data mining algorithms to facilitate and accelerate astronomical discovery within this data universe.

KD-IG Meetings

Other Interesting Meetings for KD-IG

<-- 
Next tasks
 
 
 
 
 Definition of a taxonomy of Data Mining models. This taxonomy will contribute to the Standard Vocabulary of the Semantics WG2. 
 
 Definition of the requirements which a Data Mining model needs to match in order to be imported under the VObs standards. 
 
 Inventory of existing Data Mining models of relevant astrophysical interest. 
 
 Definition of standard template data sets for Data Mining models test and debugging. 
 
 Definition of standard data sets to be used as bases of knowledge for debugging and test of supervised methods. 
 
 Definition of procedures to extract and validate robust bases of knowledge from the VObs data archives using the VObs ontology. 
 
 Study of the scalability of Data Mining models under different computing infrastructures (definition of best benchmarks). 
 
-->

Related Topics

In the past hot topics had been identified. These are following the priorities emerged during the first KD-IG meeting held at the IVOA.InterOpMay2010KDD in Victoria, and singled out by the Chair in his welcome message to the members.

Please follow the links and edit the specific pages:

1. Dictionary of Data Mining terms;
2. Census of Data Mining and Machine Learning tools and methods of astronomical interest;
3. Template datasets for algorithm benchmarking;
4. A user guide for Knowledge Discovery in Databases in Astronomy;
5. Knowledge Discovery in Databases and VO standards;
6. Specific fields of applications of KDD in astronomical research;

Members

Chair: RaffaeleDAbrusco

Vice Chair: n/a

Task Force Members:

RaffaeleDAbrusco
Rick Ebert
MatthewGraham
Steve Groom
Franck Le Petit
Kenny Lo
Jiri Nadvornik
Kai Lars Polsterer
PetrSkoda
HerveWozniak

Passive Members:

Sheelu Abraham
Lauretta Auvil
NickBall
ThomasBoch
Sen Bodhisattva
KirkBorne
MaxBrescia
Robert Brunner
Darren Davis
ReinaldoDeCarvalho
SebastienDerriere
GiuseppeLongo
CiroDonalek
Paul Eglitis
Pepi Fabbiano
PierreFernique
Peter Freeman
Fabian Gieseke
Alyssa Goodman
Alexander Gray
Paul Green
BobHanisch
Sheth Kartik
AjitKembhavi
OmarLaurino
Ann Lee
JeffLusted
AshishMahabal
BobMann
William March
Joseph Mazzarella
SabineMcConnell
Fionn Murtagh
NinanSajeethPhilip
Rebecca Nugent
PaoloPadovani
FabioPasian
Misha Pesenson
Manuel Luis Sarro Baro
RiccardoSmareglia
Antonino Staiano
RoyWilliams
Chris Stubbs
NicholasWalton
Yongheng Zhao

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

IVOA Knowledge Discovery Interest Group

Charter

Knowledge discovery is the task of processing and analyzing data-sets with the aim of extracting new knowledge. This area spans widely across multiple disceplins, including visualization, remote data exploration, machine learning techniques, statistical methods, workflow orchestration, and polymorphic data access. To support the process of discovery, the KD-IG interacts closely with the other working/interest groups and feeds back requirements of the scientific community.

The activities of the KD-IG include the following items with a strong emphasis on the first two points:

1. Participating in the definition of new data preservation and exchange formats with respect to support machine learning algorithms.
2. Introducing uncertainties and probabilistic description to VO standards and services.
   
   
3. Presenting and collecting best practice examples of scientific data analytics in astronomy.
4. Defining requirements for implementing and adding machine learning capabilities to services.
5. Coordinating and unifying the access to data visualization functionalities.
6. Discussing the aspect of data provenance with respect to data used to derive/train models.
7. Introducing proper statistical scoring and evaluation methods as services.
8. Contributing to the discussion on scripting and orchestrating the scientific discovery workflow.
9. Supporting the development of dedicated knowledge discovery applications.

<-- 
We will develop and test scalable data mining algorithms and the accompanying new standards for VO interfaces and protocols, so that these algorithms can be discovered and used transparently within VO science workflows or in standalone data exploration applications. Therefore the activities of the KDD-IG will be:
 
 
 Support the definition of an ontology of the KDD tasks required by the astronomical community. This ontology will be used to define programming and documentation standards. 
 
 Make an inventory of existing methods relevant for astrophysical applications (more than 100 new KDD models and methods appear every month on specialized journals). 
 
 Identify reference data sets to be used for comparing, debugging and testing methods and tools. 
 
 Foster the implementation, using available VO standards and methods, of general purpose data exploration and data mining methods which will allow the general user to seamlessly exploit the complex data repositories offered by the VO. 
 
 Provide/receive feedbacks to/from the WGs in order to improve the usability of VO tools and standards. 
 
 Provide/receive from the community information to improve both the usability and the potentialities of Data Mining tools under the VO. 
 
 Define and pursue specific science cases which will be used to showcase the VO capabilities to the community. 
 
More important than anything else, we wish to use this IG as an arena where different groups can share experiences and plan future developments.
 
 
 -->

Motivations

During the Strasbourg InterOp Meeting it emerged the need for an Interest Group on Data Mining (KD-IG) as an indispensable step to bridge the Virtual Observatory Infrastructure with the expected VO science. In fact, "...Data mining, or KDD, is the semi-automatic discovery of patterns, associations, changes, anomalies, and statistically significant structures and events in data. In other words, traditional data analysis is assumption driven as a hypothesis is formed and validated against the data. Data mining, in contrast, is discovery driven as the patterns are automatically extracted from data....”

<-- 
As such, Data Mining (DM) can be considered as the “frontier” of VO enabled science since it represents the only way to capture and reveal the scientific knowledge (patterns, trends, correlations, etc.) hidden behind the complexity of Massive Data Sets.
 -->

Data Mining is a rapidly evolving set of methodologies which needs to be imported under the VO umbrella and not just another application. As such, DM cannot be just a tool or a suite of tools offered by a group of developers to a “passive community”. Data Mining involves a large number of researchers across many domains. The astronomical community, which has only recently entered the Massive Data Sets era, makes use of just a handful of methods and tools which very often are far from optimal. The synergy of different expertise present in the IVOA makes it the ideal arena for exploring new and more modern approaches.

KD-IG requires a strong and continuous interaction with the scientific community which, besides testing the proposed solutions, methods, and tools, will also provide feedback and inputs aiming at extending the scientific capabilities of the VO.

The KD-IG will interfaces to many other IVOA working and interest groups: Applications, Semantics, Time Domain, Data Models, Grid & Web Services, and Resource Registry. This cross- discipline nature is also a primary reason to create a specific IG. Data Mining, in fact, addresses sophisticated and extreme modes of usage which require a careful orchestration and fine tuning of standards, methods, and tools provided by the other IVOA WGs and IGs. Typical examples are the automatic extraction of bases of knowledge from VO archives using VO ontologies; the transparent access to large computational facilities regardless the computational paradigm; the automated switching from asynchronous to synchronous mode of data access; and the extreme usage of workflows and advanced visualization methods. Furthermore, effective KD requires the possibility for an inexperienced user to contribute, or at least seamlessly use under the VO infrastructure, his/her own KD routines and methods. This situation puts strong requirements on security issues and opens new problems for ticketing and scheduling. In other words, the KD-IG will provide feedback to the solutions implemented by the WG’s and, by posing new operational problems, will stimulate the development and adoption of new solutions and standards.

We also wish to stress that, in ultimate analysis, the goal of the KD-IG is to allow the VO to produce new scientific knowledge publishable in astronomical journals. On the one end its activities will contribute to demonstrate to the community the power and necessity of federated access to the vast VO universe of data and, on the other, KD-IG will illustrate the power and performance of data mining algorithms to facilitate and accelerate astronomical discovery within this data universe.

KD-IG Meetings

Other Interesting Meetings for KD-IG

<-- 
Next tasks
 
 
 
 
 Definition of a taxonomy of Data Mining models. This taxonomy will contribute to the Standard Vocabulary of the Semantics WG2. 
 
 Definition of the requirements which a Data Mining model needs to match in order to be imported under the VObs standards. 
 
 Inventory of existing Data Mining models of relevant astrophysical interest. 
 
 Definition of standard template data sets for Data Mining models test and debugging. 
 
 Definition of standard data sets to be used as bases of knowledge for debugging and test of supervised methods. 
 
 Definition of procedures to extract and validate robust bases of knowledge from the VObs data archives using the VObs ontology. 
 
 Study of the scalability of Data Mining models under different computing infrastructures (definition of best benchmarks). 
 
-->

Related Topics

In the past hot topics had been identified. These are following the priorities emerged during the first KD-IG meeting held at the IVOA.InterOpMay2010KDD in Victoria, and singled out by the Chair in his welcome message to the members.

Please follow the links and edit the specific pages:

1. Dictionary of Data Mining terms;
2. Census of Data Mining and Machine Learning tools and methods of astronomical interest;
3. Template datasets for algorithm benchmarking;
4. A user guide for Knowledge Discovery in Databases in Astronomy;
5. Knowledge Discovery in Databases and VO standards;
6. Specific fields of applications of KDD in astronomical research;

Members

Task Force Members:

RaffaeleDAbrusco
Rick Ebert
MatthewGraham
Steve Groom
Franck Le Petit
Kenny Lo
Jiri Nadvornik
Kai Lars Polsterer
PetrSkoda
HerveWozniak

Passive Members:

Sheelu Abraham
Lauretta Auvil
NickBall
ThomasBoch
Sen Bodhisattva
KirkBorne
MaxBrescia
Robert Brunner
Darren Davis
ReinaldoDeCarvalho
SebastienDerriere
GiuseppeLongo
CiroDonalek
Paul Eglitis
Pepi Fabbiano
PierreFernique
Peter Freeman
Fabian Gieseke
Alyssa Goodman
Alexander Gray
Paul Green
BobHanisch
Sheth Kartik
AjitKembhavi
OmarLaurino
Ann Lee
JeffLusted
AshishMahabal
BobMann
William March
Joseph Mazzarella
SabineMcConnell
Fionn Murtagh
NinanSajeethPhilip
Rebecca Nugent
PaoloPadovani
FabioPasian
Misha Pesenson
Manuel Luis Sarro Baro
RiccardoSmareglia
Antonino Staiano
RoyWilliams
Chris Stubbs
NicholasWalton
Yongheng Zhao

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

IVOA Knowledge Discovery Interest Group

Charter

Knowledge discovery is the task of processing and analyzing data-sets with the aim of extracting new knowledge. This area spans widely across multiple disceplins, including visualization, remote data exploration, machine learning techniques, statistical methods, workflow orchestration, and polymorphic data access. To support the process of discovery, the KD-IG interacts closely with the other working/interest groups and feeds back requirements of the scientific community.

The activities of the KD-IG include the following items with a strong emphasis on the first two points:

1. Participating in the definition of new data preservation and exchange formats with respect to support machine learning algorithms.
2. Introducing uncertainties and probabilistic description to VO standards and services.
   
   
3. Presenting and collecting best practice examples of scientific data analytics in astronomy.
4. Defining requirements for implementing and adding machine learning capabilities to services.
5. Coordinating and unifying the access to data visualization functionalities.
6. Discussing the aspect of data provenance with respect to data used to derive/train models.
7. Introducing proper statistical scoring and evaluation methods as services.
8. Contributing to the discussion on scripting and orchestrating the scientific discovery workflow.
9. Supporting the development of dedicated knowledge discovery applications.

<-- 
We will develop and test scalable data mining algorithms and the accompanying new standards for VO interfaces and protocols, so that these algorithms can be discovered and used transparently within VO science workflows or in standalone data exploration applications. Therefore the activities of the KDD-IG will be:
 
 
 Support the definition of an ontology of the KDD tasks required by the astronomical community. This ontology will be used to define programming and documentation standards. 
 
 Make an inventory of existing methods relevant for astrophysical applications (more than 100 new KDD models and methods appear every month on specialized journals). 
 
 Identify reference data sets to be used for comparing, debugging and testing methods and tools. 
 
 Foster the implementation, using available VO standards and methods, of general purpose data exploration and data mining methods which will allow the general user to seamlessly exploit the complex data repositories offered by the VO. 
 
 Provide/receive feedbacks to/from the WGs in order to improve the usability of VO tools and standards. 
 
 Provide/receive from the community information to improve both the usability and the potentialities of Data Mining tools under the VO. 
 
 Define and pursue specific science cases which will be used to showcase the VO capabilities to the community. 
 
More important than anything else, we wish to use this IG as an arena where different groups can share experiences and plan future developments.
 
 
 -->

Motivations

During the Strasbourg InterOp Meeting it emerged the need for an Interest Group on Data Mining (KD-IG) as an indispensable step to bridge the Virtual Observatory Infrastructure with the expected VO science. In fact, "...Data mining, or KDD, is the semi-automatic discovery of patterns, associations, changes, anomalies, and statistically significant structures and events in data. In other words, traditional data analysis is assumption driven as a hypothesis is formed and validated against the data. Data mining, in contrast, is discovery driven as the patterns are automatically extracted from data....”

<-- 
As such, Data Mining (DM) can be considered as the “frontier” of VO enabled science since it represents the only way to capture and reveal the scientific knowledge (patterns, trends, correlations, etc.) hidden behind the complexity of Massive Data Sets.
 -->

Data Mining is a rapidly evolving set of methodologies which needs to be imported under the VO umbrella and not just another application. As such, DM cannot be just a tool or a suite of tools offered by a group of developers to a “passive community”. Data Mining involves a large number of researchers across many domains. The astronomical community, which has only recently entered the Massive Data Sets era, makes use of just a handful of methods and tools which very often are far from optimal. The synergy of different expertise present in the IVOA makes it the ideal arena for exploring new and more modern approaches.

KD-IG requires a strong and continuous interaction with the scientific community which, besides testing the proposed solutions, methods, and tools, will also provide feedback and inputs aiming at extending the scientific capabilities of the VO.

The KD-IG will interfaces to many other IVOA working and interest groups: Applications, Semantics, Time Domain, Data Models, Grid & Web Services, and Resource Registry. This cross- discipline nature is also a primary reason to create a specific IG. Data Mining, in fact, addresses sophisticated and extreme modes of usage which require a careful orchestration and fine tuning of standards, methods, and tools provided by the other IVOA WGs and IGs. Typical examples are the automatic extraction of bases of knowledge from VO archives using VO ontologies; the transparent access to large computational facilities regardless the computational paradigm; the automated switching from asynchronous to synchronous mode of data access; and the extreme usage of workflows and advanced visualization methods. Furthermore, effective KD requires the possibility for an inexperienced user to contribute, or at least seamlessly use under the VO infrastructure, his/her own KD routines and methods. This situation puts strong requirements on security issues and opens new problems for ticketing and scheduling. In other words, the KD-IG will provide feedback to the solutions implemented by the WG’s and, by posing new operational problems, will stimulate the development and adoption of new solutions and standards.

We also wish to stress that, in ultimate analysis, the goal of the KD-IG is to allow the VO to produce new scientific knowledge publishable in astronomical journals. On the one end its activities will contribute to demonstrate to the community the power and necessity of federated access to the vast VO universe of data and, on the other, KD-IG will illustrate the power and performance of data mining algorithms to facilitate and accelerate astronomical discovery within this data universe.

KD-IG Meetings

Other Interesting Meetings for KD-IG

<-- 
Next tasks
 
 
 
 
 Definition of a taxonomy of Data Mining models. This taxonomy will contribute to the Standard Vocabulary of the Semantics WG2. 
 
 Definition of the requirements which a Data Mining model needs to match in order to be imported under the VObs standards. 
 
 Inventory of existing Data Mining models of relevant astrophysical interest. 
 
 Definition of standard template data sets for Data Mining models test and debugging. 
 
 Definition of standard data sets to be used as bases of knowledge for debugging and test of supervised methods. 
 
 Definition of procedures to extract and validate robust bases of knowledge from the VObs data archives using the VObs ontology. 
 
 Study of the scalability of Data Mining models under different computing infrastructures (definition of best benchmarks). 
 
-->

Related Topics

In the past hot topics had been identified. These are following the priorities emerged during the first KD-IG meeting held at the IVOA.InterOpMay2010KDD in Victoria, and singled out by the Chair in his welcome message to the members.

Please follow the links and edit the specific pages:

1. Dictionary of Data Mining terms;
2. Census of Data Mining and Machine Learning tools and methods of astronomical interest;
3. Template datasets for algorithm benchmarking;
4. A user guide for Knowledge Discovery in Databases in Astronomy;
5. Knowledge Discovery in Databases and VO standards;
6. Specific fields of applications of KDD in astronomical research;

Members

Chair: Kai Lars Polsterer

Vice Chair: n/a

Task Force Members:

RaffaeleDAbrusco
Rick Ebert
MatthewGraham
Steve Groom
Franck Le Petit
Kenny Lo
Jiri Nadvornik
Kai Lars Polsterer
PetrSkoda
HerveWozniak

Passive Members:

Sheelu Abraham
Lauretta Auvil
NickBall
ThomasBoch
Sen Bodhisattva
KirkBorne
MaxBrescia
Robert Brunner
Darren Davis
ReinaldoDeCarvalho
SebastienDerriere
GiuseppeLongo
CiroDonalek
Paul Eglitis
Pepi Fabbiano
PierreFernique
Peter Freeman
Fabian Gieseke
Alyssa Goodman
Alexander Gray
Paul Green
BobHanisch
Sheth Kartik
AjitKembhavi
OmarLaurino
Ann Lee
JeffLusted
AshishMahabal
BobMann
William March
Joseph Mazzarella
SabineMcConnell
Fionn Murtagh
NinanSajeethPhilip
Rebecca Nugent
PaoloPadovani
FabioPasian
Misha Pesenson
Manuel Luis Sarro Baro
RiccardoSmareglia
Antonino Staiano
RoyWilliams
Chris Stubbs
NicholasWalton
Yongheng Zhao

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

IVOA Knowledge Discovery Interest Group

Charter

Knowledge discovery is the task of processing and analyzing data-sets with the aim of extracting new knowledge. This area spans widely across multiple disceplins, including visualization, remote data exploration, machine learning techniques, statistical methods, workflow orchestration, and polymorphic data access. To support the process of discovery, the KD-IG interacts closely with the other working/interest groups and feeds back requirements of the scientific community.

The activities of the KD-IG include the following items with a strong emphasis on the first two points:

1. Participating in the definition of new data preservation and exchange formats with respect to support machine learning algorithms.
2. Introducing uncertainties and probabilistic description to VO standards and services.
   
   
3. Presenting and collecting best practice examples of scientific data analytics in astronomy.
4. Defining requirements for implementing and adding machine learning capabilities to services.
5. Coordinating and unifying the access to data visualization functionalities.
6. Discussing the aspect of data provenance with respect to data used to derive/train models.
7. Introducing proper statistical scoring and evaluation methods as services.
8. Contributing to the discussion on scripting and orchestrating the scientific discovery workflow.
9. Supporting the development of dedicated knowledge discovery applications.

<-- 
We will develop and test scalable data mining algorithms and the accompanying new standards for VO interfaces and protocols, so that these algorithms can be discovered and used transparently within VO science workflows or in standalone data exploration applications. Therefore the activities of the KDD-IG will be:
 
 
 Support the definition of an ontology of the KDD tasks required by the astronomical community. This ontology will be used to define programming and documentation standards. 
 
 Make an inventory of existing methods relevant for astrophysical applications (more than 100 new KDD models and methods appear every month on specialized journals). 
 
 Identify reference data sets to be used for comparing, debugging and testing methods and tools. 
 
 Foster the implementation, using available VO standards and methods, of general purpose data exploration and data mining methods which will allow the general user to seamlessly exploit the complex data repositories offered by the VO. 
 
 Provide/receive feedbacks to/from the WGs in order to improve the usability of VO tools and standards. 
 
 Provide/receive from the community information to improve both the usability and the potentialities of Data Mining tools under the VO. 
 
 Define and pursue specific science cases which will be used to showcase the VO capabilities to the community. 
 
More important than anything else, we wish to use this IG as an arena where different groups can share experiences and plan future developments.
 
 
 -->

Motivations

During the Strasbourg InterOp Meeting it emerged the need for an Interest Group on Data Mining (KD-IG) as an indispensable step to bridge the Virtual Observatory Infrastructure with the expected VO science. In fact, "...Data mining, or KDD, is the semi-automatic discovery of patterns, associations, changes, anomalies, and statistically significant structures and events in data. In other words, traditional data analysis is assumption driven as a hypothesis is formed and validated against the data. Data mining, in contrast, is discovery driven as the patterns are automatically extracted from data....”

<-- 
As such, Data Mining (DM) can be considered as the “frontier” of VO enabled science since it represents the only way to capture and reveal the scientific knowledge (patterns, trends, correlations, etc.) hidden behind the complexity of Massive Data Sets.
 -->

Data Mining is a rapidly evolving set of methodologies which needs to be imported under the VO umbrella and not just another application. As such, DM cannot be just a tool or a suite of tools offered by a group of developers to a “passive community”. Data Mining involves a large number of researchers across many domains. The astronomical community, which has only recently entered the Massive Data Sets era, makes use of just a handful of methods and tools which very often are far from optimal. The synergy of different expertise present in the IVOA makes it the ideal arena for exploring new and more modern approaches.

KD-IG requires a strong and continuous interaction with the scientific community which, besides testing the proposed solutions, methods, and tools, will also provide feedback and inputs aiming at extending the scientific capabilities of the VO.

The KD-IG will interfaces to many other IVOA working and interest groups: Applications, Semantics, Time Domain, Data Models, Grid & Web Services, and Resource Registry. This cross- discipline nature is also a primary reason to create a specific IG. Data Mining, in fact, addresses sophisticated and extreme modes of usage which require a careful orchestration and fine tuning of standards, methods, and tools provided by the other IVOA WGs and IGs. Typical examples are the automatic extraction of bases of knowledge from VO archives using VO ontologies; the transparent access to large computational facilities regardless the computational paradigm; the automated switching from asynchronous to synchronous mode of data access; and the extreme usage of workflows and advanced visualization methods. Furthermore, effective KD requires the possibility for an inexperienced user to contribute, or at least seamlessly use under the VO infrastructure, his/her own KD routines and methods. This situation puts strong requirements on security issues and opens new problems for ticketing and scheduling. In other words, the KD-IG will provide feedback to the solutions implemented by the WG’s and, by posing new operational problems, will stimulate the development and adoption of new solutions and standards.

We also wish to stress that, in ultimate analysis, the goal of the KD-IG is to allow the VO to produce new scientific knowledge publishable in astronomical journals. On the one end its activities will contribute to demonstrate to the community the power and necessity of federated access to the vast VO universe of data and, on the other, KD-IG will illustrate the power and performance of data mining algorithms to facilitate and accelerate astronomical discovery within this data universe.

KD-IG Meetings

Other Interesting Meetings for KD-IG

<-- 
Next tasks
 
 
 
 
 Definition of a taxonomy of Data Mining models. This taxonomy will contribute to the Standard Vocabulary of the Semantics WG2. 
 
 Definition of the requirements which a Data Mining model needs to match in order to be imported under the VObs standards. 
 
 Inventory of existing Data Mining models of relevant astrophysical interest. 
 
 Definition of standard template data sets for Data Mining models test and debugging. 
 
 Definition of standard data sets to be used as bases of knowledge for debugging and test of supervised methods. 
 
 Definition of procedures to extract and validate robust bases of knowledge from the VObs data archives using the VObs ontology. 
 
 Study of the scalability of Data Mining models under different computing infrastructures (definition of best benchmarks). 
 
-->

Related Topics

In the past hot topics had been identified. These are following the priorities emerged during the first KD-IG meeting held at the IVOA.InterOpMay2010KDD in Victoria, and singled out by the Chair in his welcome message to the members.

Please follow the links and edit the specific pages:

1. Dictionary of Data Mining terms;
2. Census of Data Mining and Machine Learning tools and methods of astronomical interest;
3. Template datasets for algorithm benchmarking;
4. A user guide for Knowledge Discovery in Databases in Astronomy;
5. Knowledge Discovery in Databases and VO standards;
6. Specific fields of applications of KDD in astronomical research;

Members

Chair: Kai Lars Polsterer

Vice Chair: n/a

Task Force Members:

RaffaeleDAbrusco
Rick Ebert
MatthewGraham
Steve Groom
Franck Le Petit
Kenny Lo
Jiri Nadvornik
Kai Lars Polsterer
PetrSkoda
HerveWozniak

Passive Members:

Sheelu Abraham
Lauretta Auvil
NickBall
ThomasBoch
Sen Bodhisattva
KirkBorne
MaxBrescia
Robert Brunner
Darren Davis
ReinaldoDeCarvalho
SebastienDerriere
GiuseppeLongo
CiroDonalek
Paul Eglitis
Pepi Fabbiano
PierreFernique
Peter Freeman
Fabian Gieseke
Alyssa Goodman
Alexander Gray
Paul Green
BobHanisch
Sheth Kartik
AjitKembhavi
OmarLaurino
Ann Lee
JeffLusted
AshishMahabal
BobMann
William March
Joseph Mazzarella
SabineMcConnell
Fionn Murtagh
NinanSajeethPhilip
Rebecca Nugent
PaoloPadovani
FabioPasian
Misha Pesenson
Manuel Luis Sarro Baro
RiccardoSmareglia
Antonino Staiano
RoyWilliams
Chris Stubbs
NicholasWalton
Yongheng Zhao

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

IVOA Knowledge Discovery Interest Group

Charter

Knowledge discovery is the task of processing and analyzing data-sets with the aim of extracting new knowledge. This area spans widely across multiple disceplins, including visualization, remote data exploration, machine learning techniques, statistical methods, workflow orchestration, and polymorphic data access. To support the process of discovery, the KD-IG interacts closely with the other working/interest groups and feeds back requirements of the scientific community.

The activities of the KD-IG include the following items with a strong emphasis on the first two points:

1. Participating in the definition of new data preservation and exchange formats with respect to support machine learning algorithms.
2. Introducing uncertainties and probabilistic description to VO standards and services.
   
   
3. Presenting and collecting best practice examples of scientific data analytics in astronomy.
4. Defining requirements for implementing and adding machine learning capabilities to services.
5. Coordinating and unifying the access to data visualization functionalities.
6. Discussing the aspect of data provenance with respect to data used to derive/train models.
7. Introducing proper statistical scoring and evaluation methods as services.
8. Contributing to the discussion on scripting and orchestrating the scientific discovery workflow.
9. Supporting the development of dedicated knowledge discovery applications.

<-- 
We will develop and test scalable data mining algorithms and the accompanying new standards for VO interfaces and protocols, so that these algorithms can be discovered and used transparently within VO science workflows or in standalone data exploration applications. Therefore the activities of the KDD-IG will be:
 
 
 Support the definition of an ontology of the KDD tasks required by the astronomical community. This ontology will be used to define programming and documentation standards. 
 
 Make an inventory of existing methods relevant for astrophysical applications (more than 100 new KDD models and methods appear every month on specialized journals). 
 
 Identify reference data sets to be used for comparing, debugging and testing methods and tools. 
 
 Foster the implementation, using available VO standards and methods, of general purpose data exploration and data mining methods which will allow the general user to seamlessly exploit the complex data repositories offered by the VO. 
 
 Provide/receive feedbacks to/from the WGs in order to improve the usability of VO tools and standards. 
 
 Provide/receive from the community information to improve both the usability and the potentialities of Data Mining tools under the VO. 
 
 Define and pursue specific science cases which will be used to showcase the VO capabilities to the community. 
 
More important than anything else, we wish to use this IG as an arena where different groups can share experiences and plan future developments.
 
 
 -->

Motivations

During the Strasbourg InterOp Meeting it emerged the need for an Interest Group on Data Mining (KD-IG) as an indispensable step to bridge the Virtual Observatory Infrastructure with the expected VO science. In fact, "...Data mining, or KDD, is the semi-automatic discovery of patterns, associations, changes, anomalies, and statistically significant structures and events in data. In other words, traditional data analysis is assumption driven as a hypothesis is formed and validated against the data. Data mining, in contrast, is discovery driven as the patterns are automatically extracted from data....”

<-- 
As such, Data Mining (DM) can be considered as the “frontier” of VO enabled science since it represents the only way to capture and reveal the scientific knowledge (patterns, trends, correlations, etc.) hidden behind the complexity of Massive Data Sets.
 -->

Data Mining is a rapidly evolving set of methodologies which needs to be imported under the VO umbrella and not just another application. As such, DM cannot be just a tool or a suite of tools offered by a group of developers to a “passive community”. Data Mining involves a large number of researchers across many domains. The astronomical community, which has only recently entered the Massive Data Sets era, makes use of just a handful of methods and tools which very often are far from optimal. The synergy of different expertise present in the IVOA makes it the ideal arena for exploring new and more modern approaches.

KD-IG requires a strong and continuous interaction with the scientific community which, besides testing the proposed solutions, methods, and tools, will also provide feedback and inputs aiming at extending the scientific capabilities of the VO.

The KD-IG will interfaces to many other IVOA working and interest groups: Applications, Semantics, Time Domain, Data Models, Grid & Web Services, and Resource Registry. This cross- discipline nature is also a primary reason to create a specific IG. Data Mining, in fact, addresses sophisticated and extreme modes of usage which require a careful orchestration and fine tuning of standards, methods, and tools provided by the other IVOA WGs and IGs. Typical examples are the automatic extraction of bases of knowledge from VO archives using VO ontologies; the transparent access to large computational facilities regardless the computational paradigm; the automated switching from asynchronous to synchronous mode of data access; and the extreme usage of workflows and advanced visualization methods. Furthermore, effective KD requires the possibility for an inexperienced user to contribute, or at least seamlessly use under the VO infrastructure, his/her own KD routines and methods. This situation puts strong requirements on security issues and opens new problems for ticketing and scheduling. In other words, the KD-IG will provide feedback to the solutions implemented by the WG’s and, by posing new operational problems, will stimulate the development and adoption of new solutions and standards.

We also wish to stress that, in ultimate analysis, the goal of the KD-IG is to allow the VO to produce new scientific knowledge publishable in astronomical journals. On the one end its activities will contribute to demonstrate to the community the power and necessity of federated access to the vast VO universe of data and, on the other, KD-IG will illustrate the power and performance of data mining algorithms to facilitate and accelerate astronomical discovery within this data universe.

KD-IG Meetings

Other Interesting Meetings for KD-IG

<-- 
Next tasks
 
 
 
 
 Definition of a taxonomy of Data Mining models. This taxonomy will contribute to the Standard Vocabulary of the Semantics WG2. 
 
 Definition of the requirements which a Data Mining model needs to match in order to be imported under the VObs standards. 
 
 Inventory of existing Data Mining models of relevant astrophysical interest. 
 
 Definition of standard template data sets for Data Mining models test and debugging. 
 
 Definition of standard data sets to be used as bases of knowledge for debugging and test of supervised methods. 
 
 Definition of procedures to extract and validate robust bases of knowledge from the VObs data archives using the VObs ontology. 
 
 Study of the scalability of Data Mining models under different computing infrastructures (definition of best benchmarks). 
 
-->

Related Topics

In the past hot topics had been identified. These are following the priorities emerged during the first KD-IG meeting held at the IVOA.InterOpMay2010KDD in Victoria, and singled out by the Chair in his welcome message to the members.

Please follow the links and edit the specific pages:

1. Dictionary of Data Mining terms;
2. Census of Data Mining and Machine Learning tools and methods of astronomical interest;
3. Template datasets for algorithm benchmarking;
4. A user guide for Knowledge Discovery in Databases in Astronomy;
5. Knowledge Discovery in Databases and VO standards;
6. Specific fields of applications of KDD in astronomical research;

Members

Chair: Kai Lars Polsterer

Vice Chair: n/a

Task Force Members:

RaffaeleDAbrusco
Rick Ebert
MatthewGraham
Steve Groom
Franck Le Petit
Kenny Lo
Jiri Nadvornik
Kai Lars Polsterer
PetrSkoda
HerveWozniak

Passive Members:

Sheelu Abraham
Lauretta Auvil
NickBall
ThomasBoch
Sen Bodhisattva
KirkBorne
MaxBrescia
Robert Brunner
Darren Davis
ReinaldoDeCarvalho
SebastienDerriere
GiuseppeLongo
CiroDonalek
Paul Eglitis
Pepi Fabbiano
PierreFernique
Peter Freeman
Fabian Gieseke
Alyssa Goodman
Alexander Gray
Paul Green
BobHanisch
Sheth Kartik
AjitKembhavi
OmarLaurino
Ann Lee
JeffLusted
AshishMahabal
BobMann
William March
Joseph Mazzarella
SabineMcConnell
Fionn Murtagh
NinanSajeethPhilip
Rebecca Nugent
PaoloPadovani
FabioPasian
Misha Pesenson
Manuel Luis Sarro Baro
RiccardoSmareglia
Antonino Staiano
RoyWilliams
Chris Stubbs
NicholasWalton
Yongheng Zhao

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

IVOA Knowledge Discovery Interest Group

Charter

Knowledge discovery is the task of processing and analyzing data-sets with the aim of extracting new knowledge. This area spans widely across multiple disceplins, including visualization, remote data exploration, machine learning techniques, statistical methods, workflow orchestration, and polymorphic data access. To support the process of discovery, the KD-IG interacts closely with the other working/interest groups and feeds back requirements of the scientific community.

The activities of the KD-IG include the following items with a strong emphasis on the first two points:

1. Participating in the definition of new data preservation and exchange formats with respect to support machine learning algorithms.
2. Introducing uncertainties and probabilistic description to VO standards and services.
   
   
3. Presenting and collecting best practice examples of scientific data analytics in astronomy.
4. Defining requirements for implementing and adding machine learning capabilities to services.
5. Coordinating and unifying the access to data visualization functionalities.
6. Discussing the aspect of data provenance with respect to data used to derive/train models.
7. Introducing proper statistical scoring and evaluation methods as services.
8. Contributing to the discussion on scripting and orchestrating the scientific discovery workflow.
9. Supporting the development of dedicated knowledge discovery applications.

<-- 
We will develop and test scalable data mining algorithms and the accompanying new standards for VO interfaces and protocols, so that these algorithms can be discovered and used transparently within VO science workflows or in standalone data exploration applications. Therefore the activities of the KDD-IG will be:
 
 
 Support the definition of an ontology of the KDD tasks required by the astronomical community. This ontology will be used to define programming and documentation standards. 
 
 Make an inventory of existing methods relevant for astrophysical applications (more than 100 new KDD models and methods appear every month on specialized journals). 
 
 Identify reference data sets to be used for comparing, debugging and testing methods and tools. 
 
 Foster the implementation, using available VO standards and methods, of general purpose data exploration and data mining methods which will allow the general user to seamlessly exploit the complex data repositories offered by the VO. 
 
 Provide/receive feedbacks to/from the WGs in order to improve the usability of VO tools and standards. 
 
 Provide/receive from the community information to improve both the usability and the potentialities of Data Mining tools under the VO. 
 
 Define and pursue specific science cases which will be used to showcase the VO capabilities to the community. 
 
More important than anything else, we wish to use this IG as an arena where different groups can share experiences and plan future developments.
 
 
 -->

Motivations

During the Strasbourg InterOp Meeting it emerged the need for an Interest Group on Data Mining (KD-IG) as an indispensable step to bridge the Virtual Observatory Infrastructure with the expected VO science. In fact, "...Data mining, or KDD, is the semi-automatic discovery of patterns, associations, changes, anomalies, and statistically significant structures and events in data. In other words, traditional data analysis is assumption driven as a hypothesis is formed and validated against the data. Data mining, in contrast, is discovery driven as the patterns are automatically extracted from data....”

<-- 
As such, Data Mining (DM) can be considered as the “frontier” of VO enabled science since it represents the only way to capture and reveal the scientific knowledge (patterns, trends, correlations, etc.) hidden behind the complexity of Massive Data Sets.
 -->

Data Mining is a rapidly evolving set of methodologies which needs to be imported under the VO umbrella and not just another application. As such, DM cannot be just a tool or a suite of tools offered by a group of developers to a “passive community”. Data Mining involves a large number of researchers across many domains. The astronomical community, which has only recently entered the Massive Data Sets era, makes use of just a handful of methods and tools which very often are far from optimal. The synergy of different expertise present in the IVOA makes it the ideal arena for exploring new and more modern approaches.

KD-IG requires a strong and continuous interaction with the scientific community which, besides testing the proposed solutions, methods, and tools, will also provide feedback and inputs aiming at extending the scientific capabilities of the VO.

The KD-IG will interfaces to many other IVOA working and interest groups: Applications, Semantics, Time Domain, Data Models, Grid & Web Services, and Resource Registry. This cross- discipline nature is also a primary reason to create a specific IG. Data Mining, in fact, addresses sophisticated and extreme modes of usage which require a careful orchestration and fine tuning of standards, methods, and tools provided by the other IVOA WGs and IGs. Typical examples are the automatic extraction of bases of knowledge from VO archives using VO ontologies; the transparent access to large computational facilities regardless the computational paradigm; the automated switching from asynchronous to synchronous mode of data access; and the extreme usage of workflows and advanced visualization methods. Furthermore, effective KD requires the possibility for an inexperienced user to contribute, or at least seamlessly use under the VO infrastructure, his/her own KD routines and methods. This situation puts strong requirements on security issues and opens new problems for ticketing and scheduling. In other words, the KD-IG will provide feedback to the solutions implemented by the WG’s and, by posing new operational problems, will stimulate the development and adoption of new solutions and standards.

We also wish to stress that, in ultimate analysis, the goal of the KD-IG is to allow the VO to produce new scientific knowledge publishable in astronomical journals. On the one end its activities will contribute to demonstrate to the community the power and necessity of federated access to the vast VO universe of data and, on the other, KD-IG will illustrate the power and performance of data mining algorithms to facilitate and accelerate astronomical discovery within this data universe.

KD-IG Meetings

Other Interesting Meetings for KD-IG

<-- 
Next tasks
 
 
 
 
 Definition of a taxonomy of Data Mining models. This taxonomy will contribute to the Standard Vocabulary of the Semantics WG2. 
 
 Definition of the requirements which a Data Mining model needs to match in order to be imported under the VObs standards. 
 
 Inventory of existing Data Mining models of relevant astrophysical interest. 
 
 Definition of standard template data sets for Data Mining models test and debugging. 
 
 Definition of standard data sets to be used as bases of knowledge for debugging and test of supervised methods. 
 
 Definition of procedures to extract and validate robust bases of knowledge from the VObs data archives using the VObs ontology. 
 
 Study of the scalability of Data Mining models under different computing infrastructures (definition of best benchmarks). 
 
-->

Related Topics

In the past hot topics had been identified. These are following the priorities emerged during the first KD-IG meeting held at the IVOA.InterOpMay2010KDD in Victoria, and singled out by the Chair in his welcome message to the members.

Please follow the links and edit the specific pages:

1. Dictionary of Data Mining terms;
2. Census of Data Mining and Machine Learning tools and methods of astronomical interest;
3. Template datasets for algorithm benchmarking;
4. A user guide for Knowledge Discovery in Databases in Astronomy;
5. Knowledge Discovery in Databases and VO standards;
6. Specific fields of applications of KDD in astronomical research;

Members

Chair: Kai Lars Polsterer

Vice Chair: n/a

Task Force Members:

RaffaeleDAbrusco
Rick Ebert
MatthewGraham
Steve Groom
Franck Le Petit
Kenny Lo
Jiri Nadvornik
Kai Lars Polsterer
PetrSkoda
HerveWozniak

Passive Members:

Sheelu Abraham
Lauretta Auvil
NickBall
ThomasBoch
Sen Bodhisattva
KirkBorne
MaxBrescia
Robert Brunner
Darren Davis
ReinaldoDeCarvalho
SebastienDerriere
GiuseppeLongo
CiroDonalek
Paul Eglitis
Pepi Fabbiano
PierreFernique
Peter Freeman
Fabian Gieseke
Alyssa Goodman
Alexander Gray
Paul Green
BobHanisch
Sheth Kartik
AjitKembhavi
OmarLaurino
Ann Lee
JeffLusted
AshishMahabal
BobMann
William March
Joseph Mazzarella
SabineMcConnell
Fionn Murtagh
NinanSajeethPhilip
Rebecca Nugent
PaoloPadovani
FabioPasian
Misha Pesenson
Manuel Luis Sarro Baro
RiccardoSmareglia
Antonino Staiano
RoyWilliams
Chris Stubbs
NicholasWalton
Yongheng Zhao

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

IVOA Knowledge Discovery Interest Group

Charter

Knowledge discovery is the task of processing and analyzing data-sets with the aim of extracting new knowledge. This area spans widely across multiple disceplins, including visualization, remote data exploration, machine learning techniques, statistical methods, workflow orchestration, and polymorphic data access. To support the process of discovery, the KD-IG interacts closely with the other working/interest groups and feeds back requirements of the scientific community.

The activities of the KD-IG include the following items with a strong emphasis on the first two points:

1. Participating in the definition of new data preservation and exchange formats with respect to support machine learning algorithms.
2. Introducing uncertainties and probabilistic description to VO standards and services.
   
   
3. Presenting and collecting best practice examples of scientific data analytics in astronomy.
4. Defining requirements for implementing and adding machine learning capabilities to services.
5. Coordinating and unifying the access to data visualization functionalities.
6. Discussing the aspect of data provenance with respect to data used to derive/train models.
7. Introducing proper statistical scoring and evaluation methods as services.
8. Contributing to the discussion on scripting and orchestrating the scientific discovery workflow.
9. Supporting the development of dedicated knowledge discovery applications.

<-- 
We will develop and test scalable data mining algorithms and the accompanying new standards for VO interfaces and protocols, so that these algorithms can be discovered and used transparently within VO science workflows or in standalone data exploration applications. Therefore the activities of the KDD-IG will be:
 
 
 Support the definition of an ontology of the KDD tasks required by the astronomical community. This ontology will be used to define programming and documentation standards. 
 
 Make an inventory of existing methods relevant for astrophysical applications (more than 100 new KDD models and methods appear every month on specialized journals). 
 
 Identify reference data sets to be used for comparing, debugging and testing methods and tools. 
 
 Foster the implementation, using available VO standards and methods, of general purpose data exploration and data mining methods which will allow the general user to seamlessly exploit the complex data repositories offered by the VO. 
 
 Provide/receive feedbacks to/from the WGs in order to improve the usability of VO tools and standards. 
 
 Provide/receive from the community information to improve both the usability and the potentialities of Data Mining tools under the VO. 
 
 Define and pursue specific science cases which will be used to showcase the VO capabilities to the community. 
 
More important than anything else, we wish to use this IG as an arena where different groups can share experiences and plan future developments.
 
 
 -->

Motivations

During the Strasbourg InterOp Meeting it emerged the need for an Interest Group on Data Mining (KD-IG) as an indispensable step to bridge the Virtual Observatory Infrastructure with the expected VO science. In fact, "...Data mining, or KDD, is the semi-automatic discovery of patterns, associations, changes, anomalies, and statistically significant structures and events in data. In other words, traditional data analysis is assumption driven as a hypothesis is formed and validated against the data. Data mining, in contrast, is discovery driven as the patterns are automatically extracted from data....”

<-- 
As such, Data Mining (DM) can be considered as the “frontier” of VO enabled science since it represents the only way to capture and reveal the scientific knowledge (patterns, trends, correlations, etc.) hidden behind the complexity of Massive Data Sets.
 -->

Data Mining is a rapidly evolving set of methodologies which needs to be imported under the VO umbrella and not just another application. As such, DM cannot be just a tool or a suite of tools offered by a group of developers to a “passive community”. Data Mining involves a large number of researchers across many domains. The astronomical community, which has only recently entered the Massive Data Sets era, makes use of just a handful of methods and tools which very often are far from optimal. The synergy of different expertise present in the IVOA makes it the ideal arena for exploring new and more modern approaches.

KD-IG requires a strong and continuous interaction with the scientific community which, besides testing the proposed solutions, methods, and tools, will also provide feedback and inputs aiming at extending the scientific capabilities of the VO.

The KD-IG will interfaces to many other IVOA working and interest groups: Applications, Semantics, Time Domain, Data Models, Grid & Web Services, and Resource Registry. This cross- discipline nature is also a primary reason to create a specific IG. Data Mining, in fact, addresses sophisticated and extreme modes of usage which require a careful orchestration and fine tuning of standards, methods, and tools provided by the other IVOA WGs and IGs. Typical examples are the automatic extraction of bases of knowledge from VO archives using VO ontologies; the transparent access to large computational facilities regardless the computational paradigm; the automated switching from asynchronous to synchronous mode of data access; and the extreme usage of workflows and advanced visualization methods. Furthermore, effective KD requires the possibility for an inexperienced user to contribute, or at least seamlessly use under the VO infrastructure, his/her own KD routines and methods. This situation puts strong requirements on security issues and opens new problems for ticketing and scheduling. In other words, the KD-IG will provide feedback to the solutions implemented by the WG’s and, by posing new operational problems, will stimulate the development and adoption of new solutions and standards.

We also wish to stress that, in ultimate analysis, the goal of the KD-IG is to allow the VO to produce new scientific knowledge publishable in astronomical journals. On the one end its activities will contribute to demonstrate to the community the power and necessity of federated access to the vast VO universe of data and, on the other, KD-IG will illustrate the power and performance of data mining algorithms to facilitate and accelerate astronomical discovery within this data universe.

KD-IG Meetings

Other Interesting Meetings for KD-IG

<-- 
Next tasks
 
 
 
 
 Definition of a taxonomy of Data Mining models. This taxonomy will contribute to the Standard Vocabulary of the Semantics WG2. 
 
 Definition of the requirements which a Data Mining model needs to match in order to be imported under the VObs standards. 
 
 Inventory of existing Data Mining models of relevant astrophysical interest. 
 
 Definition of standard template data sets for Data Mining models test and debugging. 
 
 Definition of standard data sets to be used as bases of knowledge for debugging and test of supervised methods. 
 
 Definition of procedures to extract and validate robust bases of knowledge from the VObs data archives using the VObs ontology. 
 
 Study of the scalability of Data Mining models under different computing infrastructures (definition of best benchmarks). 
 
-->

Related Topics

In the past hot topics had been identified. These are following the priorities emerged during the first KD-IG meeting held at the IVOA.InterOpMay2010KDD in Victoria, and singled out by the Chair in his welcome message to the members.

Please follow the links and edit the specific pages:

1. Dictionary of Data Mining terms;
2. Census of Data Mining and Machine Learning tools and methods of astronomical interest;
3. Template datasets for algorithm benchmarking;
4. A user guide for Knowledge Discovery in Databases in Astronomy;
5. Knowledge Discovery in Databases and VO standards;
6. Specific fields of applications of KDD in astronomical research;

Members

Chair: Kai Lars Polsterer

Vice Chair: n/a

Task Force Members:

Sheelu Abraham
Lauretta Auvil
NickBall
ThomasBoch
Sen Bodhisattva
KirkBorne
MaxBrescia
Robert Brunner
Darren Davis
ReinaldoDeCarvalho
SebastienDerriere
GiuseppeLongo
CiroDonalek
Paul Eglitis
Pepi Fabbiano
PierreFernique
Peter Freeman
Fabian Gieseke
Alyssa Goodman
Alexander Gray
Paul Green
BobHanisch
Sheth Kartik
AjitKembhavi
OmarLaurino
Ann Lee
JeffLusted
AshishMahabal
BobMann
William March
Joseph Mazzarella
SabineMcConnell
Fionn Murtagh
NinanSajeethPhilip
Rebecca Nugent
PaoloPadovani
FabioPasian
Misha Pesenson
Manuel Luis Sarro Baro
RiccardoSmareglia
Antonino Staiano
RoyWilliams
Chris Stubbs
NicholasWalton
Yongheng Zhao

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

IVOA Knowledge Discovery Interest Group

Charter

Knowledge discovery is the task of processing and analyzing data-sets with the aim of extracting new knowledge. This area spans widely across multiple disceplins, including visualization, remote data exploration, machine learning techniques, statistical methods, workflow orchestration, and polymorphic data access. To support the process of discovery, the KD-IG interacts closely with the other working/interest groups and feeds back requirements of the scientific community.

1. Participating in the definition of new data preservation and exchange formats with respect to support machine learning algorithms.

1. Presenting and collecting best practice examples of scientific data analytics in astronomy.
2. Defining requirements for implementing and adding machine learning capabilities to services.
3. Coordinating and unifying the access to data visualization functionalities.
4. Discussing the aspect of data provenance with respect to data used to derive/train models.
5. Introducing proper statistical scoring and evaluation methods as services.
6. Contributing to the discussion on scripting and orchestrating the scientific discovery workflow.
7. Supporting the development of dedicated knowledge discovery applications.

<-- 
We will develop and test scalable data mining algorithms and the accompanying new standards for VO interfaces and protocols, so that these algorithms can be discovered and used transparently within VO science workflows or in standalone data exploration applications. Therefore the activities of the KDD-IG will be:
 
 
 Support the definition of an ontology of the KDD tasks required by the astronomical community. This ontology will be used to define programming and documentation standards. 
 
 Make an inventory of existing methods relevant for astrophysical applications (more than 100 new KDD models and methods appear every month on specialized journals). 
 
 Identify reference data sets to be used for comparing, debugging and testing methods and tools. 
 
 Foster the implementation, using available VO standards and methods, of general purpose data exploration and data mining methods which will allow the general user to seamlessly exploit the complex data repositories offered by the VO. 
 
 Provide/receive feedbacks to/from the WGs in order to improve the usability of VO tools and standards. 
 
 Provide/receive from the community information to improve both the usability and the potentialities of Data Mining tools under the VO. 
 
 Define and pursue specific science cases which will be used to showcase the VO capabilities to the community. 
 
More important than anything else, we wish to use this IG as an arena where different groups can share experiences and plan future developments.
 
 
 -->

Motivations

During the Strasbourg InterOp Meeting it emerged the need for an Interest Group on Data Mining (KD-IG) as an indispensable step to bridge the Virtual Observatory Infrastructure with the expected VO science. In fact, "...Data mining, or KDD, is the semi-automatic discovery of patterns, associations, changes, anomalies, and statistically significant structures and events in data. In other words, traditional data analysis is assumption driven as a hypothesis is formed and validated against the data. Data mining, in contrast, is discovery driven as the patterns are automatically extracted from data....”

<-- 
As such, Data Mining (DM) can be considered as the “frontier” of VO enabled science since it represents the only way to capture and reveal the scientific knowledge (patterns, trends, correlations, etc.) hidden behind the complexity of Massive Data Sets.
 -->

Data Mining is a rapidly evolving set of methodologies which needs to be imported under the VO umbrella and not just another application. As such, DM cannot be just a tool or a suite of tools offered by a group of developers to a “passive community”. Data Mining involves a large number of researchers across many domains. The astronomical community, which has only recently entered the Massive Data Sets era, makes use of just a handful of methods and tools which very often are far from optimal. The synergy of different expertise present in the IVOA makes it the ideal arena for exploring new and more modern approaches.

KD-IG requires a strong and continuous interaction with the scientific community which, besides testing the proposed solutions, methods, and tools, will also provide feedback and inputs aiming at extending the scientific capabilities of the VO.

The KD-IG will interfaces to many other IVOA working and interest groups: Applications, Semantics, Time Domain, Data Models, Grid & Web Services, and Resource Registry. This cross- discipline nature is also a primary reason to create a specific IG. Data Mining, in fact, addresses sophisticated and extreme modes of usage which require a careful orchestration and fine tuning of standards, methods, and tools provided by the other IVOA WGs and IGs. Typical examples are the automatic extraction of bases of knowledge from VO archives using VO ontologies; the transparent access to large computational facilities regardless the computational paradigm; the automated switching from asynchronous to synchronous mode of data access; and the extreme usage of workflows and advanced visualization methods. Furthermore, effective KD requires the possibility for an inexperienced user to contribute, or at least seamlessly use under the VO infrastructure, his/her own KD routines and methods. This situation puts strong requirements on security issues and opens new problems for ticketing and scheduling. In other words, the KD-IG will provide feedback to the solutions implemented by the WG’s and, by posing new operational problems, will stimulate the development and adoption of new solutions and standards.

We also wish to stress that, in ultimate analysis, the goal of the KD-IG is to allow the VO to produce new scientific knowledge publishable in astronomical journals. On the one end its activities will contribute to demonstrate to the community the power and necessity of federated access to the vast VO universe of data and, on the other, KD-IG will illustrate the power and performance of data mining algorithms to facilitate and accelerate astronomical discovery within this data universe.

KD-IG Meetings

Other Interesting Meetings for KD-IG

<-- 
Next tasks
 
 
 
 
 Definition of a taxonomy of Data Mining models. This taxonomy will contribute to the Standard Vocabulary of the Semantics WG2. 
 
 Definition of the requirements which a Data Mining model needs to match in order to be imported under the VObs standards. 
 
 Inventory of existing Data Mining models of relevant astrophysical interest. 
 
 Definition of standard template data sets for Data Mining models test and debugging. 
 
 Definition of standard data sets to be used as bases of knowledge for debugging and test of supervised methods. 
 
 Definition of procedures to extract and validate robust bases of knowledge from the VObs data archives using the VObs ontology. 
 
 Study of the scalability of Data Mining models under different computing infrastructures (definition of best benchmarks). 
 
-->

Related Topics

In the past hot topics had been identified. These are following the priorities emerged during the first KD-IG meeting held at the IVOA.InterOpMay2010KDD in Victoria, and singled out by the Chair in his welcome message to the members.

Please follow the links and edit the specific pages:

1. Dictionary of Data Mining terms;
2. Census of Data Mining and Machine Learning tools and methods of astronomical interest;
3. Template datasets for algorithm benchmarking;
4. A user guide for Knowledge Discovery in Databases in Astronomy;
5. Knowledge Discovery in Databases and VO standards;
6. Specific fields of applications of KDD in astronomical research;

Members

Chair: Kai Lars Polsterer

Vice Chair: n/a

Task Force Members:

Sheelu Abraham
Lauretta Auvil
NickBall
ThomasBoch
Sen Bodhisattva
KirkBorne
MaxBrescia
Robert Brunner
Darren Davis
ReinaldoDeCarvalho
SebastienDerriere
GiuseppeLongo
CiroDonalek
Paul Eglitis
Pepi Fabbiano
PierreFernique
Peter Freeman
Fabian Gieseke
Alyssa Goodman
Alexander Gray
Paul Green
BobHanisch
Sheth Kartik
AjitKembhavi
OmarLaurino
Ann Lee
JeffLusted
AshishMahabal
BobMann
William March
Joseph Mazzarella
SabineMcConnell
Fionn Murtagh
NinanSajeethPhilip
Rebecca Nugent
PaoloPadovani
FabioPasian
Misha Pesenson
Manuel Luis Sarro Baro
RiccardoSmareglia
Antonino Staiano
RoyWilliams
Chris Stubbs
NicholasWalton
Yongheng Zhao

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

IVOA Knowledge Discovery Interest Group

Charter

Knowledge discovery is the task of processing and analyzing data-sets with the aim of extracting new knowledge. This area spans widely across multiple disceplins, including visualization, remote data exploration, machine learning techniques, statistical methods, workflow orchestration, and polymorphic data access. To support the process of discovery, the KD-IG interacts closely with the other working/interest groups and feeds back requirements of the scientific community.

The activities of the KD-IG include the following items with a strong emphasis on the first two points :

1. Participating in the definition of new data preservation and exchange formats with respect to support machine learning algorithms.
2. Introducing uncertainties and probabilistic description to VO standards and services
3. Presenting and collecting best practice examples of scientific data analytics in astronomy.
4. Defining requirements for implementing and adding machine learning capabilities to services.
5. Coordinating and unifying the access to data visualization functionalities.
6. Discussing the aspect of data provenance with respect to data used to derive/train models.
7. Introducing proper statistical scoring and evaluation methods as services.
8. Contributing to the discussion on scripting and orchestrating the scientific discovery workflow.
9. Supporting the development of dedicated knowledge discovery applications.

<-- 
We will develop and test scalable data mining algorithms and the accompanying new standards for VO interfaces and protocols, so that these algorithms can be discovered and used transparently within VO science workflows or in standalone data exploration applications. Therefore the activities of the KDD-IG will be:
 
 
 Support the definition of an ontology of the KDD tasks required by the astronomical community. This ontology will be used to define programming and documentation standards. 
 
 Make an inventory of existing methods relevant for astrophysical applications (more than 100 new KDD models and methods appear every month on specialized journals). 
 
 Identify reference data sets to be used for comparing, debugging and testing methods and tools. 
 
 Foster the implementation, using available VO standards and methods, of general purpose data exploration and data mining methods which will allow the general user to seamlessly exploit the complex data repositories offered by the VO. 
 
 Provide/receive feedbacks to/from the WGs in order to improve the usability of VO tools and standards. 
 
 Provide/receive from the community information to improve both the usability and the potentialities of Data Mining tools under the VO. 
 
 Define and pursue specific science cases which will be used to showcase the VO capabilities to the community. 
 
More important than anything else, we wish to use this IG as an arena where different groups can share experiences and plan future developments.
 
 
 -->

Motivations

During the Strasbourg InterOp Meeting it emerged the need for an Interest Group on Data Mining (KD-IG) as an indispensable step to bridge the Virtual Observatory Infrastructure with the expected VO science. In fact, "...Data mining, or KDD, is the semi-automatic discovery of patterns, associations, changes, anomalies, and statistically significant structures and events in data. In other words, traditional data analysis is assumption driven as a hypothesis is formed and validated against the data. Data mining, in contrast, is discovery driven as the patterns are automatically extracted from data....”

<-- 
As such, Data Mining (DM) can be considered as the “frontier” of VO enabled science since it represents the only way to capture and reveal the scientific knowledge (patterns, trends, correlations, etc.) hidden behind the complexity of Massive Data Sets.
 -->

Data Mining is a rapidly evolving set of methodologies which needs to be imported under the VO umbrella and not just another application. As such, DM cannot be just a tool or a suite of tools offered by a group of developers to a “passive community”. Data Mining involves a large number of researchers across many domains. The astronomical community, which has only recently entered the Massive Data Sets era, makes use of just a handful of methods and tools which very often are far from optimal. The synergy of different expertise present in the IVOA makes it the ideal arena for exploring new and more modern approaches.

KD-IG requires a strong and continuous interaction with the scientific community which, besides testing the proposed solutions, methods, and tools, will also provide feedback and inputs aiming at extending the scientific capabilities of the VO.

The KD-IG will interfaces to many other IVOA working and interest groups: Applications, Semantics, Time Domain, Data Models, Grid & Web Services, and Resource Registry. This cross- discipline nature is also a primary reason to create a specific IG. Data Mining, in fact, addresses sophisticated and extreme modes of usage which require a careful orchestration and fine tuning of standards, methods, and tools provided by the other IVOA WGs and IGs. Typical examples are the automatic extraction of bases of knowledge from VO archives using VO ontologies; the transparent access to large computational facilities regardless the computational paradigm; the automated switching from asynchronous to synchronous mode of data access; and the extreme usage of workflows and advanced visualization methods. Furthermore, effective KD requires the possibility for an inexperienced user to contribute, or at least seamlessly use under the VO infrastructure, his/her own KD routines and methods. This situation puts strong requirements on security issues and opens new problems for ticketing and scheduling. In other words, the KD-IG will provide feedback to the solutions implemented by the WG’s and, by posing new operational problems, will stimulate the development and adoption of new solutions and standards.

We also wish to stress that, in ultimate analysis, the goal of the KD-IG is to allow the VO to produce new scientific knowledge publishable in astronomical journals. On the one end its activities will contribute to demonstrate to the community the power and necessity of federated access to the vast VO universe of data and, on the other, KD-IG will illustrate the power and performance of data mining algorithms to facilitate and accelerate astronomical discovery within this data universe.

KD-IG Meetings

Other Interesting Meetings for KD-IG

<-- 
Next tasks
 
 
 
 
 Definition of a taxonomy of Data Mining models. This taxonomy will contribute to the Standard Vocabulary of the Semantics WG2. 
 
 Definition of the requirements which a Data Mining model needs to match in order to be imported under the VObs standards. 
 
 Inventory of existing Data Mining models of relevant astrophysical interest. 
 
 Definition of standard template data sets for Data Mining models test and debugging. 
 
 Definition of standard data sets to be used as bases of knowledge for debugging and test of supervised methods. 
 
 Definition of procedures to extract and validate robust bases of knowledge from the VObs data archives using the VObs ontology. 
 
 Study of the scalability of Data Mining models under different computing infrastructures (definition of best benchmarks). 
 
-->

Related Topics

In the past hot topics had been identified. These are following the priorities emerged during the first KD-IG meeting held at the IVOA.InterOpMay2010KDD in Victoria, and singled out by the Chair in his welcome message to the members.

Please follow the links and edit the specific pages:

1. Dictionary of Data Mining terms;
2. Census of Data Mining and Machine Learning tools and methods of astronomical interest;
3. Template datasets for algorithm benchmarking;
4. A user guide for Knowledge Discovery in Databases in Astronomy;
5. Knowledge Discovery in Databases and VO standards;
6. Specific fields of applications of KDD in astronomical research;

Members

Chair: Kai Lars Polsterer

Vice Chair: n/a

Task Force Members:

Sheelu Abraham
Lauretta Auvil
NickBall
ThomasBoch
Sen Bodhisattva
KirkBorne
MaxBrescia
Robert Brunner
Darren Davis
ReinaldoDeCarvalho
SebastienDerriere
GiuseppeLongo
CiroDonalek
Paul Eglitis
Pepi Fabbiano
PierreFernique
Peter Freeman
Fabian Gieseke
Alyssa Goodman
Alexander Gray
Paul Green
BobHanisch
Sheth Kartik
AjitKembhavi
OmarLaurino
Ann Lee
JeffLusted
AshishMahabal
BobMann
William March
Joseph Mazzarella
SabineMcConnell
Fionn Murtagh
NinanSajeethPhilip
Rebecca Nugent
PaoloPadovani
FabioPasian
Misha Pesenson
Manuel Luis Sarro Baro
RiccardoSmareglia
Antonino Staiano
RoyWilliams
Chris Stubbs
NicholasWalton
Yongheng Zhao

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

IVOA Knowledge Discovery Interest Group

Charter

Knowledge discovery is the task of processing and analyzing data-sets with the aim of extracting new knowledge. This area spans widely across multiple disceplins, including visualization, remote data exploration, machine learning techniques, statistical methods, workflow orchestration, and polymorphic data access. To support the process of discovery, the KD-IG interacts closely with the other working/interest groups and feeds back requirements of the scientific community.

1. Presenting and collecting best practice examples of scientific data analytics in astronomy.

1. Defining requirements for implementing and adding machine learning capabilities to services.
2. Coordinating and unifying the access to data visualization functionalities.
3. Discussing the aspect of data provenance with respect to data used to derive/train models.
4. Introducing proper statistical scoring and evaluation methods as services.
5. Contributing to the discussion on scripting and orchestrating the scientific discovery workflow.
6. Supporting the development of dedicated knowledge discovery applications.

<-- 
We will develop and test scalable data mining algorithms and the accompanying new standards for VO interfaces and protocols, so that these algorithms can be discovered and used transparently within VO science workflows or in standalone data exploration applications. Therefore the activities of the KDD-IG will be:
 
 
 Support the definition of an ontology of the KDD tasks required by the astronomical community. This ontology will be used to define programming and documentation standards. 
 
 Make an inventory of existing methods relevant for astrophysical applications (more than 100 new KDD models and methods appear every month on specialized journals). 
 
 Identify reference data sets to be used for comparing, debugging and testing methods and tools. 
 
 Foster the implementation, using available VO standards and methods, of general purpose data exploration and data mining methods which will allow the general user to seamlessly exploit the complex data repositories offered by the VO. 
 
 Provide/receive feedbacks to/from the WGs in order to improve the usability of VO tools and standards. 
 
 Provide/receive from the community information to improve both the usability and the potentialities of Data Mining tools under the VO. 
 
 Define and pursue specific science cases which will be used to showcase the VO capabilities to the community. 
 
More important than anything else, we wish to use this IG as an arena where different groups can share experiences and plan future developments.
 
 
 -->

Motivations

During the Strasbourg InterOp Meeting it emerged the need for an Interest Group on Data Mining (KD-IG) as an indispensable step to bridge the Virtual Observatory Infrastructure with the expected VO science. In fact, "...Data mining, or KDD, is the semi-automatic discovery of patterns, associations, changes, anomalies, and statistically significant structures and events in data. In other words, traditional data analysis is assumption driven as a hypothesis is formed and validated against the data. Data mining, in contrast, is discovery driven as the patterns are automatically extracted from data....”

<-- 
As such, Data Mining (DM) can be considered as the “frontier” of VO enabled science since it represents the only way to capture and reveal the scientific knowledge (patterns, trends, correlations, etc.) hidden behind the complexity of Massive Data Sets.
 -->

Data Mining is a rapidly evolving set of methodologies which needs to be imported under the VO umbrella and not just another application. As such, DM cannot be just a tool or a suite of tools offered by a group of developers to a “passive community”. Data Mining involves a large number of researchers across many domains. The astronomical community, which has only recently entered the Massive Data Sets era, makes use of just a handful of methods and tools which very often are far from optimal. The synergy of different expertise present in the IVOA makes it the ideal arena for exploring new and more modern approaches.

KD-IG requires a strong and continuous interaction with the scientific community which, besides testing the proposed solutions, methods, and tools, will also provide feedback and inputs aiming at extending the scientific capabilities of the VO.

The KD-IG will interfaces to many other IVOA working and interest groups: Applications, Semantics, Time Domain, Data Models, Grid & Web Services, and Resource Registry. This cross- discipline nature is also a primary reason to create a specific IG. Data Mining, in fact, addresses sophisticated and extreme modes of usage which require a careful orchestration and fine tuning of standards, methods, and tools provided by the other IVOA WGs and IGs. Typical examples are the automatic extraction of bases of knowledge from VO archives using VO ontologies; the transparent access to large computational facilities regardless the computational paradigm; the automated switching from asynchronous to synchronous mode of data access; and the extreme usage of workflows and advanced visualization methods. Furthermore, effective KD requires the possibility for an inexperienced user to contribute, or at least seamlessly use under the VO infrastructure, his/her own KD routines and methods. This situation puts strong requirements on security issues and opens new problems for ticketing and scheduling. In other words, the KD-IG will provide feedback to the solutions implemented by the WG’s and, by posing new operational problems, will stimulate the development and adoption of new solutions and standards.

We also wish to stress that, in ultimate analysis, the goal of the KD-IG is to allow the VO to produce new scientific knowledge publishable in astronomical journals. On the one end its activities will contribute to demonstrate to the community the power and necessity of federated access to the vast VO universe of data and, on the other, KD-IG will illustrate the power and performance of data mining algorithms to facilitate and accelerate astronomical discovery within this data universe.

KD-IG Meetings

Other Interesting Meetings for KD-IG

<-- 
Next tasks
 
 
 
 
 Definition of a taxonomy of Data Mining models. This taxonomy will contribute to the Standard Vocabulary of the Semantics WG2. 
 
 Definition of the requirements which a Data Mining model needs to match in order to be imported under the VObs standards. 
 
 Inventory of existing Data Mining models of relevant astrophysical interest. 
 
 Definition of standard template data sets for Data Mining models test and debugging. 
 
 Definition of standard data sets to be used as bases of knowledge for debugging and test of supervised methods. 
 
 Definition of procedures to extract and validate robust bases of knowledge from the VObs data archives using the VObs ontology. 
 
 Study of the scalability of Data Mining models under different computing infrastructures (definition of best benchmarks). 
 
-->

Related Topics

In the past hot topics had been identified. These are following the priorities emerged during the first KD-IG meeting held at the IVOA.InterOpMay2010KDD in Victoria, and singled out by the Chair in his welcome message to the members.

Please follow the links and edit the specific pages:

1. Dictionary of Data Mining terms;
2. Census of Data Mining and Machine Learning tools and methods of astronomical interest;
3. Template datasets for algorithm benchmarking;
4. A user guide for Knowledge Discovery in Databases in Astronomy;
5. Knowledge Discovery in Databases and VO standards;
6. Specific fields of applications of KDD in astronomical research;

Members

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

IVOA Knowledge Discovery Interest Group

Charter

Knowledge discovery is the task of processing and analyzing data-sets with the aim of extracting new knowledge. This area spans widely across multiple disceplins, including visualization, remote data exploration, machine learning techniques, statistical methods, workflow orchestration, and polymorphic data access. To support the process of discovery, the KD-IG interacts closely with the other working/interest groups and feeds back requirements of the scientific community.

The activities of the KD-IG include:

1. Presenting and collecting best practice examples of scientific data analytics in astronomy.
2. Participating in the definition of new data preservation and exchange formats with respect to support machine learning algorithms.
3. Defining requirements for implementing and adding machine learning capabilities to services.
4. Coordinating and unifying the access to data visualization functionalities.
5. Discussing the aspect of data provenance with respect to data used to derive/train models.
6. Introducing proper statistical scoring and evaluation methods as services.
7. Contributing to the discussion on scripting and orchestrating the scientific discovery workflow.
8. Supporting the development of dedicated knowledge discovery applications.

<-- 
We will develop and test scalable data mining algorithms and the accompanying new standards for VO interfaces and protocols, so that these algorithms can be discovered and used transparently within VO science workflows or in standalone data exploration applications. Therefore the activities of the KDD-IG will be:
 
 
 Support the definition of an ontology of the KDD tasks required by the astronomical community. This ontology will be used to define programming and documentation standards. 
 
 Make an inventory of existing methods relevant for astrophysical applications (more than 100 new KDD models and methods appear every month on specialized journals). 
 
 Identify reference data sets to be used for comparing, debugging and testing methods and tools. 
 
 Foster the implementation, using available VO standards and methods, of general purpose data exploration and data mining methods which will allow the general user to seamlessly exploit the complex data repositories offered by the VO. 
 
 Provide/receive feedbacks to/from the WGs in order to improve the usability of VO tools and standards. 
 
 Provide/receive from the community information to improve both the usability and the potentialities of Data Mining tools under the VO. 
 
 Define and pursue specific science cases which will be used to showcase the VO capabilities to the community. 
 
More important than anything else, we wish to use this IG as an arena where different groups can share experiences and plan future developments.
 
 
 -->

Motivations

During the Strasbourg InterOp Meeting it emerged the need for an Interest Group on Data Mining (KD-IG) as an indispensable step to bridge the Virtual Observatory Infrastructure with the expected VO science. In fact, "...Data mining, or KDD, is the semi-automatic discovery of patterns, associations, changes, anomalies, and statistically significant structures and events in data. In other words, traditional data analysis is assumption driven as a hypothesis is formed and validated against the data. Data mining, in contrast, is discovery driven as the patterns are automatically extracted from data....”

<-- 
As such, Data Mining (DM) can be considered as the “frontier” of VO enabled science since it represents the only way to capture and reveal the scientific knowledge (patterns, trends, correlations, etc.) hidden behind the complexity of Massive Data Sets.
 -->

Data Mining is a rapidly evolving set of methodologies which needs to be imported under the VO umbrella and not just another application. As such, DM cannot be just a tool or a suite of tools offered by a group of developers to a “passive community”. Data Mining involves a large number of researchers across many domains. The astronomical community, which has only recently entered the Massive Data Sets era, makes use of just a handful of methods and tools which very often are far from optimal. The synergy of different expertise present in the IVOA makes it the ideal arena for exploring new and more modern approaches.

KD-IG requires a strong and continuous interaction with the scientific community which, besides testing the proposed solutions, methods, and tools, will also provide feedback and inputs aiming at extending the scientific capabilities of the VO.

The KD-IG will interfaces to many other IVOA working and interest groups: Applications, Semantics, Time Domain, Data Models, Grid & Web Services, and Resource Registry. This cross- discipline nature is also a primary reason to create a specific IG. Data Mining, in fact, addresses sophisticated and extreme modes of usage which require a careful orchestration and fine tuning of standards, methods, and tools provided by the other IVOA WGs and IGs. Typical examples are the automatic extraction of bases of knowledge from VO archives using VO ontologies; the transparent access to large computational facilities regardless the computational paradigm; the automated switching from asynchronous to synchronous mode of data access; and the extreme usage of workflows and advanced visualization methods. Furthermore, effective KD requires the possibility for an inexperienced user to contribute, or at least seamlessly use under the VO infrastructure, his/her own KD routines and methods. This situation puts strong requirements on security issues and opens new problems for ticketing and scheduling. In other words, the KD-IG will provide feedback to the solutions implemented by the WG’s and, by posing new operational problems, will stimulate the development and adoption of new solutions and standards.

We also wish to stress that, in ultimate analysis, the goal of the KD-IG is to allow the VO to produce new scientific knowledge publishable in astronomical journals. On the one end its activities will contribute to demonstrate to the community the power and necessity of federated access to the vast VO universe of data and, on the other, KD-IG will illustrate the power and performance of data mining algorithms to facilitate and accelerate astronomical discovery within this data universe.

KD-IG Meetings

Other Interesting Meetings for KD-IG

<-- 
Next tasks
 
 
 
 
 Definition of a taxonomy of Data Mining models. This taxonomy will contribute to the Standard Vocabulary of the Semantics WG2. 
 
 Definition of the requirements which a Data Mining model needs to match in order to be imported under the VObs standards. 
 
 Inventory of existing Data Mining models of relevant astrophysical interest. 
 
 Definition of standard template data sets for Data Mining models test and debugging. 
 
 Definition of standard data sets to be used as bases of knowledge for debugging and test of supervised methods. 
 
 Definition of procedures to extract and validate robust bases of knowledge from the VObs data archives using the VObs ontology. 
 
 Study of the scalability of Data Mining models under different computing infrastructures (definition of best benchmarks). 
 
-->

Related Topics

In the past hot topics had been identified. These are following the priorities emerged during the first KD-IG meeting held at the IVOA.InterOpMay2010KDD in Victoria, and singled out by the Chair in his welcome message to the members.

Please follow the links and edit the specific pages:

1. Dictionary of Data Mining terms;
2. Census of Data Mining and Machine Learning tools and methods of astronomical interest;
3. Template datasets for algorithm benchmarking;
4. A user guide for Knowledge Discovery in Databases in Astronomy;
5. Knowledge Discovery in Databases and VO standards;
6. Specific fields of applications of KDD in astronomical research;

Members

Chair: Kai Lars Polsterer

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

IVOA Knowledge Discovery Interest Group

Charter

Knowledge discovery is the task of processing and analyzing data-sets with the aim of extracting new knowledge. This area spans widely across multiple disceplins, including visualization, remote data exploration, machine learning techniques, statistical methods, workflow orchestration, and polymorphic data access. To support the process of discovery, the KD-IG interacts closely with the other working/interest groups and feeds back requirements of the scientific community.

The activities of the KD-IG include:

1. Presenting and collecting best practice examples of scientific data analytics in astronomy.
2. Participating in the definition of new data preservation and exchange formats with respect to support machine learning algorithms.
3. Defining requirements for implementing and adding machine learning capabilities to services.
4. Coordinating and unifying the access to data visualization functionalities.
5. Discussing the aspect of data provenance with respect to data used to derive/train models.
6. Introducing proper statistical scoring and evaluation methods as services.
7. Contributing to the discussion on scripting and orchestrating the scientific discovery workflow.
8. Supporting the development of dedicated knowledge discovery applications.

<-- 
We will develop and test scalable data mining algorithms and the accompanying new standards for VO interfaces and protocols, so that these algorithms can be discovered and used transparently within VO science workflows or in standalone data exploration applications. Therefore the activities of the KDD-IG will be:
 
 
 Support the definition of an ontology of the KDD tasks required by the astronomical community. This ontology will be used to define programming and documentation standards. 
 
 Make an inventory of existing methods relevant for astrophysical applications (more than 100 new KDD models and methods appear every month on specialized journals). 
 
 Identify reference data sets to be used for comparing, debugging and testing methods and tools. 
 
 Foster the implementation, using available VO standards and methods, of general purpose data exploration and data mining methods which will allow the general user to seamlessly exploit the complex data repositories offered by the VO. 
 
 Provide/receive feedbacks to/from the WGs in order to improve the usability of VO tools and standards. 
 
 Provide/receive from the community information to improve both the usability and the potentialities of Data Mining tools under the VO. 
 
 Define and pursue specific science cases which will be used to showcase the VO capabilities to the community. 
 
More important than anything else, we wish to use this IG as an arena where different groups can share experiences and plan future developments.
 
 
 -->

Motivations

During the Strasbourg InterOp Meeting it emerged the need for an Interest Group on Data Mining (KD-IG) as an indispensable step to bridge the Virtual Observatory Infrastructure with the expected VO science. In fact, "...Data mining, or KDD, is the semi-automatic discovery of patterns, associations, changes, anomalies, and statistically significant structures and events in data. In other words, traditional data analysis is assumption driven as a hypothesis is formed and validated against the data. Data mining, in contrast, is discovery driven as the patterns are automatically extracted from data....”

<-- 
As such, Data Mining (DM) can be considered as the “frontier” of VO enabled science since it represents the only way to capture and reveal the scientific knowledge (patterns, trends, correlations, etc.) hidden behind the complexity of Massive Data Sets.
 -->

Data Mining is a rapidly evolving set of methodologies which needs to be imported under the VO umbrella and not just another application. As such, DM cannot be just a tool or a suite of tools offered by a group of developers to a “passive community”. Data Mining involves a large number of researchers across many domains. The astronomical community, which has only recently entered the Massive Data Sets era, makes use of just a handful of methods and tools which very often are far from optimal. The synergy of different expertise present in the IVOA makes it the ideal arena for exploring new and more modern approaches.

KD-IG requires a strong and continuous interaction with the scientific community which, besides testing the proposed solutions, methods, and tools, will also provide feedback and inputs aiming at extending the scientific capabilities of the VO.

The KD-IG will interfaces to many other IVOA working and interest groups: Applications, Semantics, Time Domain, Data Models, Grid & Web Services, and Resource Registry. This cross- discipline nature is also a primary reason to create a specific IG. Data Mining, in fact, addresses sophisticated and extreme modes of usage which require a careful orchestration and fine tuning of standards, methods, and tools provided by the other IVOA WGs and IGs. Typical examples are the automatic extraction of bases of knowledge from VO archives using VO ontologies; the transparent access to large computational facilities regardless the computational paradigm; the automated switching from asynchronous to synchronous mode of data access; and the extreme usage of workflows and advanced visualization methods. Furthermore, effective KD requires the possibility for an inexperienced user to contribute, or at least seamlessly use under the VO infrastructure, his/her own KD routines and methods. This situation puts strong requirements on security issues and opens new problems for ticketing and scheduling. In other words, the KD-IG will provide feedback to the solutions implemented by the WG’s and, by posing new operational problems, will stimulate the development and adoption of new solutions and standards.

We also wish to stress that, in ultimate analysis, the goal of the KD-IG is to allow the VO to produce new scientific knowledge publishable in astronomical journals. On the one end its activities will contribute to demonstrate to the community the power and necessity of federated access to the vast VO universe of data and, on the other, KD-IG will illustrate the power and performance of data mining algorithms to facilitate and accelerate astronomical discovery within this data universe.

KD-IG Meetings

Other Interesting Meetings for KD-IG

<-- 
Next tasks
 
 
 
 
 Definition of a taxonomy of Data Mining models. This taxonomy will contribute to the Standard Vocabulary of the Semantics WG2. 
 
 Definition of the requirements which a Data Mining model needs to match in order to be imported under the VObs standards. 
 
 Inventory of existing Data Mining models of relevant astrophysical interest. 
 
 Definition of standard template data sets for Data Mining models test and debugging. 
 
 Definition of standard data sets to be used as bases of knowledge for debugging and test of supervised methods. 
 
 Definition of procedures to extract and validate robust bases of knowledge from the VObs data archives using the VObs ontology. 
 
 Study of the scalability of Data Mining models under different computing infrastructures (definition of best benchmarks). 
 
-->

Related Topics

In the past hot topics had been identified. These are following the priorities emerged during the first KD-IG meeting held at the IVOA.InterOpMay2010KDD in Victoria, and singled out by the Chair in his welcome message to the members.

Please follow the links and edit the specific pages:

1. Dictionary of Data Mining terms;
2. Census of Data Mining and Machine Learning tools and methods of astronomical interest;
3. Template datasets for algorithm benchmarking;
4. A user guide for Knowledge Discovery in Databases in Astronomy;
5. Knowledge Discovery in Databases and VO standards;
6. Specific fields of applications of KDD in astronomical research;

Members

Chair: Kai Lars Polsterer

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

IVOA Knowledge Discovery Interest Group

Charter

Knowledge discovery is the task of processing and analyzing data-sets with the aim of extracting new knowledge. This area spans widely across multiple disceplins, including visualization, remote data exploration, machine learning techniques, statistical methods, workflow orchestration, and polymorphic data access. To support the process of discovery, the KD-IG interacts closely with the other working/interest groups and feeds back requirements of the scientific community.

The activities of the KD-IG include:

1. Presenting and collecting best practice examples of scientific data analytics in astronomy.
2. Participating in the definition of new data preservation and exchange formats with respect to support machine learning algorithms.
3. Defining requirements for implementing and adding machine learning capabilities to services.
4. Coordinating and unifying the access to data visualization functionalities.
5. Discussing the aspect of data provenance with respect to data used to derive/train models.
6. Introducing proper statistical scoring and evaluation methods as services.
7. Contributing to the discussion on scripting and orchestrating the scientific discovery workflow.
8. Supporting the development of dedicated knowledge discovery applications.

<-- 
We will develop and test scalable data mining algorithms and the accompanying new standards for VO interfaces and protocols, so that these algorithms can be discovered and used transparently within VO science workflows or in standalone data exploration applications. Therefore the activities of the KDD-IG will be:
 
 
 Support the definition of an ontology of the KDD tasks required by the astronomical community. This ontology will be used to define programming and documentation standards. 
 
 Make an inventory of existing methods relevant for astrophysical applications (more than 100 new KDD models and methods appear every month on specialized journals). 
 
 Identify reference data sets to be used for comparing, debugging and testing methods and tools. 
 
 Foster the implementation, using available VO standards and methods, of general purpose data exploration and data mining methods which will allow the general user to seamlessly exploit the complex data repositories offered by the VO. 
 
 Provide/receive feedbacks to/from the WGs in order to improve the usability of VO tools and standards. 
 
 Provide/receive from the community information to improve both the usability and the potentialities of Data Mining tools under the VO. 
 
 Define and pursue specific science cases which will be used to showcase the VO capabilities to the community. 
 
More important than anything else, we wish to use this IG as an arena where different groups can share experiences and plan future developments.
 
 
 -->

Motivations

During the Strasbourg InterOp Meeting it emerged the need for an Interest Group on Data Mining (KD-IG) as an indispensable step to bridge the Virtual Observatory Infrastructure with the expected VO science. In fact, "...Data mining, or KDD, is the semi-automatic discovery of patterns, associations, changes, anomalies, and statistically significant structures and events in data. In other words, traditional data analysis is assumption driven as a hypothesis is formed and validated against the data. Data mining, in contrast, is discovery driven as the patterns are automatically extracted from data....”

<-- 
As such, Data Mining (DM) can be considered as the “frontier” of VO enabled science since it represents the only way to capture and reveal the scientific knowledge (patterns, trends, correlations, etc.) hidden behind the complexity of Massive Data Sets.
 -->

Data Mining is a rapidly evolving set of methodologies which needs to be imported under the VO umbrella and not just another application. As such, DM cannot be just a tool or a suite of tools offered by a group of developers to a “passive community”. Data Mining involves a large number of researchers across many domains. The astronomical community, which has only recently entered the Massive Data Sets era, makes use of just a handful of methods and tools which very often are far from optimal. The synergy of different expertise present in the IVOA makes it the ideal arena for exploring new and more modern approaches.

KD-IG requires a strong and continuous interaction with the scientific community which, besides testing the proposed solutions, methods, and tools, will also provide feedback and inputs aiming at extending the scientific capabilities of the VO.

The KD-IG will interfaces to many other IVOA working and interest groups: Applications, Semantics, Time Domain, Data Models, Grid & Web Services, and Resource Registry. This cross- discipline nature is also a primary reason to create a specific IG. Data Mining, in fact, addresses sophisticated and extreme modes of usage which require a careful orchestration and fine tuning of standards, methods, and tools provided by the other IVOA WGs and IGs. Typical examples are the automatic extraction of bases of knowledge from VO archives using VO ontologies; the transparent access to large computational facilities regardless the computational paradigm; the automated switching from asynchronous to synchronous mode of data access; and the extreme usage of workflows and advanced visualization methods. Furthermore, effective KD requires the possibility for an inexperienced user to contribute, or at least seamlessly use under the VO infrastructure, his/her own KD routines and methods. This situation puts strong requirements on security issues and opens new problems for ticketing and scheduling. In other words, the KD-IG will provide feedback to the solutions implemented by the WG’s and, by posing new operational problems, will stimulate the development and adoption of new solutions and standards.

We also wish to stress that, in ultimate analysis, the goal of the KD-IG is to allow the VO to produce new scientific knowledge publishable in astronomical journals. On the one end its activities will contribute to demonstrate to the community the power and necessity of federated access to the vast VO universe of data and, on the other, KD-IG will illustrate the power and performance of data mining algorithms to facilitate and accelerate astronomical discovery within this data universe.

KD-IG Meetings

Other Interesting Meetings for KD-IG

<-- 
Next tasks
 
 
 
 
 Definition of a taxonomy of Data Mining models. This taxonomy will contribute to the Standard Vocabulary of the Semantics WG2. 
 
 Definition of the requirements which a Data Mining model needs to match in order to be imported under the VObs standards. 
 
 Inventory of existing Data Mining models of relevant astrophysical interest. 
 
 Definition of standard template data sets for Data Mining models test and debugging. 
 
 Definition of standard data sets to be used as bases of knowledge for debugging and test of supervised methods. 
 
 Definition of procedures to extract and validate robust bases of knowledge from the VObs data archives using the VObs ontology. 
 
 Study of the scalability of Data Mining models under different computing infrastructures (definition of best benchmarks). 
 
-->

Related Topics

In the past hot topics had been identified. These are following the priorities emerged during the first KD-IG meeting held at the IVOA.InterOpMay2010KDD in Victoria, and singled out by the Chair in his welcome message to the members.

Please follow the links and edit the specific pages:

1. Dictionary of Data Mining terms;
2. Census of Data Mining and Machine Learning tools and methods of astronomical interest;
3. Template datasets for algorithm benchmarking;
4. A user guide for Knowledge Discovery in Databases in Astronomy;
5. Knowledge Discovery in Databases and VO standards;
6. Specific fields of applications of KDD in astronomical research;

Members

Chair: Kai Lars Polsterer

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

IVOA Knowledge Discovery Interest Group

Charter

1. Presenting and collecting best practice examples of scientific data analytics in astronomy.
2. Participating in the definition of new data preservation and exchange formats with respect to support machine learning algorithms.
3. Defining requirements for implementing and adding machine learning capabilities to services.
4. Coordinating and unifying the access to data visualization functionalities.
5. Discussing the aspect of data provenance with respect to data used to derive/train models.
6. Introducing proper statistical scoring and evaluation methods as services.
7. Contributing to the discussion on scripting and orchestrating the scientific discovery workflow.
8. Supporting the development of dedicated knowledge discovery applications.

<-- 
We will develop and test scalable data mining algorithms and the accompanying new standards for VO interfaces and protocols, so that these algorithms can be discovered and used transparently within VO science workflows or in standalone data exploration applications. Therefore the activities of the KDD-IG will be:
 
 
 Support the definition of an ontology of the KDD tasks required by the astronomical community. This ontology will be used to define programming and documentation standards. 
 
 Make an inventory of existing methods relevant for astrophysical applications (more than 100 new KDD models and methods appear every month on specialized journals). 
 
 Identify reference data sets to be used for comparing, debugging and testing methods and tools. 
 
 Foster the implementation, using available VO standards and methods, of general purpose data exploration and data mining methods which will allow the general user to seamlessly exploit the complex data repositories offered by the VO. 
 
 Provide/receive feedbacks to/from the WGs in order to improve the usability of VO tools and standards. 
 
 Provide/receive from the community information to improve both the usability and the potentialities of Data Mining tools under the VO. 
 
 Define and pursue specific science cases which will be used to showcase the VO capabilities to the community. 
 
More important than anything else, we wish to use this IG as an arena where different groups can share experiences and plan future developments.
 
 
 -->

Motivations

KD-IG Meetings

Other Interesting Meetings for KD-IG

1. Dictionary of Data Mining terms;
2. Census of Data Mining and Machine Learning tools and methods of astronomical interest;
3. Template datasets for algorithm benchmarking;
4. A user guide for Knowledge Discovery in Databases in Astronomy;
5. Knowledge Discovery in Databases and VO standards;
6. Specific fields of applications of KDD in astronomical research;

Members

Chair: Kai Lars Polsterer

Sheelu Abraham
Lauretta Auvil
NickBall
ThomasBoch
Sen Bodhisattva
KirkBorne
MaxBrescia
Robert Brunner
RaffaeleDAbrusco
Darren Davis
ReinaldoDeCarvalho
DaveDeYoung
SebastienDerriere
GiuseppeLongo
CiroDonalek
Paul Eglitis
Pepi Fabbiano
PierreFernique
Peter Freeman
Fabian Gieseke
Alyssa Goodman
MatthewGraham
Alexander Gray
Paul Green
BobHanisch
Sheth Kartik
AjitKembhavi
OmarLaurino
Ann Lee
JeffLusted
AshishMahabal
BobMann
William March
Joseph Mazzarella
SabineMcConnell
Fionn Murtagh
NinanSajeethPhilip
Rebecca Nugent
PaoloPadovani
FabioPasian
Misha Pesenson
Kai Lars Polsterer
Manuel Luis Sarro Baro
PetrSkoda
RiccardoSmareglia
Antonino Staiano
RoyWilliams
Chris Stubbs
NicholasWalton
Yongheng Zhao

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

Charter

Motivations

During the Strasbourg InterOp Meeting it emerged the need for an Interest Group on Data Mining (KDD-IG) as an indispensable step to bridge the Virtual Observatory Infrastructure with the expected VO science. In fact, "...Data mining, or KDD, is the semi-automatic discovery of patterns, associations, changes, anomalies, and statistically significant structures and events in data. In other words, traditional data analysis is assumption driven as a hypothesis is formed and validated against the data. Data mining, in contrast, is discovery driven as the patterns are automatically extracted from data....”

As such, Data Mining (DM) can be considered as the “frontier” of VO enabled science since it represents the only way to capture and reveal the scientific knowledge (patterns, trends, correlations, etc.) hidden behind the complexity of Massive Data Sets.

Data Mining is a rapidly evolving set of methodologies which needs to be imported under the VO umbrella and not just another application. As such, DM cannot be just a tool or a suite of tools offered by a group of developers to a “passive community”. Data Mining involves a large number of researchers across many domains. The astronomical community, which has only recently entered the Massive Data Sets era, makes use of just a handful of methods and tools which very often are far from optimal. The synergy of different expertise present in the IVOA makes it the ideal arena for exploring new and more modern approaches.

KDD-IG requires a strong and continuous interaction with the scientific community which, besides testing the proposed solutions, methods, and tools, will also provide feedback and inputs aiming at extending the scientific capabilities of the VO.

The KDD-IG will interfaces to many other IVOA working and interest groups: Applications, Semantics, VOEvent, Data Models, Grid & Web Services, and Resource Registry. This cross- discipline nature is also a primary reason to create a specific IG. Data Mining, in fact, addresses sophisticated and extreme modes of usage which require a careful orchestration and fine tuning of standards, methods, and tools provided by the other IVOA WGs and IGs. Typical examples are the automatic extraction of bases of knowledge from VO archives using VO ontologies; the transparent access to large computational facilities regardless the computational paradigm; the automated switching from asynchronous to synchronous mode of data access; and the extreme usage of workflows and advanced visualization methods. Furthermore, effective KDD requires the possibility for an inexperienced user to contribute, or at least seamlessly use under the VO infrastructure, his/her own KDD routines and methods. This situation puts strong requirements on security issues and opens new problems for ticketing and scheduling. In other words, the KDD-IG will provide feedback to the solutions implemented by the WG’s and, by posing new operational problems, will stimulate the development and adoption of new solutions and standards.

We also wish to stress that, in ultimate analysis, the goal of the KDD-IG is to allow the VO to produce new scientific knowledge publishable in astronomical journals. On the one end its activities will contribute to demonstrate to the community the power and necessity of federated access to the vast VO universe of data and, on the other, KDD-IG will illustrate the power and performance of data mining algorithms to facilitate and accelerate astronomical discovery within this data universe.

Next tasks

1. Definition of a taxonomy of Data Mining models. This taxonomy will contribute to the Standard Vocabulary of the Semantics WG2.
2. Definition of the requirements which a Data Mining model needs to match in order to be imported under the VObs standards.
3. Inventory of existing Data Mining models of relevant astrophysical interest.
4. Definition of standard template data sets for Data Mining models test and debugging.
5. Definition of standard data sets to be used as bases of knowledge for debugging and test of supervised methods.
6. Definition of procedures to extract and validate robust bases of knowledge from the VObs data archives using the VObs ontology.
7. Study of the scalability of Data Mining models under different computing infrastructures (definition of best benchmarks).

Priorities

These are the hot topics that will be tackled in the early stages of the KDD-IG, following the priorities emerged during the first KDD-IG meeting held at the IVOA.InterOpMay2010KDD in Victoria, and singled out by the Chair in his welcome message to the members.

Please follow the links and edit the specific pages:

1. Dictionary of Data Mining terms;
2. Census of Data Mining and Machine Learning tools and methods of astronomical interest;
3. Template datasets for algorithm benchmarking;
4. A user guide for Knowledge Discovery in Databases in Astronomy;
5. Knowledge Discovery in Databases and VO standards;
6. Specific fields of applications of KDD in astronomical research;

Members

Chair: Kai Lars Polsterer

Sheelu Abraham
Lauretta Auvil
NickBall
ThomasBoch
Sen Bodhisattva
KirkBorne
MaxBrescia
Robert Brunner
RaffaeleDAbrusco
Darren Davis
ReinaldoDeCarvalho
DaveDeYoung
SebastienDerriere
GiuseppeLongo
CiroDonalek
Paul Eglitis
Pepi Fabbiano
PierreFernique
Peter Freeman
Fabian Gieseke
Alyssa Goodman
MatthewGraham
Alexander Gray
Paul Green
BobHanisch
Sheth Kartik
AjitKembhavi
OmarLaurino
Ann Lee
JeffLusted
AshishMahabal
BobMann
William March
Joseph Mazzarella
SabineMcConnell
Fionn Murtagh
NinanSajeethPhilip
Rebecca Nugent
PaoloPadovani
FabioPasian
Misha Pesenson
Kai Lars Polsterer
Manuel Luis Sarro Baro
PetrSkoda
RiccardoSmareglia
Antonino Staiano
RoyWilliams
Chris Stubbs
NicholasWalton
Yongheng Zhao

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

IVOA Knowledge Discovery in Databases

Charter

We will develop and test scalable data mining algorithms and the accompanying new standards for VO interfaces and protocols, so that these algorithms can be discovered and used transparently within VO science workflows or in standalone data exploration applications. Therefore the activities of the KDD-IG will be:

1. Support the definition of an ontology of the KDD tasks required by the astronomical community. This ontology will be used to define programming and documentation standards.
2. Make an inventory of existing methods relevant for astrophysical applications (more than 100 new KDD models and methods appear every month on specialized journals).
3. Identify reference data sets to be used for comparing, debugging and testing methods and tools.
4. Foster the implementation, using available VO standards and methods, of general purpose data exploration and data mining methods which will allow the general user to seamlessly exploit the complex data repositories offered by the VO.
5. Provide/receive feedbacks to/from the WGs in order to improve the usability of VO tools and standards.
6. Provide/receive from the community information to improve both the usability and the potentialities of Data Mining tools under the VO.
7. Define and pursue specific science cases which will be used to showcase the VO capabilities to the community.

Motivations

During the Strasbourg InterOp Meeting it emerged the need for an Interest Group on Data Mining (KDD-IG) as an indispensable step to bridge the Virtual Observatory Infrastructure with the expected VO science. In fact, "...Data mining, or KDD, is the semi-automatic discovery of patterns, associations, changes, anomalies, and statistically significant structures and events in data. In other words, traditional data analysis is assumption driven as a hypothesis is formed and validated against the data. Data mining, in contrast, is discovery driven as the patterns are automatically extracted from data....”

As such, Data Mining (DM) can be considered as the “frontier” of VO enabled science since it represents the only way to capture and reveal the scientific knowledge (patterns, trends, correlations, etc.) hidden behind the complexity of Massive Data Sets.

Data Mining is a rapidly evolving set of methodologies which needs to be imported under the VO umbrella and not just another application. As such, DM cannot be just a tool or a suite of tools offered by a group of developers to a “passive community”. Data Mining involves a large number of researchers across many domains. The astronomical community, which has only recently entered the Massive Data Sets era, makes use of just a handful of methods and tools which very often are far from optimal. The synergy of different expertise present in the IVOA makes it the ideal arena for exploring new and more modern approaches.

KDD-IG requires a strong and continuous interaction with the scientific community which, besides testing the proposed solutions, methods, and tools, will also provide feedback and inputs aiming at extending the scientific capabilities of the VO.

The KDD-IG will interfaces to many other IVOA working and interest groups: Applications, Semantics, VOEvent, Data Models, Grid & Web Services, and Resource Registry. This cross- discipline nature is also a primary reason to create a specific IG. Data Mining, in fact, addresses sophisticated and extreme modes of usage which require a careful orchestration and fine tuning of standards, methods, and tools provided by the other IVOA WGs and IGs. Typical examples are the automatic extraction of bases of knowledge from VO archives using VO ontologies; the transparent access to large computational facilities regardless the computational paradigm; the automated switching from asynchronous to synchronous mode of data access; and the extreme usage of workflows and advanced visualization methods. Furthermore, effective KDD requires the possibility for an inexperienced user to contribute, or at least seamlessly use under the VO infrastructure, his/her own KDD routines and methods. This situation puts strong requirements on security issues and opens new problems for ticketing and scheduling. In other words, the KDD-IG will provide feedback to the solutions implemented by the WG’s and, by posing new operational problems, will stimulate the development and adoption of new solutions and standards.

We also wish to stress that, in ultimate analysis, the goal of the KDD-IG is to allow the VO to produce new scientific knowledge publishable in astronomical journals. On the one end its activities will contribute to demonstrate to the community the power and necessity of federated access to the vast VO universe of data and, on the other, KDD-IG will illustrate the power and performance of data mining algorithms to facilitate and accelerate astronomical discovery within this data universe.

KDD-IG Meetings

Other Interesting Meetings for KDD-IG

Next tasks

1. Definition of a taxonomy of Data Mining models. This taxonomy will contribute to the Standard Vocabulary of the Semantics WG2.
2. Definition of the requirements which a Data Mining model needs to match in order to be imported under the VObs standards.
3. Inventory of existing Data Mining models of relevant astrophysical interest.
4. Definition of standard template data sets for Data Mining models test and debugging.
5. Definition of standard data sets to be used as bases of knowledge for debugging and test of supervised methods.
6. Definition of procedures to extract and validate robust bases of knowledge from the VObs data archives using the VObs ontology.
7. Study of the scalability of Data Mining models under different computing infrastructures (definition of best benchmarks).

Priorities

These are the hot topics that will be tackled in the early stages of the KDD-IG, following the priorities emerged during the first KDD-IG meeting held at the IVOA.InterOpMay2010KDD in Victoria, and singled out by the Chair in his welcome message to the members.

Please follow the links and edit the specific pages:

1. Dictionary of Data Mining terms;
2. Census of Data Mining and Machine Learning tools and methods of astronomical interest;
3. Template datasets for algorithm benchmarking;
4. A user guide for Knowledge Discovery in Databases in Astronomy;
5. Knowledge Discovery in Databases and VO standards;
6. Specific fields of applications of KDD in astronomical research;

Members

<-- 

 

 

 Set ALLOWTOPICRENAME = TWikiAdminGroup
 
 
-->

Charter

We will develop and test scalable data mining algorithms and the accompanying new standards for VO interfaces and protocols, so that these algorithms can be discovered and used transparently within VO science workflows or in standalone data exploration applications. Therefore the activities of the KDD-IG will be:

Motivations

KDD-IG Meetings

Other Interesting Meetings for KDD-IG

Next tasks

Priorities

These are the hot topics that will be tackled in the early stages of the KDD-IG, following the priorities emerged during the first KDD-IG meeting held at the IVOA.InterOpMay2010KDD in Victoria, and singled out by the Chair in his welcome message to the members.

Please follow the links and edit the specific pages:

Members