Difference: ChiVO (1 vs. 4)

Revision 42019-05-16 - MauricioSolar

 
META TOPICPARENT name="IvoaEvents"
Changed:
<
<
Currently ChiVO has indexed 28,236 FITS with a size of 4.5 [TB] corresponding to cycles 0,1,2 and 3 of ALMA, SIA and SCS services are available on these FITS. In addition we are looking to store the raw data of ALMA, the ASDM. Of these files we have already stored 2079 files with a total weight of 41 [TB], we will continue with this process of ingestion until we have stored the cycle 3 of ALMA. In order to make this data available, we are creating the resource to enable the currently available VO services, and we will try to offer SSAP over them in the future.
>
>
ChiVO, as the official Chilean node of the International Virtual Observatory Alliance (IVOA) has been recently connected (since January 2019) to the academic network of REUNA (the Chilean NREN) at 10 Gbps, so this will benefit the Quality of Service offered by ChiVO’s Datacenter.
 
Changed:
<
<
In the future plans, first we will look for to generate a series of mirrors of repositories of other VO, looking for to take advantage of the storage capacities of our datacenter and in the same line we will work in the ingestion of the data of MUSE, since we already have the MUSE reduction pipeline installed so that ESO evaluates our data reduction capabilities this data. In addition, we will take advantage of the availability of ASDM and Reduction Script to generate instances of re-processing by changing the parameters of the reduction script, to make this process under the IVOA standards we will use GWS. Currently we already have a table that allows us to map in ASDM -> Reduction Script (scriptForPI.py, scriptforImaging.py, etc..)-> FITS.
>
>
The datacenter has the objective of providing storage and processing capacities to the local and foreign astronomers, including mirroring astronomical data generated in Chile. Most of ChiVO services are currently hosted in this data center, and specifically part of the ALMA data is currently replicated here and accessed through VO services, like SIA and SCS.
Added:
>
>
The ALMA-VO Data Repositoryservice offers ALMA data access through IVOA standard web-services/VO-apps or through a web-page. Currently ChiVO has indexed 28,236 FITS with a size of 4.5 TB corresponding to cycles 0, 1, 2 and 3 of ALMA. SIA and SCS services are available on these FITS. In addition we have already stored 2079 files of the raw data of ALMA, the ASDM, with a total weight of 41 TB.
 
<--  
-->

Revision 32018-05-30 - MauricioSolar

 
META TOPICPARENT name="IvoaEvents"
Currently ChiVO has indexed 28,236 FITS with a size of 4.5 [TB] corresponding to cycles 0,1,2 and 3 of ALMA, SIA and SCS services are available on these FITS. In addition we are looking to store the raw data of ALMA, the ASDM. Of these files we have already stored 2079 files with a total weight of 41 [TB], we will continue with this process of ingestion until we have stored the cycle 3 of ALMA. In order to make this data available, we are creating the resource to enable the currently available VO services, and we will try to offer SSAP over them in the future.
Changed:
<
<
In the future plans, first we will look for to generate a series of mirrors of repositories of other VO, looking for to take advantage of the storage capacities of our datacenter and in the same line we will work in the ingestion of the data of MUSE, since we already have the MUSE reduction pipeline installed so that ESO evaluates our data reduction capabilities this data. In addition, we will take advantage of the availability of ASDM and Reduction Script to generate instances of re-processing by changing the parameters of the reduction script, to make this process under the IVOA standards we will use GWS. Currently we already have a table that allows us to map in ASDM -> Reduction Script (scriptForPI.py, scriptforImaging.py, etc..)-> FITS.​
>
>
In the future plans, first we will look for to generate a series of mirrors of repositories of other VO, looking for to take advantage of the storage capacities of our datacenter and in the same line we will work in the ingestion of the data of MUSE, since we already have the MUSE reduction pipeline installed so that ESO evaluates our data reduction capabilities this data. In addition, we will take advantage of the availability of ASDM and Reduction Script to generate instances of re-processing by changing the parameters of the reduction script, to make this process under the IVOA standards we will use GWS. Currently we already have a table that allows us to map in ASDM -> Reduction Script (scriptForPI.py, scriptforImaging.py, etc..)-> FITS.
 
<--  
-->

Revision 22017-10-26 - MauricioSolar

 
META TOPICPARENT name="IvoaEvents"
Changed:
<
<
1) ChiVO is currently migrating our IVOA-compatible services to a new data centre acquired recently with 1 PB of storage. ALMA data (only fits) should be again publicly available through SCS, SIA and TAP soon.
>
>
Currently ChiVO has indexed 28,236 FITS with a size of 4.5 [TB] corresponding to cycles 0,1,2 and 3 of ALMA, SIA and SCS services are available on these FITS. In addition we are looking to store the raw data of ALMA, the ASDM. Of these files we have already stored 2079 files with a total weight of 41 [TB], we will continue with this process of ingestion until we have stored the cycle 3 of ALMA. In order to make this data available, we are creating the resource to enable the currently available VO services, and we will try to offer SSAP over them in the future.
 
Changed:
<
<
2) In terms of development, we are working on integrating all the prototype applications developed as stand-alone examples into a coherent package strongly-based on astropy (ACALib). This will allow us to develop a REST API for processing data (ALMA data for instance) online. This API will include stacking procedures, clumping algorithms, synthetic spectroscopic cube generation and region of interest indexation. During this process we will try to comply as much as possible with GWS standards.
>
>
In the future plans, first we will look for to generate a series of mirrors of repositories of other VO, looking for to take advantage of the storage capacities of our datacenter and in the same line we will work in the ingestion of the data of MUSE, since we already have the MUSE reduction pipeline installed so that ESO evaluates our data reduction capabilities this data. In addition, we will take advantage of the availability of ASDM and Reduction Script to generate instances of re-processing by changing the parameters of the reduction script, to make this process under the IVOA standards we will use GWS. Currently we already have a table that allows us to map in ASDM -> Reduction Script (scriptForPI.py, scriptforImaging.py, etc..)-> FITS.​
Deleted:
<
<
3) A concrete development to report is the "ElasticSlap" service. The synthetic cube generator of point 2) uses the splatalogue database very often. We have reimplemented the Splatalogue database using ElasticSearch, and implemented a complete SLAP service over it. This allow us to quickly search for emission lines very quickly due to the ElasticSearch engine, and obviously the locality of the service. We need to address the problem of maintaining consistency with splatalogue before publishing the service through the registry.

4) ChiVO is now starting a second phase of development with funding for the next 2 years (2016-2018).

 
<--  
-->

Revision 12016-12-13 - MauricioSolar

 
META TOPICPARENT name="IvoaEvents"
1) ChiVO is currently migrating our IVOA-compatible services to a new data centre acquired recently with 1 PB of storage. ALMA data (only fits) should be again publicly available through SCS, SIA and TAP soon.

2) In terms of development, we are working on integrating all the prototype applications developed as stand-alone examples into a coherent package strongly-based on astropy (ACALib). This will allow us to develop a REST API for processing data (ALMA data for instance) online. This API will include stacking procedures, clumping algorithms, synthetic spectroscopic cube generation and region of interest indexation. During this process we will try to comply as much as possible with GWS standards.

3) A concrete development to report is the "ElasticSlap" service. The synthetic cube generator of point 2) uses the splatalogue database very often. We have reimplemented the Splatalogue database using ElasticSearch, and implemented a complete SLAP service over it. This allow us to quickly search for emission lines very quickly due to the ElasticSearch engine, and obviously the locality of the service. We need to address the problem of maintaining consistency with splatalogue before publishing the service through the registry.

4) ChiVO is now starting a second phase of development with funding for the next 2 years (2016-2018).

<--  
-->
 
This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback