Survey among Spectral Data Providers and Consumers

At the IVOA meeting in Cambridge in May, one of the highest priority projects identified for the DAL working group was to define a Simple Spectral Access (SSA) protocol for accessing 1D spectra and SEDs.

To help plan this effort a little survey was conducted among potential data providers and consumers. The results are summarized below. Of course, the summary is only as complete as the response we've received. Thanks a lot to all who took the time to answer to our questions!

More feedback is always welcome. Regards,
DougTody, MarkusDolensky



1a. Data Provider

1b. Data Consumer


About Data Providers

2. Spectral Data Collections

- Sloan Digital Sky Survey (SDSS), Data Release 1
186.000 objects, 3800-9200 Å, 1300 square degrees, resolution 1800

- NCSA Astronomy Digital Image Library (NCSA BIMA Data Archive)

- Variety of spectral data sources from X-ray and gamma-ray missions, including both relatively high and low resolution spectral data (HEASARC)

- Indo-U.S. Library of Coude Feed Stellar Spectra (NOAO & CFLIB)
1300 spectra, 3465-9469 Å @ 0.44A/pix

- at NOAO:

  • High Resolution Atlas of Arcturus Spectra - 0.92-5.36 microns
  • High Resolution Atlas of Arcturus Spectra - 3727-9300 Å
  • Library of High Resolution Infrared Stellar Spectra in K-Band
  • Library of Medium Resolution Infrared Stellar Spectra
  • Coude Feed Spectral Library
  • Library of Stellar Spectra
  • Optical Spectrophotometric Atlas of SN 1987A
  • Atlases of ThAR, FeAr, and HeNeAr comparison spectra

- at ESO/ST-ECF as of June 26, 03

Observ./Instr. No. of Spectra Observ./Instr. No. of Spectra
HST/ACS ? ESO/EMMI 30332
HST/FOS 16058 ESO/FORS1+2 14721
HST/GHRS 12340 ESO/ISAAC 66274
HST/NICMOS ? ESO/UVES 50024
HST/STIS 35766 ESO/VIMOS 6754
ESO/CES 9272 ESO/WFI ?
ESO/EFOSC2 11054    
*ESO/GIRAFFE and ESO/HARPS are going on-line withing the coming months.

- XMM-Newton 1D Scpectral Data
XMM/RGS, 20000 spectra (expected), 0.35-2.5 keV, resolution 200-800
XMM/EPIC, 1.5 mio. spectra (expected), 0.35-15 keV, resolution 20-50

- ISO 1D Spectral Data
10000 files, 2.4-197 micron, resolution 40-30000

- IUE Newly-Extracted Spectra (INES)
110000 spectra, 9500 objects, 1150-1980 Å & 1850-3350 Å resolution 1.676 & 2.669 Å/pix

- at NAOJ/SMOKA as of Aug. 25, 03:
(National Astronomical Observatory of Japan / Subaru Mitaka Okayama Kiso Archive)

Observ./Instr. No. of Spectra Size Description
Subaru/HDS 2158 16MB extremely high-dispersion optical echelle spectroscopy
Subaru/IRCS 7546 4MB low-resolution and echelle spectroscopy from 1-5 microns
Subaru/FOCAS 3121 16MB optical longslit and multi-slit spectroscopy over a 6 arcmin field of view
Subaru/OHS 4664 2MB low-resolution spectroscopy in the near-infrared
Subaru/COMICS 9902 1.7MB spectroscopy from 8-26 microns
OAO/HIDES 1390 16MB extremely high-dispersion optical echelle spectroscopy
OAO/SNG 8441 0.3MB low-resolution optical spectroscopy

OAO : 188cm telescope at Okayama Astronomical Observatory
Subaru : 8.2m optical-infrared telescope at the summit of Mauna Kea

- Hyperleda
10000 reference spectra of stars and galaxies, resolution 300 - 1000000, 1300 Å - 2.5 microns total volume 5 GB, collected from literature and reformatted

- Elodie
25000 extracted echelle spectra, resolution 42000, 4000 - 6800 Å on the fly processing options available: wavelength resampling, flux calibration etc. total volume 50 - 100 GB

3. Characteristics of Data

  • number: ~100 K - 1 Mio. spectra/data provider; often only several 100s in very specific catalogs
  • size: few KBs - few MBs/spectrum
  • wavelength/energy ranges: gamma, X-ray, UV, optical, IR, radio
  • resolution: 20 - 1.000.000
  • linear and non-linear/irregularly sampling (for instance in energy domain)
  • often with WCS
  • stellar parameters: e.g.: Teff, log(g), [Fe/H]
  • noise
  • masks (coded aperture mask data, INTEGRAL)
  • observation dependent transfer matrix to convert counts to phys. units
  • spectral features (SLOAN): emission, absorption lines and parameters, emission redshift params. cross-correlation parameters
  • time resolved spectral information (XTE, BATSE, Swift, HETE)

Columns/data items:

  • wavelength (better fravergy: frequency, wavelength, energy)
  • flux (magnitude, flux, flux density, counts)
  • flux error
  • quality flag (quality of what exactly?)
  • variance arrays
  • photon events
  • antenna temperature vs. frequency (e.g. SWAS spectra)

4. Current Storage Format

  • ASCII tables (catalogs, often highly processed data)
  • Database tables (e.g. Sloan)
  • FITS binary tables
  • FITS images (often used for raw data (CCD images) or stacks thereof; radio spectral image cubes)

5. Is the data available online?

Yes. Data are generally on-line, but proprietary periods and restricted access depending on nationality may apply.


About Data Consumers

6. Name of application, package, demo, etc.

  • aXe
  • MIDAS-MOS
  • NOAO/IRAF-Specplot
  • Pleinpot
  • Specview

Tools for SEDs:

  • NED (on-line)
  • AVO Prototype SED utility

7. Summarize capabilities of software

  • reading various instrument dependent FITS and ASCII formats
  • overplotting of multiple arrays
  • various display options (labels, zoom, colors, log/linear scaling, ...)
  • physical unit conversion
  • model fitting capability
  • classification against standards
  • rebinning
  • quality filters

see, for instance http://www.stsci.edu/resources/software_hardware/specview/what_is_specview

8. Desired characteristics of input data

  • As a minimum, data should consist of (x,y) pairs of wavelength-flux values expressed in wavelength/frequency/energy units, and spectral flux density units respectively. The units information should be present as well, in header keywords or table column descriptors or any other suitable form.
  • Additional information that could be included with the above, (if available) is a third value associated with each (x,y) pair representing the measurement error in flux density. Also, it won't hurt if obvious pieces of information such as the object name, are provided as well.
  • Uncalibrated data (such as in counts/s units) can also be ingested but not much else can be done with it besides plain plotting.
  • FITS WCS; useful for well sampled rasters

9. Desired input data format or formats (e.g. graphics, FITS, XML)

FITS (images, tables), XML, and plain ASCII are all supported by Specview at this point. It is fairly easy to write ingestor modules for almost any conceivable spectral format that uses one of those media.


10. General Comments

An interesting service would be to generate 1-D spectra from cubes on the fly. In addition to the usual search parameters, users would have to provide a spatial resolution. This service would be analogous to an image cutout service in that the output is generated to user specs on-the-fly from the underlying data. (Ray Plante)

Even at this level of detail it may be inappropriate to discuss the HEASARC monolithically. The characteristics of individual missions/instruments varies enormously. (Tom McGlynn)

1D and 2D currently have more applications than 3D. FITS binary tables format and support under development. Will be evolved to support VO spectral standards. (Francisco Valdes)

[...] the different storage formats and associated data tables would provide for some interesting use cases. Something like a cutout service might be appropriate for some of the data, the challenging thing for us however to put all these into an SSA is simply the variety of local data access methods required. (Mike Fitzpatrick)




Topic attachments
I Attachment History Action Size Date Who Comment
PDFpdf ssadass.pdf r1 manage 1944.6 K 2003-10-09 - 15:10 MarkusDolensky A0 Poster of SSA Survey
Edit | Attach | Watch | Print version | History: r12 < r11 < r10 < r9 < r8 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r12 - 2003-10-27 - MarkusDolensky
 
This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback