The Astro-WISE approach to quality control for astronomical data

We present a novel approach to quality control during the processing of astronomical data. Quality control in the Astro-WISE Information System is integral to all aspects of data handing and provides transparent access to quality estimators for all stages of data reduction from the raw image to the final catalog. The implementation of quality control mechanisms relies on the core features in this Astro-WISE Environment (AWE): an object-oriented framework, full data lineage, and both forward and backward chaining. Quality control information can be accessed via the command-line awe-prompt and the web-based Quality-WISE service. The quality control system is described and qualified using archive data from the 8-CCD Wide Field Imager (WFI) instrument (http://www.eso.org/lasilla/instruments/wfi/) on the 2.2-m MPG/ESO telescope at La Silla and (pre-)survey data from the 32-CCD OmegaCAM instrument (http://www.astro-wise.org/~omegacam/) on the VST telescope at Paranal.Comment: Accepted for publication in topical issue of Experimental Astronomy on Astro-WISE information syste

Mining archival data from wide-field astronomical surveys in search of near-Earth objects

Context. Increasing our knowledge of the orbits and compositions of near-earth objects (NEOs) is important for a better understanding of the evolution of the Solar System and life. The detection of serendipitous NEO appearances among the millions of archived exposures from large astronomical imaging surveys can provide a contribution which is complementary to NEO surveys. Aims. Using the ASTROWISE information system, this work aims to assess the detectability rate, the achieved recovery rate and the quality of astrometry when data mining the European Southern Observatory (ESO) archive for the OmegaCAM wide-field imager at the VLT Survey Telescope (VST). Methods. We developed an automatic pipeline that searches for NEO appearances inside the ASTROWISE environment. Throughout the recovery process the pipeline uses several public web tools (SSOIS, NEODyS, JPL Horizons) to identify possible images that overlap with the positions of NEOs, and acquires information on the NEOs’ predicted position and other properties (e.g. magnitude, rate, and direction of motion) at the time of observations. Considering these properties, the pipeline narrows down the search to potentially detectable NEOs, searches for streak-like objects across the images, and finds a matching streak for the NEOs. Results. We recovered 196 appearances of NEOs from a set of 968 appearances predicted to be recoverable. It includes appearances for three NEOs that were on the impact risk list at that point. These appearances occurred well before their discovery. The subsequent risk assessment using the extracted astrometry removes these NEOs from the risk list. More generally, we estimate a detectability rate of ~0.05 per NEO at a signal-to-noise ratio higher than 3 for NEOs in the OmegaCAM archive. Our automatic recovery rates are 40% and 20% for NEOs on the risk list and the full list, respectively. The achieved astrometric and photometric accuracy is on average 0.12″ and 0.1 mag. Conclusions. These results show the high potential of the archival imaging data of the ground-based wide-field surveys as useful instruments for the search, (p)recovery, and characterization of NEOs. Highly automated approaches, as possible using ASTROWISE, make this undertaking feasible.This work was executed as part of ESA contract no. 4000134667/21/D/MRP (CARMEN) with their Planetary Defence Office. The Big Data Layer of the Target Field Lab project “Mining Big Data” was used. The Target Field Lab is supported by the Northern Netherlands Alliance (SNN) and is financially supported by the European Regional Development Fund. The data science software system ASTROWISE runs on powerful databases and computing clusters at the Donald Smits Center of the University of Groningen and is supported, among other parties, by NOVA (the Dutch Research School for Astronomy). TSR acknowledges funding from the NEO-MAPP project (H2020-EU-2-1-6/870377). This work was (partially) supported by the Spanish MICIN/AEI/10.13039/501100011033 and by “ERDF A way of making Europe” by the “European Union” through grant PID2021-122842OB-C21, and the Institute of Cosmos Sciences University of Barcelona (ICCUB, Unidad de Excelencia “María de Maeztu”) through grant CEX2019-000918-M

The third data release of the Kilo-Degree Survey and associated data products

The Kilo-Degree Survey (KiDS) is an ongoing optical wide-field imaging survey with the OmegaCAM camera at the VLT Survey Telescope. It aims to image 1500 square degrees in four filters (ugri). The core science driver is mapping the large-scale matter distribution in the Universe, using weak lensing shear and photometric redshift measurements. Further science cases include galaxy evolution, Milky Way structure, detection of high-redshift clusters, and finding rare sources such as strong lenses and quasars. Here we present the third public data release (DR3) and several associated data products, adding further area, homogenized photometric calibration, photometric redshifts and weak lensing shear measurements to the first two releases. A dedicated pipeline embedded in the Astro-WISE information system is used for the production of the main release. Modifications with respect to earlier releases are described in detail. Photometric redshifts have been derived using both Bayesian template fitting, and machine-learning techniques. For the weak lensing measurements, optimized procedures based on the THELI data reduction and lensfit shear measurement packages are used. In DR3 stacked ugri images, weight maps, masks, and source lists for 292 new survey tiles (~300 sq.deg) are made available. The multi-band catalogue, including homogenized photometry and photometric redshifts, covers the combined DR1, DR2 and DR3 footprint of 440 survey tiles (447 sq.deg). Limiting magnitudes are typically 24.3, 25.1, 24.9, 23.8 (5 sigma in a 2 arcsec aperture) in ugri, respectively, and the typical r-band PSF size is less than 0.7 arcsec. The photometric homogenization scheme ensures accurate colors and an absolute calibration stable to ~2% for gri and ~3% in u. Separately released are a weak lensing shear catalogue and photometric redshifts based on two different machine-learning techniques.Comment: small modifications; 27 pages, 12 figures, accepted for publication in Astronomy & Astrophysic

The Astro-WISE optical image pipeline Development and implementation

<p>We have designed and implemented a novel way to process wide-field astronomical data within a distributed environment of hardware resources and humanpower. The system is characterized by integration of archiving, calibration, and post-calibration analysis of data from raw, through intermediate, to final data products. It is a true integration thanks to complete linking of data lineage from the final catalogs back to the raw data. This paper describes the pipeline processing of optical wide-field astronomical data from the WFI (http://www.eso.org/lasilla/instruments/wfi/) and OmegaCAM (http://www.astro-wise.org/similar to omegacam/) instruments using the Astro-WISE information system (the Astro-WISE Environment or simply AWE). This information system is an environment of hardware resources and humanpower distributed over Europe. AWE is characterized by integration of archiving, data calibration, post-calibration analysis, and archiving of raw, intermediate, and final data products. The true integration enables a complete data processing cycle from the raw data up to the publication of science-ready catalogs. The advantages of this system for very large datasets are in the areas of: survey operations management, quality control, calibration analyses, and massive processing.</p>

Properties of near-Earth objects

VizieR online Data Catalogue associated with article published in journal Astronomy and Astrophysics with title ' Mining archival data from wide-field astronomical surveys in search of near-Earth objects.' (bibcode: 2023A&amp;A...673A..93S

Searching for galaxy clusters in the VST-KIDS survey

We present the methods and first results of the search for galaxy clusters in the Kilo Degree Survey (KiDS). The adopted algorithm and the criterium for selecting the member galaxies are illustrated. Here we report the preliminary results obtained over a small area (7 deg2), and the comparison of our cluster candidates with those found in the c and SZ Planck catalogues; the analysis to a larger area (148 deg2) is currently in progress. By the KiDS cluster search, we expect to increase the completeness of the clusters catalogue to z = 0.6–0.7 compared to RedMapper.</p

Gravitational lensing analysis of the Kilo-Degree Survey

The Kilo-Degree Survey (KiDS) is a multi-band imaging survey designed for cosmological studies from weak lensing and photometric redshifts. It uses the European Southern Observatory VLT Survey Telescope with its wide-field camera OmegaCAM. KiDS images are taken in four filters similar to the Sloan Digital Sky Survey ugri bands. The best seeing time is reserved for deep r-band observations. The median 5σ limiting AB magnitude is 24.9 and the median seeing is below 0.7 arcsec. Initial KiDS observations have concentrated on the Galaxy and Mass Assembly (GAMA) regions near the celestial equator, where extensive, highly complete redshift catalogues are available. A total of 109 survey tiles, 1 square degree each, form the basis of the first set of lensing analyses of halo properties of GAMA galaxies. Nine galaxies per square arcminute enter the lensing analysis, for an effective inverse shear variance of 69 arcmin-2. Accounting for the shape measurement weight, the median redshift of the sources is 0.53. KiDS data processing follows two parallel tracks, one optimized for weak lensing measurement and one for accurate matched-aperture photometry (for photometric redshifts). This technical paper describes the lensing and photometric redshift measurements (including a detailed description of the Gaussian aperture and photometry pipeline), summarizes the data quality and presents extensive tests for systematic errors that might affect the lensing analyses. We also provide first demonstrations of the suitability of the data for cosmological measurements, and describe our blinding procedure for preventing confirmation bias in the scientific analyses. The KiDS catalogues presented in this paper are released to the community through http://kids.strw.leidenuniv.nl