LSST Science Data Quality Analysis Subsystem Design

Software and Cyberinfrastructure for Astronomy V, 2018

Modern astronomical surveys such as the Large Synoptic Sky Survey (LSST) promise an unprecedented wealth of discoveries, delivered in the form of 10 million alerts of time-variable events per night. Astronomers are faced with the daunting challenge of identifying the most scientifically important events from this flood of data in order to conduct effective and timely follow-up observations. Several ongoing observing programs have proven databases to be extremely valuable in conducting efficient follow-up, particularly when combined with tools to select targets, submit observation requests directly to groundand space-based facilities (manual, remotely-operated and robotic), handle the resulting data, interface with analysis software and share information with collaborators. We draw on experience from a number of follow-up programs running at LCOGT, all of which have independently developed systems to provide these capabilities, including the Microlensing Key Project (RoboNet, PI: Tsapras, co-I Street), the Global Supernova Project (SNEx, PI: Howell) and the Near-Earth Object Project (NEOExchange, PI: Lister). We refer to these systems in general as Target and Observation Managers (TOMs). Future projects, facing a much greater and rapidly-growing list of potential targets, will find such tools to be indispensable, but the systems developed to date are highly specialized to the projects they serve and are not designed to scale to the LSST alert rate. We present a project to develop a general-purpose software toolkit that will enable astronomers to easily build TOM systems that they can customize to suit their needs, while a professionally-developed codebase will ensure that the systems are capable of scaling to future programs.

We summarize the Sloan Digital Sky Survey data acquisition and processing steps, and describe runQA, a pipeline designed for automated data quality assessment. In particular, we show how the position of the stellar locus in color-color diagrams can be used to estimate the accuracy of photometric zeropoint calibration to better than 0.01 mag in 0.03 deg 2 patches. Using this method, we estimate that typical photometric zeropoint calibration errors for SDSS imaging data are not larger than ∼ 0.01 mag in the g, r, and i bands, 0.02 mag in the z band, and 0.03 mag in the u band (root-mean-scatter for zeropoint offsets).

2006

We present a summary of the major contributions to the Special Session on Data Management held at the IAU General Assembly in Prague in 2006. While recent years have seen enormous improvements in access to astronomical data, and the Virtual Observatory aims to provide astronomers with seamless access to on-line resources, more attention needs to be paid to ensuring the quality and completeness of those resources. For example, data produced by telescopes are not always made available to the astronomical community, and new instruments are sometimes designed and built with insufficient planning for data management, while older but valuable legacy data often remain undigitised. Data and results published in journals do not always appear in the data centres, and astronomers in developing countries sometimes have inadequate access to on-line resources. To address these issues, an 'Astronomers Data Manifesto' has been formulated with the aim of initiating a discussion that will lead to the development of a 'code of best practice' in astronomical data management.

Publications of the Astronomical Society of the Pacific, 2007

We have developed an end-to-end photometric data processing pipeline to compare current photometric algorithms commonly used on ground-based imaging data. This testbed is exceedingly adaptable, and enables us to perform many research and development tasks, including image subtraction and co-addition, object detection and measurements, the production of photometric catalogs, and the creation and stocking of database tables with time-series information. This testing has been undertaken to evaluate existing photometry algorithms for consideration by a next-generation image processing pipeline for the Large Synoptic Survey Telescope (LSST). We outline the results of our tests for four packages: The Sloan Digital Sky Survey's (SDSS) Photo package, DAOPhot and allframe, DoPhot, and two versions of Source Extractor (SExtractor). The ability of these algorithms to perform pointsource photometry, astrometry, shape measurements, star-galaxy separation, and to measure objects at low signal-to-noise is quantified. We also perform a detailed crowded field comparison of DAOPhot and allframe, and profile the speed and memory requirements in detail for SExtractor. We find that both DAOPhot and Photo are able to perform aperture photometry to high enough precision to meet LSST's science requirements, and less adequately at PSF-fitting photometry. Photo performs the best at simultaneous point and extended-source shape and brightness measurements. SExtractor is the fastest algorithm, and recent upgrades in the software yield high-quality centroid and shape measurements with little bias towards faint magnitudes. Allframe yields the best photometric results in crowded fields.

arXiv: Earth and Planetary Astrophysics, 2019

The Large Synoptic Survey Telescope (LSST) is expected to increase known small solar system object populations by an order of magnitude or more over the next decade, enabling a broad array of transformative solar system science investigations to be performed. In this white paper, we discuss software tools and infrastructure that we anticipate will be needed to conduct these investigations and outline possible approaches for implementing them. Feedback from the community or contributions to future updates of this work are welcome. Our aim is for this white paper to encourage further consideration of the software development needs of the LSST solar system science community, and also to be a call to action for working to meet those needs in advance of the expected start of the survey in late 2022.

XVI Congresso Nazionale di Scienze Planetarie (National Conference on Planetary Sciences). Padova, Italy, 2020

Abstract. Planetary science space missions need high quality software ed efficient algorithms in order to extract innovative scientific results from flight data. Reliable and efficient software technologies are increasingly vital to improve and prolong the exploiting of the results of a mission, to allow the application of established algorithms and technologies also to future space missions and for the scientific analysis of archived data. Here after will be given an in-depth analysis study accompanied by implementation examples on ESA and ASI missions and some remarkable results fruit of decades of important experience reached by space agencies and research institutes in the field. Space applications software quality analysis is not different from other application contexts, among the hi-tech and hi-reliability fields. We describe here a Software Quality study in general, then we will focus on the quality of space mission software (s/w) with details on some notable cases. References. [1] European Space Agency. BepiColombo science mission homepage. https://sci.esa.int/web/bepicolombo. [2] Astropy Collaboration and Price-Whelan et al. The Astropy Project: Building an Open-science Project and Status of the v2.0 Core Package. Astrophysics Journal, 156(3):123, Sep 2018. [3] Rene Brun and Fons Rademakers. Root - an object oriented data analysis framework, aihenp’96 workshop, lausanne. volume 389, pages 81–86, Sep 1996. http://root.cern.ch/ . [4] Gabriele Cremonese et al. The Stereo Camera on the Bepicolombo Esa/jaxa Mission: a Novel Approach. In Advances in Geosciences, Volume 15: Planetary Science (PS), volume 15, pages 305–322, Mar 2009. [5] Sara De La Fuente, Angela Carasa, Iñaki Ortiz, Pedro Rodriguez, Mauro Casale, Johannes Benkhoff, and Joe Zender. Planning Bepicolombo MPO Science Operations to study Mercury Interior. In EGU Conference, page 15149, Apr 2017. [6] Astropy Collaboration : Robitaille et al. Astropy: A community Python package for astronomy. Astronomy and Astrophysics, 558:A33, Oct 2013. [7] M. Galassi et al. GNU Scientific Library Reference Manual (3rd Ed.), publisher = Network Theory Ltd, year = 2009, note = free numerical software library for C and C++ programmers callable from many other computer languages, isbn = 0954612078. [8] E. Flamini, F. Capaccioni, Cremonese, and SIMBIO-SYS Team. SIMBIO-SYS for BepiColombo: status and issues. Memorie della Societa Astronomica Italiana, 87:171, Jan 2016. [9] GDAL/OGR contributors. GDAL/OGR Geospatial Data Abstraction software Library. Open Source Geospatial Foundation, 2019. [10] Richard Soley Object Management Group. Omg whitepaper: How to deliver resilient, secure, efficient, and easily changed it systems in line with cisq recommendations. https://www.it-cisq.org/omg-cisq-wp , 2019. [11] K. H. Jeeja, K. Keerthi, A. Lakshmi, H. ShantalaS., Jothy Soman, P. S. Sura, and N. Valarmathi. An architecture of baseband data handling system for deep space mission and realization for mars orbiter mission. 2015 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pages 694–699, 2015. [12] Jose P. Miguel, David Mauricio, and Glen Rodriguez. A review of software quality models for the evaluation of software products. 2014. [13] S. Orsini, S. Livi, K. Torkar, S. Barabash, and the SERENA Team. ”SERENA: A suite of four instruments (ELENA, STROFIO, PICAM and MIPA) on board BepiColombo-MPO for particle detection in the hermean environment”. Planetary and Space Science, 58(1-2):166–181, Jan 2010. [14] Stefano Orsini et al. The BepiColombo/SERENA package: first signal from space. In EPSC-DPS Joint Meeting 2019, volume 2019, pages EPSC–DPS2019–1737, Sep 2019. [15] Fernando Perez-Lopez, Santa Martinez, Sara de la Fuente, Jayne Lefort, and Mauro Casale. BepiColombo MPO Data Handling and Archiving Operations Strategy. In DASIA 2013 - DAta Systems In Aerospace, volume 720 of ESA Special Publication, page 3, Aug 2013. [16] Jonathan McAuliffe, Sara de la Fuente and Mauro Casale. Science operations planning concept for bepicolombo mercury planetary orbiter. https://doi.org/10.2514/6 . 2016-2591, 2016. [17] QGIS Development Team. Qgis documentation and manuals. https://www.qgis.org/en/docs/index.html , urldate = 15/01/2019.

2004

The Large Synoptic Survey Telescope (LSST) will be a large, wide-field ground-based telescope designed to obtain sequential images of the entire visible sky every few nights. The optical design involves a novel 3-mirror system with an 8.4 m primary, which feeds three refractive correcting elements inside a camera, thereby providing a 7 square degree field of view sampled by a 2.3 Gpixel focal plane array. The total system throughput, AΩ = 270 m 2 deg 2 , is nearly two orders of magnitude larger than that of any existing facility. LSST will enable a wide variety of complementary scientific investigations, all utilizing a common database. These range from searches for small bodies in the solar system to the first systematic monitoring campaign for transient phenomena in the optical sky. Of particular interest to high energy physics, LSST images can be co-added to provide a weak lensing survey of the entire sky with unprecedented sensitivity. Measurement of the dark matter power density spectrum through weak lensing will provide tight constraints on models of dark energy, such as the equation of state parameter, w, and its derivative with respect to cosmic time. These constraints are complementary to those which will come from other approaches to studying dark energy (such as the apparent magnitude redshift relation of Type 1a supernovae), but are sensitive to different aspects of the cosmological model and involve quite different systematics. A collaboration has been formed to launch a design and development program for the LSST, leading to commencement of operations in 2011. This collaboration involves both NSF and DOE funded groups, working together under a common management structure. The DOE effort, which will be led by SLAC with significant components at BNL, LLNL and university-based HEP groups, will take overall responsibility for the LSST camera, the data acquisition system, and aspects of the pipeline software.

1998

Service mode observing simultaneously provides convenience, observing efficiency, cost-savings, and scheduling flexibility. To effectively optimize these advantages, the observer must exactly specify an observation with no real time interaction with the observatory staff. In this respect, ground-based service-mode observing and HST observing are similar. There are numerous details which, if unspecified, are either ambiguous or are left to chance, sometimes with undesirable results. Minimization of ambiguous/unspecified details is critical to the success of both HST and groundbased service observing. Smart observing proposal development tools which have built in flexibility are therefore essential for both the proposer and the observatory staff.

2013

Observatory data are the foundation for international scientific research. Valuable results can be achieved only if the data are precise and faultless. High quality instruments and a high level of ability and motivation of the observatory personnel are necessary, but a rigorous process of checking is as important for data quality control. Observatory data are useful for science only if the quality can be assured through peer review prior to publication. INTERMAGNET encourages participating observatories (IMOs) to check their definitive data before they are submitted. Furthermore, definitive data are carefully double-checked by volunteers and by the Definitive Data Subcommittee before they are published to the scientific community. This procedure is labour-intensive, but is essential to maintain a consistently high level of data quality. Different methods of data checks are described and their efficiency is discussed with consideration to the different instrument base of an observatory. Reason will be given as to why each data check is necessary and the tools available for effective data checks are described.

2004