Comparison of MTF measurements using edge method: towards reference data set

A sensor's spatial resolution has traditionally been a difficult concept to define, but all would agree that it is inextricably linked to the Ground Sampling Distance (GSD) and Instantaneous Field of View (IFOV) of an imaging sensor system. As a measure of the geospatial quality of imagery, the Modulation Transfer Function (MTF) of the system is often used along with the signal-to-noise ratio (SNR). However, their calculation is not fully standardized. Further, consistent measurements and comparisons are often hard to obtain. Therefore, in the Infrared and Visible Optical Sensors (IVOS) subgroup of the Working Group on Calibration Validation (WGCV) of the Committee for Earth Observation Satellites (CEOS), a team from various countries and professional entities who are involved in MTF measurement was established to address the issue of on-orbit MTF measurements and comparisons. As a first step, a blind comparison of MTF measurements based on the slanted edge approach has been undertaken. A set of both artificial and actual satellite edge images was developed and a first comparison of processing results was generated. In all, seven organizations contributed to the experiment and several significant results were generated in 2016. No single participant produced the best results for all test images as measured by either the closest to the mean result, or closest to the truth for the synthetic test images. In addition, close estimates of the MTF value at Nyquist did not ensure the accuracy of other MTF values at other spatial frequencies. Some algorithm results showed that the accuracy of their estimates depended upon the type of MTF curve that was being analyzed. After the initial analysis, participants were allowed to modify their methodology and reprocess the test images since, in several cases, the results contained errors. Results from the second iteration, in 2017, verified that the anomalies in the experiment's first iteration were due to errors in either coding or methodology, or both. One organization implemented a third trial to fix software errors. This emphasizes the importance of fully understanding both methodology and implementation, in order to ensure accurate and repeatable results. To extend this comparison study, a reference data set, which is composed of edge images and corresponding MTF curves, will be built. A broader audience will be able to access the edge images through the CEOS CalVal Portal (http://calvalportal.ceos.org/). This paper, which is associated with the reference data set, can serve as a new tool to either implement or check, or both, the MTF measurement that relies on the slanted edge method.