A sorting image sensor: an example of massively parallel intensity-to-time processing for low-latency computational sensors

The need for low-latency vision systems is growing: high speed visual servoing and vision-based human computer interface. In this paper we present a new intensity-to-time processing paradigm suitable for low-latency massively parallel global computation over fine-grained data such as images. As an example of a low-latency global computation, we have developed a VLSI sorting computational sensor-a sensor which sorts all pixels of an input image by their intensities, as the image is being sensed. The first sorting sensor prototype is a 21 by 26 array of cells. It detects an image focused thereon and computes the image of indices as well as the image's cumulative histogram, before the intensity data are readout. The image of indices never saturates and has uniform histogram. Under user's control, the chip can perform other operations including simple segmentation and labeling.