Attentive Panoramic Sensor

SUMMARY – The current generation of visual sensors consist of a homogenous array of receptors. However, it is notable that biological systems have a different architecture with a high resolution foveal region and a lower resolution surround. Bottom-up or top-down attentive processes orient the fovea to areas of interest.

One possibility would be to develop a camera system which has spatially inhomogenous sampling. However, a similar effect can be acheived by having two cameras (top right). One has a wide-field or panoramic field of viewand is fixed in position. The second camera is has a narrower field of view and is attached to pan and tilt motors. Since the same number of pixels are spread across different fields of view, the mobile camera provides much more detail.

We fuse the data from the two views by finding the best homography relating the two images, and warping the foveal ouput on top of the paranoramic view (left). The position of the foveal sensor can be manipulated by clicking on a given position in the lower field image.

We are developing algorithms to make the attentive process automatic. These combine bottom up (background subtraction, motion differencing, face detection) measurements and top-down constraints such as a tendency to track objects, and to get bored of looking at the same object all the time. The result is that the sensor automatically attends to the people in the scene. The system records snapshots of the people in the room.

This system has potential applications in security and e-learning. Further details avalable from the main Elderlab website.