Face recognition using photometric stereo

This work is being researched in partnership with the Communications and Signal processing group at Imperial College London. We are also in collaboration with the Home Office Scientific Development Branch, General Dynamics UK and Identity Solutions.

The main aims of this EPSRC funded project, which commenced in April 2007, are twofold:

1. Develop a photometric stereo-based technique for accurately recovering the three-dimensional structure of moving faces.
2. Utilise the three-dimensional data to advance existing face recognition technology.

The device we have constructed is shown below. We are currently using four light sources and a camera operating at 200fps. This means that the total image capture time is less than 20ms.

• Download a video demonstration of the device (16MB Windows Media Video)

The figure below shows an example of a raw image set recovered using the device.

Application of Lambertian photometric stereo then gives the following field of surface normals:

The figures below show the result of integrating the surface normals to recover a depth map. The second image shows the result of warping one of the raw images onto the surface.

In addition to designing the hardware for this work we have published four conference papers on the work (see Gary Atkinson's webpage). These papers cover a range of new techniques relating to image alignment, feature detection, the effects of makeup and facial hair on the accuracy of the reconstruction, optimising the reconstructions with the addition of a profile-view camera and (in collaboration with the University of Bath), designed a recognition algorithm based on the nose ridge shape. We are currently preparing a publication on using infrared illumination sources and a detailed study into the sensitivity of the methods.

For some applications, it may be useful to compare 3D (or 2.5D) data to 2D images. In these cases it is necessary to use the 2.5D data to render images that have matching illumination conditions to the 2D images. A video illustrating our ability to re-render images in this way can be viewed in either, a greyscale AVI video, or a colour AVI video.

One area of particular interest is the construction of a database of raw face images. This unique database will be very different from existing databases in several respects:

• Each record will consist of four images of a face, corresponding to each of the light sources of our photometric stereo rig.
• The various volunteers will be imaged on many occasions over a period of months, allowing extensive testing of our new methods as people change over time. These changes may be due to expression/mood, pose, headgear, hair (including facial hair), tanning, injury, etc.
• The data is being collected from a real working environment, rather than in controlled laboratory settings. This is in line with the laboratory’s aim to develop machine vision techniques for real applications.