Using the Eyes of Killer Shrimp to Design A Super Camera

A camera that mimics the eyes of a mantis shrimp may one day help autonomous cars see hazards through shadows and fog. The camera is 1 million times more sensitive to light levels than conventional cameras and should only cost a few dollars to manufacture.

The camera is the work of Viktor Gruev's Biosensors Lab at University of Illinois at Urbana-Champaign, where Gruev is an associate professor of electrical and computer engineering. The lab studies the vision of animals as diverse as shrimp and butterflies for ways to engineer better cameras and sensors.

When it comes to vision, some animals leave humans flying blind, Gruev explained. Light has three properties: color, intensity, and polarization. Humans see color and intensity, but they are totally blind to polarization. Yet filters that polarize light--limiting its vibrations to a single plane--enable us to reduce glare when driving or skiing and increase contrast in microscopy and astronomy.

For You: New Light-based Hearing Devices

Some animals, such as crabs, octopus, and shrimp, use polarized light to find mates, mark their territory, and hunt prey. Yet Gruev knew none of that when he began his quest for what he described as one of the "holy grails" of optics, a single chip that integrates a polarization filter onto a silicon CCD light sensor similar to the ones used for cameras.

By 2010, after eight years of development, he had succeeded in fabricating nanoscale wires that polarized light just as it reached of a CCD sensor. Because he worked with sensors made by silicon semiconductor processing, he estimates his devices will cost only a few dollars at most to produce.

Yet his success left him with a new challenge: "I would go out with my colleagues to grab a beer and talk about our work," he said. "When I told them about the camera, they asked me, 'What can you use it for?' I wasn't too sure."

Fortunately, Tom Cronin, a University of Maryland marine biologist, heard about the camera. He studies how ocean animals communicate with polarized light. "We've been looking for a technology like this for decades," Cronin told Gruev.

After slapping the prototype into an underwater casing and bringing it to Cronin's lab, the two decided to collaborate. Cronin recruited an undergraduate for the summer to design a water-proof casing that would dissipate heat from the CMOS and have room for lots of data. Then Gruev and Cronin took off for Australia to test the results.

The prototype was not a very good camera, Gruev recalled. "It couldn't change its focus or exposure time,” he said. “We wound up with 30 hours of blurry photographs and two minutes of really good stuff."

But the trip did introduce Gruev to mantis shrimp.

We may eat them for cocktails, but in the ocean mantis shrimp are fearsome predators who use polarized light to track prey and communicate with one another. Males, for example, have certain patterns that show only with circularly polarized filters. Those patterns alert males when they enter another male's territory, which can lead to vicious fights.

"From mantis shrimp, I learned some clever tricks that nature implemented," Gruev said.

To start, the eyes of mantis shrimp detect six types of polarization: horizontal, vertical, and two types each of diagonal and spiral.

They have 16 color receptors, 15 more than humans, stacked atop one another. Shorter wavelength receptors go on top and longer wavelengths, which penetrate tissue better, on the bottom. This enables them to accurately differentiate shades of color.

Shrimp eyes also adjust to light intensity. With a typical camera, bright sunlight will often bleach out details that are not in direct sunlight. Parts of the shrimp’s retina exposed to bright light transmit fewer signals, while those looking at dark areas share information from neighboring receptors to resolve details. This enables them to see details even in bright light.

Gruev's new camera copies these traits. The chip uses nanowires to produce four types of polarization (horizontal, vertical, and two diagonals). It also stacks three layers of color-sensitive receptors on top of one another for extra sensitivity.

Unlike conventional cameras, each pixel can attenuate its signal if the light is too bright. This enables the other pixels to resolve details in poorly lit areas. The result is a camera that is 1 million times more sensitive across all light intensities, and that can see details in shadows even in bright sunlight.

Gruev believes those CMOS cameras will eventually outperform conventional cameras used for collision avoidance on cars, especially in fog and low-light conditions. In medicine, their greater sensitivity might help detect small cancers in the body.

"There is plenty of room for all of this technology to find a home," he said.