Although self-driving cars and sophisticated robots use advanced cameras, computer algorithms and artificial intelligence (AI) to picture their surroundings, these artificial eyes struggle to remain reliable in mixed lighting conditions. A team of researchers, co-led by an engineer from Penn State, has proposed a solution that mimics the mechanics of the human eye to adapt from bright to dark light in seconds.
They did this by adjusting how one of the main electrical components used in these optical systems are built, employing a new design that swells or desorbs with water depending on the light levels present. The approach, detailed in a paper published today (June 9) in Nature Communications, illuminates a road to building systems that could potentially process light data faster and more adaptively than humans.
The improved components are known as memory resistors, or “memristors” — tiny electrical devices that can store information or data in a system, even if the original power source fueling the application is removed. These devices mimic the complex way neurons process and store data in the brain. Photomemristors are a type of memristor capable of sensing and collecting light information then translating it into an electrical current, a process that could more effectively power advanced cameras and optical systems.
According to Larry Cheng, James L. Henderson Jr. Memorial Associate Professor of Engineering Science and Mechanics at Penn State and co-corresponding author on the paper, traditional photomemristors are calibrated and optimized for consistent lighting conditions. Although this allows the systems to work well in both bright and dark environments, maintaining recognition accuracy in changing or mixed lighting can be challenging.
“Self-driving cars are exposed to a mixture of light levels in use — imagine the contrast of the dark sky with the bright headlights of other cars when driving at night,” Cheng explained. “It can be difficult for an artificial optical system to distinguish details, like the glow of a red light, in these mixed lightning conditions.”
Inside the human eye, a series of rod and cone cells helps adjust vision to different lighting conditions. Specific pigments in the rod cells allow the eye to distinguish details, even in the dark. In bright light, though, these pigments in the rod cells “bleach” before slowly regenerating, while the cone cells remain to allow the eye to discern contrasting details. The team theorized that this process, mimicked in a photomemristor, could offer monitoring more adaptive and accurate than traditional designs.
To achieve this, the team mainly built their photomemristors out of two different materials: a stretchy, gel-like plastic known as PEDOT:PSS; and titanium oxide (TiO2), a white, powdery compound derived from the metal titanium. According to Cheng, the TiO2 can capture light from the environment and convert it into an electrical current, known as photocurrent — that voltage then passes through the conductive surface of the PEDOT:PSS and adjusts how much water is allowed to absorb into the plastic from the surrounding environment.
Cheng said the material rapidly absorbs water in dark environments, and it desorbs in light conditions, drying out the PEDOT:PSS. This effect allows the device to self-regulate its sensitivity based on light information pulled from the environment.
“This key design difference allows us to dynamically adapt to changing light conditions, compared to traditional systems that are usually developed for one static scenario,” Cheng said. “By mimicking the way the eye works, we can create photomemristors that work much more reliably for applications in mixed lighting environments.”
The team first tested their devices by exposing them to different intensities of ultraviolet (UV) light. Tests showed that the new photomemristors could more efficiently and accurately detect the intensity of UV light, while producing consistent readings regardless of the external humidity. That flexibility and usefulness comes in a small package, with each photomemristor measuring only half a millimeter across — slightly smaller than a credit card’s thickness.
“In principle, they can be scaled up and down to fit their application,” Cheng said. “By connecting individual photomemristors to form an array, we can better recognize large light patterns in the environment without increasing the size of the photomemristor, keeping it flexible.”
To further assess the components, the team devised an experiment reminiscent of a test given by eye doctors. A four-by-four array of photomemristors was integrated with a neural network, a form of artificial intelligence that mimics the way neurons process data to identify patterns, to create a rudimentary vision system like those used in cars and robots. The team then placed a series of LED lights in the shape of the letter “F” in front of a larger LED backdrop, which they were able to tune to varying levels of lightness and dimness. According to Cheng, this is because “F” is similar to the “E” used in traditional eye tests but is slightly easier to directionally distinguish. After adjusting the lighting, the team would instruct the vision system to accurately identify the “F” from the backdrop.
After just seven iterations of training, the device combined with the neural network could identify letter patterns with over 95% accuracy in a mixed-light environment.
“Our eyes are more adaptive to differing lighting conditions, but that adjustment can take 20 to 30 minutes to fully complete,” Cheng said. “These photomemristors can adapt to lighting conditions much faster than the human eye, while still capturing detailed information about the external environment.”
Going forward, the team plans to further develop the photomemristors into a larger, multi-modal sensing system capable of simultaneously interpreting visual and tactile data from the environment. By integrating multiple forms of sensing into a singular device, the power usage of these systems could be substantially reduced.
“In the far future, we could see this technology being applied to help visually impaired persons see with the help of artificial optics,” Cheng said, explaining another potential application could be in existing systems powering self-driving cars. “It could also play a major role in human-robot interaction and collaboration, allowing systems like factory robots to better operate in dark or rapidly changing environments.”
Cheng holds additional affiliations in mechanical engineering, biomedical engineering, architectural engineering, industrial and manufacturing engineering, materials science and engineering and the Materials Research Institute. Further collaborator and funding details can be found in the paper. The team has filed a provisional patent for this technology.
At Penn State, researchers are solving real problems that impact the health, safety and quality of life of people across the commonwealth, the nation and around the world.
