Even though the human visual system has sophisticated machinery for processing color, the brain has no problem recognizing objects in black-and-white images. A new study from MIT offers a possible explanation for how the brain comes to be so adept at identifying both color and color-degraded images.
Using experimental data and computational modeling, the researchers found evidence suggesting the roots of this ability may lie in development. Early in life, when newborns receive strongly limited color information, the brain is forced to learn to distinguish objects based on their luminance, or intensity of light they emit, rather than their color. Later in life, when the retina and cortex are better equipped to process colors, the brain incorporates color information as well but also maintains its previously acquired ability to recognize images without critical reliance on color cues.
Children who receive treatment through a program in India called Project Prakash, started in 2005 by lead study author Pawan Sinha, may also participate in studies of their visual development.
In this study, Sinha and his colleagues gave children a simple test of object recognition, presenting both color and black-and-white images. For children born with normal sight, converting color images to grayscale had no effect at all on their ability to recognize the depicted object. However, when children who underwent cataract removal were presented with black-and-white images, their performance dropped significantly.
To rigorously test their hypothesis that normal visual development depends on lower color information early in life, the researchers used a standard convolutional neural network, AlexNet, as a computational model of vision.
The researchers found that a developmentally inspired model, which is trained first on grayscale before adding color information, could accurately recognize objects in either colored or black-and-white images and was also resilient to other color manipulations. However, a model trained only on color images did not show good generalization to grayscale or hue-manipulated images.
By analyzing the internal organization of the models, the researchers found that those that begin with grayscale inputs learn to rely on luminance to identify objects. The researchers suggest a similar phenomenon may occur in the human brain.