Can knowledge about how the cortex functions help us to develop new learning algorithms?
Poggio: Yes. If we define intelligence as the ability to pass the Turing test, which is a test of human intelligence, then understanding the human brain is definitely going to help. And neuroscience is doing a good job of getting us there. It has been developing at an exponential rate over the last 20 years. At this point, I believe that it is just a question of time before our knowledge of how the brain works can directly help in engineering areas such as computer vision and machine learning.
Have you done any work along these lines?
Poggio: Yes. Most of our work has been with physiologists in recording signals from the brains of macaque monkeys using electrodes. This produces very precise information because it makes it possible to record data from single neurons. As a result of this work, we have been able to produce a mathematical model of the macaque visual cortex that simulates the learning activity of about one million neurons. We run this as a computer program and have trained it — using thousands of photographs — to recognize eight kinds of behaviors — hanging, running, sleeping, feeding, etc. — among mice that have been genetically altered to have autism, depression or schizophrenia. It simply marks a behavior as “hanging,” “running,” etc. on a video and enters the duration of the behavior into a statistical database. The program also detects transitions from one behavior to another, all of which adds up to a kind of behavioral fingerprint. By automating this process we have been able to objectively relate behavior to the genome.
How accurate is the system?
Poggio: We have compared the system to the output of human annotators and have found that it is at least as good, or better. And it works 24/7 without getting bored!
Could such a technology lead to surveillance systems capable of providing descriptions of human activities?
Poggio: In principle, yes. But of course such a system would need an immense amount of training. And human behaviors are far more complicated than those of mice.
Being able to show a picture to an intelligent artificial system and get a description of what is happening — is that something you are also working on?