Artificial Intelligence continues to move forward creating hundreds of applications in the process. While the whole world seems to be surprised by this progress, AI pioneer Geoffrey Hinton believes this “rapid progress in artificial intelligence will continue to accelerate.”
On the eve of the 10-year anniversary of key neural network research, Hinton and other leading AI pioneers rebuked critics who expressed that deep learning has “hit a wall.”
“We’re going to see big advances in robotics — dexterous, agile, more compliant robots that do things more efficiently and gently like we do,” Hinton told VentureBeat in an interview.
AI pioneers defend the success of ImageNet database
Yann LeCun, head of AI and chief scientist at Meta, and Fei-Fei Li, Sequoia Capital Professor of Computer Science at Stanford University, agree with Hinton’s remarks. In 2012, Hinton along with Alex Krizhevsky and Ilya Sutskever published groundbreaking research on the ImageNet database.
The ImageNet database built on previous work to unlock significant advancements in computer vision specifically and deep learning overall. The research was responsible for making deep learning mainstream and the AI pioneers believe this momentum will be hard to stop even a decade later.
In an interview with VentureBeat, LeCun said that obstacles are being cleared at an incredible and accelerating speed. “The progress over just the last four or five years has been astonishing,” he added.
ImageNet, a large-scale dataset of human-annotated photos for developing vision algorithms, was invented by Li in 2006. She says that the evolution of deep learning since the research paper was first published in 2012 has been “a phenomenal revolution that I could not have dreamed of.”
Deep learning faces criticism
For AI professionals, researchers, and academics, the immediate challenge is to train their models to imitate the way humans gain certain types of knowledge. Deep learning has now emerged as the preferred method and is the technology behind driverless cars, helping AI systems recognise a stop sign or distinguish a pedestrian from other objects.
Despite its success, deep learning has faced criticism with critics saying its success is narrow in scope. Some have also claimed that neural nets are not close to being the fundamental breakthrough that will allow for artificial general intelligence.
Gary Marcus, professor emeritus at NYU and the founder and CEO of Robust.AI, wrote about deep learning “hitting a wall” earlier this year. Marcus argues that deep learning has seen some progress but “we are fairly stuck on common sense knowledge and reasoning about the physical world.”
Marcus was joined by regular critic Emily Bender, professor of computational linguistics at the University of Washington. Bender calls it a “deep learning bubble” and said she doesn’t think the natural language processing (NLP) and computer vision models add up to “substantial steps” towards “what other people mean by AI and AGI.”
While the criticism continues to grow, they are unable to take away the fact that deep learning has shown progress in key areas of AI applications like computer vision. Today, the likes of Google, Amazon, Microsoft, Facebook parent Meta, and Apple are racing to harness the power of deep learning and build softwares such as recommendation engines, translation software, chatbots, and more.
Criticism aside, there are issues of AI bias and ethics that loom large on deep learning and its progress.
“I do think that the ability (compute power + effective algorithms) to process very large datasets into systems that can generate synthetic text and images has led to us getting way out over our skis in several ways,” says Bender.
She adds, “We seem to be stuck in a cycle of people ‘discovering’ that models are biassed and proposing trying to debias them, despite well-established results that there is no such thing as a fully debiased dataset or model.”
Bender also says there is a need for the field to be held to real standards of accountability. However, LeCun says “these are complicated, important questions that people tend to simplify.”
As AI pioneers like Hinton, Li, and LeCun mark a decade of progress seen with deep learning, the debates will continue. However, Li believes all debates are in the spirit of science.
“Science is not the truth, science is a journey to seek the truth,” she said. “It’s the journey to discover and to improve — so the debates, the criticisms, the celebration is all part of it.”