Deep Learning, an iteration of machine learning that aims to mimic the human brain without actually matching its ability, has taken the industry by storm. Today, some of the most powerful tools and services powered by AI rely on deep learning to get better with time.
Some of the most commonly used products and services like news aggregators, chatbots, virtual assistants, robotics, and recommendation engines, are all powered by deep learning. However, within deep learning, there is a subset called Cybernetics that has largely gone unstudied by most researchers.
What is Cybernetics?
In an essay, Carlos E. Perez, author of Artificial Intuition and the Deep Learning Playbook, argues that the foundations of deep learning can be traced back not only to the work of McCulloch–Pitts’ model of the artificial neuron, but to the work of Norbert Wiener, who wrote the book “Cybernetics: Control and Communication in the Animal and the Machine.”
The essay further argues that deep learning researchers continue to “reinvent the wheel” but are losing sight of wisdom found in Cybernetics. Cybernetics can be defined as the scientific study of communication and control. With the core concept of circular causality or feedback, it tries to compare human and animal brains with that of machines and electronic devices.
If that sounds familiar then it is because deep learning aims to achieve a similar result. Perez says that deep learning narrative, which became common in 2012, has now become the dominant narrative. He says this narrative circles back to the time when Wiener published Cybernetics in 1948 looking at connectionist thinking.
He further notes that even Alan Turing explored connectionist thinking but his papers were not published until 14 years after his untimely death in 1954. Wiener, who collaborated with Turing, focussed on connectionist thinking but Turing’s papers were published when symbolist thinking began to emerge.
Perez also accuses symbolist thinkers as responsible for burying the connectionist or cybernetics thinking. “The Neural Network narrative was treated as a toxic research topic for several decades. Yann LeCun reminisces that in 1983, Geoffrey Hinton and Terrence Sejnowski had to disguise a paper “Optimal Perceptual Inference” using terminology that would not reveal its neural network origins,” Perez writes.
How does cybernetics explain AI?
In his essay, Perez cites Paul Pangaro as one of the few remaining scholars in the field of cybernetics. He offers an explanation on AI with cybernetics that differs from the classical approach.
- Cognitive systems are autonomous: The traditional AI approach is one where cognitive systems are believed to have an inside and outside. Cybernetics changes that approach by realising the distinction between biological life and inanimate objects. The approach is that biological life is autonomous and they exhibit their own intentional behaviour. These cognitive systems evolve towards surviving within their adapted environments with their autonomous behaviour.
- Organisms map through an environment back into themselves: The second argument is that cognition originates from embodied learning. Cybernetics argues that an organism learns by interacting with its environment and there is a relationship between environment and organism related to both memory and representation. In essence, Cybernetics takes into account the fact that an organism’s representation depends a lot on their environment.
- Nervous systems reproduce adaptive relationships: As established earlier, all biological life is autonomous and that autonomy leads to its own adaptability. Cybernetics says that this same adaptiveness seen in biological life can also be simulated in biological brains.
- Social agreement is primary objectivity: Cybernetics approaches the intersection between knowledge and reality with the framework of Semiotics. “The gist of the argument is that knowledge is captured by icons, indexes, and symbols and that our cognitive development needs to be grounded by icons. Indexes are learned affordances. Symbols arise from the use of words that originate from their use,” writes Carlos E. Perez.
- Intelligence resides in observed conversations: As a connectionist thinking, it should not come as a surprise that cybernetics looks at intelligence being one that gains knowledge through conversations. It aims to weigh in that complete human intelligence is possible from our ability to manage conversations within a social environment.
Why is cybernetics better than conventional AI approaches?
All this argument around cybernetics brings us to the crucial point of how it is better than conventional approaches. Perez writes that conventional AI approaches ignore the holistic nature of organisms and ecosystems. He argues that everything in AI today is being framed from a mechanistic and objective point of view.
“The thinking is that intelligence can be independent of the environment or context. These are all artefacts of GOFAI thinking, but unfortunately, it has infected connectionist thinking,” he adds.
With cybernetics, the approach changes to provide a richer foundation to understand learning as compared to the disembodied and context-free viewpoint of classical AI. With AI becoming centred around humans and biological life, deep learning can be seen as more compatible with this approach than the traditional approach. However, the demise of cybernetics as a narrative for AI makes it a tall order for it to come back and change the discourse.
What to read next?
❄️From stealth mode to data cloud pioneer: a look at Snowflake’s history, key members, major achievements, and future
📕11 best books on AI, its impact in marketing and technology you must read right now
⚖️Former lawyers launch AI-startup Uncover to transform legal litigation