Symbolic vs Subsymbolic AI Paradigms for AI Explainability by Orhan G. Yalçın

Crucially, these hybrids need far less training data then standard deep nets and use logic that’s easier to understand, making it possible for humans to track how the AI makes its decisions. For almost any type of programming outside of statistical learning algorithms, symbolic processing is used; consequently, it is in some way a necessary part of every AI system. Indeed, Seddiqi said he finds it’s often easier to program a few logical rules to implement some function than to deduce them with machine learning. It is also usually the case that the data needed to train a machine learning model either doesn’t exist or is insufficient. In those cases, rules derived from domain knowledge can help generate training data.

The rule-based nature of Symbolic AI aligns with the increasing focus on ethical AI and compliance, essential in AI Research and AI Applications. Symbolic AI offers clear advantages, including its ability to handle complex what is symbolic ai logic systems and provide explainable AI decisions. Symbolic AI’s role in industrial automation highlights its practical application in AI Research and AI Applications, where precise rule-based processes are essential.

Symbolic artificial intelligence

Traditionally, in neuro-symbolic AI research, emphasis is on either incorporating symbolic abilities in a neural approach, or coupling neural and symbolic components such that they seamlessly interact [2]. The work in AI started by projects like the General Problem Solver and other rule-based reasoning systems like Logic Theorist became the foundation for almost 40 years of research. Symbolic AI (or Classical AI) is the branch of artificial intelligence research that concerns itself with attempting to explicitly represent human knowledge in a declarative form (i.e. facts and rules). If such an approach is to be successful in producing human-like intelligence then it is necessary to translate often implicit or procedural knowledge possessed by humans into an explicit form using symbols and rules for their manipulation. Artificial systems mimicking human expertise such as Expert Systems are emerging in a variety of fields that constitute narrow but deep knowledge domains.

The true resurgence of neural networks then started by their rapid empirical success in increasing accuracy on speech recognition tasks in 2010 [2], launching what is now mostly recognized as the modern deep learning era. Shortly afterward, neural networks started to demonstrate the same success in computer vision, too. Lake and other colleagues had previously solved the problem using a purely symbolic approach, in which they collected a large set of questions from human players, then designed a grammar to represent these questions. “This grammar can generate all the questions people ask and also infinitely many other questions,” says Lake. “You could think of it as the space of possible questions that people can ask.” For a given state of the game board, the symbolic AI has to search this enormous space of possible questions to find a good question, which makes it extremely slow. Once trained, the deep nets far outperform the purely symbolic AI at generating questions.

The Rise of Deep Learning

The advantage of neural networks is that they can deal with messy and unstructured data. Instead of manually laboring through the rules of detecting cat pixels, you can train a deep learning algorithm on many pictures of cats. When you provide it with a new image, it will return the probability that it contains a cat.

However, there is a principled issue with such approaches based on fixed-size numeric vector (or tensor) representations in that these are inherently insufficient to capture the unbound structures of relational logic reasoning. Consequently, all these methods are merely approximations of the true underlying relational semantics. While the aforementioned correspondence between the propositional logic formulae and neural networks has been very direct, transferring the same principle to the relational setting was a major challenge NSI researchers have been traditionally struggling with. The issue is that in the propositional setting, only the (binary) values of the existing input propositions are changing, with the structure of the logical program being fixed. And while these concepts are commonly instantiated by the computation of hidden neurons/layers in deep learning, such hierarchical abstractions are generally very common to human thinking and logical reasoning, too. It wasn’t until the 1980’s, when the chain rule for differentiation of nested functions was introduced as the backpropagation method to calculate gradients in such neural networks which, in turn, could be trained by gradient descent methods.

The researchers trained this neurosymbolic hybrid on a subset of question-answer pairs from the CLEVR dataset, so that the deep nets learned how to recognize the objects and their properties from the images and how to process the questions properly. Then, they tested it on the remaining part of the dataset, on images and questions it hadn’t seen before. Overall, the hybrid was 98.9 percent accurate — even beating humans, who answered the same questions correctly only about 92.6 percent of the time. The second module uses something called a recurrent neural network, another type of deep net designed to uncover patterns in inputs that come sequentially. (Speech is sequential information, for example, and speech recognition programs like Apple’s Siri use a recurrent network.) In this case, the network takes a question and transforms it into a query in the form of a symbolic program.

Symbolic AI is still relevant and beneficial for environments with explicit rules and for tasks that require human-like reasoning, such as planning, natural language processing, and knowledge representation. It is also being explored in combination with other AI techniques to address more challenging reasoning tasks and to create more sophisticated AI systems. In this line of effort, deep learning systems are trained to solve problems such as term rewriting, planning, elementary algebra, logical deduction or abduction or rule learning. These problems are known to often require sophisticated and non-trivial symbolic algorithms. Attempting these hard but well-understood problems using deep learning adds to the general understanding of the capabilities and limits of deep learning.

Symbolic Reasoning (Symbolic AI) and Machine Learning

Meanwhile, with the progress in computing power and amounts of available data, another approach to AI has begun to gain momentum. Statistical machine learning, originally targeting “narrow” problems, such as regression and classification, has begun to penetrate the AI field. In the CLEVR challenge, artificial intelligences were faced with a world containing geometric objects of various sizes, shapes, colors and materials.

The Symbolic AI paradigm led to seminal ideas in search, symbolic programming languages, agents, multi-agent systems, the semantic web, and the strengths and limitations of formal knowledge and reasoning systems. Complex problem solving through coupling of deep learning and symbolic components. Coupled neuro-symbolic systems are increasingly used to solve complex problems such as game playing or scene, word, sentence interpretation. In a different line of work, logic tensor networks in particular have been designed to capture logical background knowledge to improve image interpretation, and neural theorem provers can provide natural language reasoning by also taking knowledge bases into account.

His team has been exploring different ways to bridge the gap between the two AI approaches. One solution is to take pictures of your cat from different angles and create new rules for your application to compare each input against all those images. Even if you take a million pictures of your cat, you still won’t account for every possible case.

In natural language processing, researchers have built large models with massive amounts of data using deep neural networks that cost millions of dollars to train. The next step lies in studying the networks to see how this can improve the construction of symbolic representations required for higher order language tasks. The second reason is tied to the field of AI and is based on the observation that neural and symbolic approaches to AI complement each other with respect to their strengths and weaknesses. For example, deep learning systems are trainable from raw data and are robust against outliers or errors in the base data, while symbolic systems are brittle with respect to outliers and data errors, and are far less trainable. It is therefore natural to ask how neural and symbolic approaches can be combined or even unified in order to overcome the weaknesses of either approach.