Author: Ehud Reiter
I sometimes find that people are confused about the difference between NLG, NLP and AI. This blog aims to make the differences clear. Apologies in advance for all the acronyms!
What is NLG?
Arria specializes in Natural Language Generation (NLG), i.e. software systems which can automatically generate narratives and reports, in easy-to-read language, which describe, summarize or explain input data. In simple terms – Data in, language out!
How does NLG relate to NLP?
NLG is a type of Natural Language Processing (NLP) (also sometimes referred to as Computational Linguistics [CL]). NLP encompasses all software which interprets or produces human language, in either spoken or written form. There are four key types of NLP, as show in the diagram and table.
|Type of NLP||Description/example||Simple Terms|
Software which understands or transcribes spoken language, such as Dragon NaturallySpeaking.
|Natural language understanding (NLU)||
Software which extracts information from written text, such as some of the tools that make up IBM Watson.
|Text mining, text analytics|
|Natural language generation (NLG)||
Software which produces narratives and reports, in easy-to-read language, such as Arria’s NLG Platform.
|Data in, language out|
Software which speaks or reads out text, such as CereProc CereVoice.
NLP applications can combine different aspects of NLP. For example, speech-to-speech applications, such as Apple’s Siri and other intelligent personal assistants, use all the above types of NLP.
How do NLP and NLG relate to AI?
NLP (and hence NLG) is a type of Artificial Intelligence (AI). AI encompasses software which does “intelligent” things. What counts as “intelligent” changes over time. Current examples of AI include (in addition to NLP and NLG):
- Robotics and computer vision: e.g. face recognition software and self-driving cars.
- Game playing: e.g. AI players and characters in computer games.
- Reasoning and advisory systems: e.g. systems which recommend approving or denying mortgages, and systems which identify potential fraud in financial transactions.
Many AI and NLP systems today use machine learning. Machine learning systems learn how to perform tasks by analyzing examples, instead of being taught explicit rules. For example, a modern face recognition system would learn how to recognize my face by analyzing many images of my face; it would not be programmed with explicit rules such as “wears glasses” and “has grey hair”. There are many algorithms and techniques which can be used in machine learning, including deep learning, decision trees, support vector machines and neural networks.
Arria uses some machine learning (ML) in its NLG systems, but we primarily use alternative “rule-based” technology. This is because we don’t think that NLG systems built purely with today’s ML technology can consistently and reliably produce high-quality narratives. Of course, ML technology is constantly evolving; as it improves, we will incorporate more ML into our technology stack.
Machine learning is related to analytics. Analytics originally evolved out of statistical analysis of large data sets (often done with packages such as SPSS, R or Matlab), but in recent years it has become increasingly similar to AI machine learning; both machine learning and analytics in essence attempt to understand what is going on in large data sets. Some common types of analytics include predictive analytics (e.g. predicting whether a component of a complex machine is likely to fail soon, based on analyzing the pre-failure behaviour of similar components which have failed), and prescriptive analytics (e.g. recommending treatment for a hospital patient, based on analyzing symptoms, treatment and outcome of previous patients).
Using NLG with other forms of AI
The boundary lines between the different types of NLP and AI can be fuzzy, and Arria is often asked to build systems which include other types of NLP (such as text analytics) and AI (such as predictive analytics) as well as NLG. Although Arria’s core expertise is NLG, we have good knowledge of other aspects of NLP and AI, and can work with our partners (such as IBM Watson) to deliver such solutions.
How to access Arria’s NLG
There are several ways to embed NLG If you’re looking to add an NLG component to your data or an existing application – check out our NLG Cloud software development toolkits Articulator Lite and Articulator Pro. Additionally we can work with you to help guide and train your developers to build an application – see our Platform page for more details.
Author: Ehud Reiter is the Chief Scientist of Arria NLG, and also a Professor of Computing Science at the University of Aberdeen. He has worked on natural language generation for the past 30 years, and has published 125 peer-reviewed academic papers and been awarded 5 patents; he also is the author of a widely used NLG textbook. In his university role he has been involved in several medical projects, and has published papers in British Medical Journal and other leading medical journals.