People often ask me which words an NLG system should use to describe numbers. For example, if profits rose by 3%, should this be described as a “slight rise” or a “moderate rise”? Using words to communicate numbers can make narratives easier to read and understand, especially for readers who are “verbal” thinkers.
Unfortunately, it is difficult to give crisp rules for when a particular word such as “moderate” should be used, because different people interpret such words in different ways. Many years ago, Yaji Sripada and I built an NLG weather forecast generator (as a university research project, not as an Arria project), and we did a lot of investigation of how both writers and readers use words that expressed data, especially time phrases such as “by evening”. We discovered that:
- Some people thought “by evening” meant 1800, others thought it meant 2100, and still others thought it meant 0000.
- A few people thought that the meaning of “by evening” depended on either sunset time, or meal time; most thought meaning was not influenced by these factors.
If a weather forecast is going to be read by lots of people, and these people interpret “by evening” in different ways, then the NLG system clearly has a problem! For example, the forecasts that Yaji and I looked at were for offshore oil rigs, and we were concerned that if a forecast said that heavy winds were expected “by evening” meaning 1800, but the rig staff thought “by evening” meant 0000, the rig staff might plan to unload a supply boat at 2100, which would not be good.
We also discovered that some phrases were less ambiguous than others, for example pretty much everyone agreed that “by midnight” meant 0000. So we ended up recommending that NLG-system builders look for words that are interpreted in the same way by most people, and use these wherever possible. This strategy worked well, and indeed human readers in many cases preferred our computer-written forecasts over human-written forecasts, in part because wording (as above) was more consistent and less confusing. If anyone wants to know more about this work, you can read our research paper “Choosing words in computer-generated weather forecasts”.
I have seen many other cases where different people interpret words differently; I discuss some of these in another paper “Human variation and lexical choice”. Other people have also observed this; for example the psychologist Dianne Berry has observed that in the context of communicating risks of side effects of medication, words like “common” are interpreted very differently by different people. Berry also pointed out that if the word “common” is used to indicate a 1-10% chance of a nasty side effect (which is recommended usage in the EU for medication leaflets), but a patient thinks that “common” means a 50% chance (which is how many of Berry’s subjects interpreted the word), then the patient might refuse to take the medication because of this misunderstanding.
I suspect that people who work closely together will probably tend to agree on word meanings, because of a psycholinguistic phenomenon known as “alignment”. But if a narrative is intended for a wide audience, then choosing words that are interpreted consistently by readers may not be straightforward.
My recommendation is to do a careful analysis in order to understand how different users will interpret different words. Don’t just rely on your intuition, because your intuition will tell you how you interpret words, not how other people interpret words. I would start with a questionnaire that asks people about their interpretation of specific words. Ideally you should also do a “corpus analysis” where you analyze how words are used in existing texts; Yaji and I did this when we analyzed word usage in weather forecasts. However, while corpus analysis provides excellent data on word usage, doing a good corpus analysis is a fair amount of work and also requires some specialist skills.
About the author: Arria Chief Scientist, Prof. Ehud Reiter, is a pioneer in the science of Natural Language Generation (NLG) and one of the world’s foremost authorities in the field of NLG. He is responsible for the overall direction of Arria’s core technology development as well as supervision of specific NLG projects. He is Professor of Computing Science in the University of Aberdeen School of Natural and Computing Sciences. Visit his blog here.