Now home in Scotland after taking part in the 12th International Conference on Natural Language Generation, Arria’s Chief Scientist & University of Aberdeen Professor Ehud Reiter and Applied Research Scientist Craig Thomson had lots to share about their experiences.
With the conference’s jam-packed schedule – an extra half-day was added this year to accommodate all the presentations – it’s no wonder Ehud and Craig had so much to be excited about. Presenters covered topics as diverse as a personalized data-to-text support tool for cancer patients, generation of Finnish poetry with aesthetics and framing, and generation of hip-hop lyrics with hierarchical modeling and conditional templates. (Craig said that one of the more memorable moments from the conference was when our esteemed Professor Reiter raised his hand and asked “what constitutes good hip hop?”)
From a commercial perspective, Ehud found it noteworthy that Amazon, Facebook, Microsoft, and Google were all present and talking about NLG work. Amazon and Facebook seemed mostly interested in social dialogue in Alexa-like contexts” he said.
Ehud also noted interesting applications already in place commercially, including the generation of hotel descriptions for an online travel site, and the generation of personalized response letters for people who write to their congresspeople. Clearly, the range of commercial applications of NLG continues to expand as enterprises recognize the value of human-language explanation and machine-generated, personalized writing that feels distinctly human.
Several conference speakers presented papers on the subject of reducing “hallucination” in neural-net NLG systems. Hallucination in this context is when NLG systems start saying things in their narratives that are completely bogus. “Hallucination is a big barrier to the use of neural techniques in NLG, so if it can be addressed, then neural techniques look much more attractive,” said Ehud.
Craig also found the neural NLG discussions interesting. “Neural NLG is neat for various reasons,” he said, “not least of which is that it is easier to build quickly. But this is why evaluation is such an important topic right now. How do you make it easier to evaluate the quality of NLG outputs?”
Evaluation of NLG systems was indeed a key theme this year, and Ehud’s involvement in the conference was largely focused on this. An example of NLG evaluation would include testing whether a new weather-forecast generator produces better forecasts than existing forecast generators. Throughout the four days – in many of the workshops, presentations, and discussions among researchers – there was a lot of discussion about methods of evaluation for NLG systems.
On Friday at the DSNNLG workshop, participants discussed designing experiments that rely upon crowdsourced workers (e.g. Amazon Mechanical Turk). Craig explained that recruiting people through such online platforms is becoming increasingly popular when evaluating systems (not just in NLG). These discussions focused on ways to make NLG experiments better, while also improving the work from the participants’ perspective. For example, participants talked about making the work more fun or interesting by gamifying the task, and paying workers ethically.
Craig greatly appreciated his conversations with other researchers, including some who expressed interest in future collaboration, or even just helping each other out, and how they might do that better. “Networking like this is a very important part of the conference. Speaking to others helped me clarify my research. I feel better able to explain it and have more concrete ideas for my next experiments,” he said.
One last thing to mention here before we close out this conference wrap-up is this: There is an exciting new multi-million-pound EU project, called NL4XAI (Natural Language for Explainable AI), that is focused on using NLG and NLP to explain the reasoning of AI systems. Arria is thrilled to be a partner in this project. At the INLG conference, it was the NL4XAI project that organized the Interactive Natural Language Technology for Explainable AI workshop.
The Interactive Natural Language Technology for Explainable AI workshop was where Ehud gave his thoughts on NLG challenges in explainable AI. XAI is attempting to make artificial intelligence systems self-explanatory. Ehud’s work reminds us all that it is high-quality, effective NLG that will enable AI systems to communicate successfully with humans in natural language.
Ehud has more to say about the conference here: https://ehudreiter.com/2019/11/05/evaluation-and-methodology-at-inlg/
Professor Ehud Reiter, a founder and chief scientist at Arria NLG, is one of the world’s leading experts on natural language generation. A scholar at the University of Aberdeen, he also serves as chair of the Special Interest Group on Generation (SIGGEN) of the Association for Computational Linguistics. Professor Reiter has a long-standing interest in using NLG to communicate effectively with patients and people in stressful situations, which he has pursued in projects such as STOP and Babytalk. He is currently a partner in two EU projects: one in Personal Health Interfaces Leveraging Human-Machine Natural Interactions (PhilHumans), which is enhancing how medical apps communicate with patients; and another in the Natural Language for Explainable AI (NL4XAI) project, which is supporting the right to explanation by AI-powered systems. Professor Reiter's work has been highlighted in impact success stories from both the UK’s Engineering & Physical Sciences Research Council and the UK government’s Digital Economy research theme.
Craig Thomson is an Applied Research Scientist at Arria, with us on a doctoral studentship from the Engineering & Physical Sciences Research Council. He holds an MSc in computing science from Robert Gordon University, and is two years into his doctoral degree at University of Aberdeen, under supervisors Ehud Reiter and Yaji Sripada. Before beginning his doctoral studies, Craig worked in industry as a software developer. His PhD work focuses on deep learning to determine the discourse structure in which we should say things within NLG systems. His current work for Arria involves figuring out how recent academic advances can be incorporated into Arria’s software, developing and testing prototypes, thus ensuring that Arria continues to broaden its competitive advantage as the most advanced NLG platform.