Version 1

Lecture: A Pictograph is Worth a Thousand Words

Text-to-Pictograph Translation and Vice Versa for People with an Intellectual Disability

In this lecture, I will present the Picto technology, a set of tools that help improve accessibility to the Internet for people with an intellectual disability (ID). Picto automatically translates Dutch natural language text into pictographs and vice versa, allowing people with limited literacy skills to read and write status updates, emails, and chat messages in online environments.

For the conversion of texts into pictographs, we start from an existing system. We evaluate the baseline Text-to-Pictograph translation system using a combination of automated metrics, manual assessments, and user studies, and we propose three major improvements: We create a spelling correction tool for people with ID, we develop a syntactic simplification tool and a temporality detection module, and we implement a word sense disambiguation tool for improved semantic analysis.

Conversely, the Pictograph-to-Text translation tool provides help in constructing textual messages by allowing the user to input a series of pictographs, and translates these messages into natural language text. The challenge in Pictograph-to-Text translation is twofold. The first task concerns the development of an accessible interface that allows people with ID to find the pictographs of their choice. The second task is the actual development of the Pictograph-to-Text translation engine. We discuss a variety of approaches, including language modelling and (neural) machine translation techniques, toward the generation of rich natural language text from underspecified pictograph input.


Day: 2019-04-28
Start time: 14:45
Duration: 00:30
Room: lecture room

Language: en


Concurrent Events