IRL engine and visualization

The objective of this artificial intelligence-related project was to develop a software system that enables autonomous agents to represent, interpret, acquire and conceptualize the compositional meaning of referential phrases. This system represents the meaning in terms of a constraint network, such that the interpretation of the often ambiguous meaning becomes a constraint satisfaction problem. The composition takes place through an incremental network construction/search process.

The resulting system is used in experiments on the evolution of artificial languages through language games in which agents –software agents as well as actual robots– conceptualize a topic in a scene, e.g. the big blue ball next to the red box, in order to produce an utterance that refers to that topic. Another agent then reconstructs the meaning given the utterance and interprets the resulting constraint network in order to guess the intended topic. The core system was programmed in Common Lisp.

Next to the main core system, I also developed an interactive visualization, a screenshot of which is shown above. This application was meant to facilitate the development, as well as demonstrate the resulting system. The application shows the scene/context that contains the conceptualized topic (top-left), the composition search-tree used to generate all possible networks (on the right), one selected meaning networks (in the middle). The layout of the meaning networks is controlled by a constrained force-directed layout algorithm. The application also provided the necessary tools for controlling the visualizations as well as the engine. This visualization was initially developed as a browser-based application using Adobe Flash, and later as a desktop application using Adobe AIR.

This research and development was conducted in the context of the ECAgents project.

In collaboration with:

Prof. Luc Steels, and Joris Bleys

Publications

• Constraint-based Compositional Semantics. Van den Broeck, Wouter. In Andrew D. M. Smith and Kenny Smith and Ramon Ferrer i Chancho, editor, The Evolution of Language: Evolang 7, 2008. World Scientific.
• A constraint-based model of grounded compositional semantics. Van den Broeck, Wouter. In Luis Seabra Lopes, Tony Belpaeme, Stephen J. Cowley, editor, Language and Robots: Proceedings of the Symposium. 10-12 December 2007, Aveiro, Portugal, December 2007.Universidade de Aveiro.