IRL engine and visualization

The objective of this artificial intelligence-related project was to develop a software system that enables autonomous agents to represent, interpret, acquire and conceptualize the compositional meaning of referential phrases. This system represents the meaning in terms of a constraint network, such that the interpretation of the often ambiguous meaning becomes a constraint satisfaction problem. The composition takes place through an incremental network construction/search process.

The resulting system is used in experiments on the evolution of artificial languages through language games in which agents –software agents as well as actual robots– conceptualize a topic in a scene, e.g. the big blue ball next to the red box, in order to produce an utterance that refers to that topic. Another agent then reconstructs the meaning given the utterance and interprets the resulting constraint network in order to guess the intended topic. The core system was programmed in Common Lisp.

Next to the main core system, I also developed an interactive visualization in order to facilitate the development and demonstrate the resulting system. The visualization represents the current meaning network, the composition search-tree that constructed to produce the network, the scene in which context the topic is conceptualized, and interface elements for controlling the visualization as well as the engine. The layout of the meaning networks is controlled by a force-directed layout algorithm. This visualization was initially developed as a browser-based application using Adobe Flash, later as a desktop application using Adobe AIR.

This research and development was conducted in the context of the ECAgents project, and is being further developed in the ALEAR project.

In collaboration with:

Prof. Luc Steels, and Joris Bleys

Publications

• Constraint-based Compositional Semantics. Van den Broeck, Wouter. In Andrew D. M. Smith and Kenny Smith and Ramon Ferrer i Chancho, editor, The Evolution of Language: Evolang 7, 2008. World Scientific.
• A constraint-based model of grounded compositional semantics. Van den Broeck, Wouter. In Luis Seabra Lopes, Tony Belpaeme, Stephen J. Cowley, editor, Language and Robots: Proceedings of the Symposium. 10-12 December 2007, Aveiro, Portugal, December 2007.Universidade de Aveiro.