Talking to an Even Smarter Robot
How a Robot Should Talk to a Human

Task: Improving and Expanding Kitchen-Based Dialog Modeling for Robots

The user interacts with a kitchen assistant robot (e.g., ARMAR III) and wants to eat something for lunch.

1. Making Kitchen / Recipe / Mensa Assistance more robust:
• kitchen assistance: “Please bring me the ingredients for pasta napoli!”
• recipe assistance: “What do I have to do first for spaghetti bolognese?”
• mensa information: “Which meals are served on line 1 today?”
2. Adding the vision channel to the interaction
• face recognition: “Hi Steve. How are you? I haven’t seen you in years!”
• gaze detection:
  “Why are you looking away, if I’m talking to you?
  Is that other robot over there more interesting than me?!”
3. Social Interaction
   • improving social interaction using dialog and vision cues

Project Setup

• expanding kitchen dialogs and making them more robust
• familiarizing oneself with the existing spoken dialog framework
• integrating the vision channel using off-the-shelf toolkits
• programming in Java (or C/C++) and Tcl

Please note :

• project is conducted in English
• German clarification/support available, if needed