Robots that Talk and Listen

Robots that Talk and Listen

by Judith Markowitz, Sean Andrist, Guido Bugmann, Jonathan Connell, David Dufty, Francois Grondin, Francois Michaud, Nicole Mirnig, Roger Moore, Steve Mushkin, Bilge Mutlu, Fumiko Nazikian, Victor Ng-Thow-Hing, Sandra Okita, Allison Sauppé, Manfred Tscheligi, Alan Wagner, Jörg Wolf
Released December 2014
Publisher(s): De Gruyter
ISBN: 9781614519157

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Robots That Talk and Listen provides a forward-looking examination of speech and language in robots from technical, functional, and social perspectives.

Contributors address cultural foundations as well as the linguistic skills and technologies that robots need to function effectively in real-world settings. Among the most difficult and complex is the ability to understand and use language.

Speech-enabled automata are already serving as interactive toys, teacher’s aides, and research assistants. These robots will soon be joined by personal companions, industrial co-workers, and military support automata.

The social impact of these and other robots extends well beyond the specific tasks they perform. Contributors tackle the most knotty of those issues, notably acceptance of advanced, speech-enabled robots and developing ethical and moral controls for robots.

Topics in this book include:

• Language and Beyond: The True Meaning of “Speech Enabled”
• Robots in Myth and Media
• Enabling Robots to Converse
• Language Learning by Automata
• Handling Noisy Settings
• Empirical Studies of Robots in Real-World Environments
• Acceptance of Intelligent Robots
• Managing Robots that Can Lie and Deceive
• Envisioning a World Shared with Intelligent Robots

Table of contents

  1. Title Page
  2. Copyright Page
  3. Table of Contents
  4. List of contributing authors
  5. About the Editor
  6. Preface
  7. Part I: Images
    1. My robot
      1. I Introduction
      2. II Generative-research methodology
      3. III What children want from technology
        1. A Methodology
        2. B Findings and discussion
      4. IV Robots
        1. A Prior research
        2. B Robot study
          1. 1 Methodology
          2. 2 Findings and discussion
      5. V Conclusions
      6. Acknowledgements
      7. References
    2. Cultural icons
      1. I Introduction
      2. II Robot as killer
        1. A The androids of R.U.R.
          1. 1 History
          2. 2 Speech and language
        2. B The Frankenstein monster
          1. 1 History
          2. 2 Prometheus
          3. 3 Speech and language
        3. C Managing killer robots
      3. III Robot as servant
        1. A Karakuri ningyo
          1. 1 History
          2. 2 Modern robots as karakuri
          3. 3 Speech and language
          4. 4 New incarnations
        2. B Golem
          1. 1 History
          2. 2 Robot as golem
          3. 3 Speech and language
      4. IV Robot as lover
        1. A Pygmalion’s statue
          1. 1 History
          2. 2 Humans loving robots
          3. 3 Speech and language
        2. B Robots loving humans
          1. 1 History
          2. 2 Speech and language
        3. C Managing robot love
      5. V Conclusion
      6. References
    3. Android aesthetics: Humanoid robots as works of art
      1. I Introduction
      2. II The history of automata for art and entertainment
        1. A Automata in ancient times
        2. B Clockmaking and automata
        3. C Karakuri ningyo
        4. D Animatronics
        5. E An android as a self portrait: the work of Hiroshi Ishiguro
        6. F Hanson robotics
      3. III The Philip K. Dickandroid
      4. IV Robot components
      5. V Building a dialogue system
        1. A Gricean maxims
        2. B Architecture
        3. C Conversing like Philip K. Dick
      6. VI Conversational competence
        1. A Be orderly
        2. B Background noise
        3. C Cooperative principle
        4. D Competence
      7. VII The scientific value of androids
      8. VIII The value of androids as art
      9. IX The uncanny valley: a possible obstacle to artistic androids
      10. X Consciousness
      11. XI Conclusion
      12. References
  8. Part II: Frameworks and Guidelines
    1. Enabling human-robot dialogue
      1. I Introduction
      2. II Review of the literature
        1. A Multimodal, multiparty dialogue
        2. B Situated interaction
        3. C Joint action
        4. D Linguistic and nonverbal effectiveness
        5. E Adaptive dialogue
      3. III A framework for human-robot dialogue-systems
        1. A Multimodal language processing
        2. B Domain processing
        3. C Task model
        4. D Dialogue model
        5. E Multimodal production
        6. F Adaptive dialogue
      4. IV Enabling effective human-robot dialogue
        1. A Task model for instruction and repair
          1. 1 Study of how humans detect and repair breakdowns
          2. 2 Task model and system structure
          3. 3 Evaluation of the task model in robot-human instruction
          4. 4 Discussion
        2. B A production model for expert robot speech
          1. 1 Models of expertise
          2. 2 Production model for how experts talk
          3. 3 Evaluation of production model in human-robot dialogue
          4. 4 Discussion
        3. C Summary
      5. V Opportunities and challenges for future work
        1. A Linking task and dialogue models
        2. B Development of reusable models
        3. C Open sharing of models and components
      6. VI Conclusion
      7. Acknowledgements
      8. References
    2. Robots can talk – but can they teach?
      1. I Introduction
      2. II Androids in the classroom
      3. III Foreign-language teaching
        1. A The evolution of foreign-language teaching
        2. B The ACTFL guidelines
        3. C Assessing accuracy in Japanese
      4. IV Robots and the ACTFL guidelines
      5. V Identifying the difficulties facing Japanese-language learners
        1. A Learning sounds and prosody
        2. B Grammar
        3. C Sociocultural aspects
        4. D Pragmatic strategies
        5. E Can a speech-enabled robot teach?
      6. VI Global communication and the intercultural speaker
        1. A Dialogue and the intercultural speaker
        2. B Robots as an intercultural link
      7. VII Conclusion
      8. References
    3. Comprehension, Coherence and Consistency: Essentials of Robot Feedback
      1. I Introduction
      2. II Prior work
      3. III A framework for human-robot interaction
        1. A Introduction
        2. B Robot feedback
        3. C Mental models
        4. D Three basic principles
          1. 1 Comprehension
          2. 2 Coherence
          3. 3 Consistency
      4. IV Conclusion
      5. Acknowledgements
      6. References
  9. Part III: Learning
    1. Extensible grounding of speech for robot instruction
      1. I Introduction
        1. A Eldercare as a domain
        2. B Language and learning
        3. C Cultural bootstrapping
      2. II Grounding substrate
        1. A Object finding
        2. B Object properties
        3. C Gesture recognition
        4. D Speech interpretation
        5. E Manipulation routines
      3. III Demonstration of abilities
        1. A Scene understanding
        2. B Object naming
        3. C Semantic web access
        4. D Procedure learning
      4. IV Adding motivation
      5. References
    2. Lies and deception: Robots that use falsehood as a social strategy
      1. I Introduction
      2. II Prior work
      3. III Basic elements
      4. IV Framework
        1. A Representing an interaction
        2. B Outcome-matrix transformation
        3. C Stereotyping
      5. V Implementation
        1. A Examining the factors that influence the decision to lie
        2. B Using stereotypes and partner modeling to predict the cost of lying
      6. VI Summary and future work
      7. VII Conclusion
      8. References
    3. Robotic learning from multimodal instructions: a card game case study
      1. I Introduction
      2. II Related work
      3. III Human-to-human instruction
      4. IV System components: instructor input
        1. A Overview
        2. B Speech
          1. 1 Corpus-based grammar
          2. 2 Grammar implementation
        3. C Non-verbal input
          1. 1 Action primitives
          2. 2 Classification and grouping
        4. D Multimodal integration
        5. E Temporal and semantic integration
      5. V Robot agent learning
        1. A Overview
        2. B Rule frames
        3. C Action selection at execution time
        4. D Mapping issues
          1. 1 Underspecification
          2. 2 Generalization
          3. 3 Perspective mirroring
      6. VI Dialogue management (DM)
      7. VII System evaluation
        1. A Approach
        2. B Experiment 1: dealing instruction
          1. 1 Methodology
          2. 2 Observations
        3. C Experiment 2: teaching four rules
          1. 1 Methodology
          2. 2 General observations
          3. 3 Error categories and frequency
        4. D Errors per rule
      8. VIII Discussion of errors
        1. A Human error
        2. B Dialogue errors
        3. C Grammar coverage
        4. D Manipulation recognition and multimodal integration
      9. IX Summary and conclusions
        1. A Summary
        2. B Corpus-based approach
        3. C Demonstration channel
        4. D Multimodal integration
        5. E Conclusion
      10. References
  10. Part IV: Design
    1. Real-time audition system for autonomous mobile robots
      1. I Introduction
      2. II Issues and challenges in robot audition
        1. A Microphones
        2. B Reverberation
        3. C Environmental noise
        4. D Ego noise
        5. E Real-time performance
      3. III ManyEars: an open framework for robot audition
        1. A Localization
        2. B Tracking
        3. C Separation
        4. D Post-filtering
      4. IV Recognition
        1. A Automatic speech recognition (ASR)
        2. B Speaker recognition
        3. C Emotion, music and daily sounds recognition
      5. V Conclusion
      6. Acknowledgements
      7. References
    2. The effects of design choices on human-robot interactions in children and adults
      1. I Introduction
      2. II Prior work – the evolution of the role of robots
      3. III Social schemas and social metaphors
      4. IV Design choices in lower-level communication modalities
        1. A Type of voice
        2. B Speed setting of gestures
      5. V Design choices in higher-level communication modalities
        1. A Proxemics and social schemas
        2. B Level of attention
      6. VI Effects of developmental differences on design choices
        1. A Level of intelligent behavior in robots
        2. B Realism and contingency level
      7. VII Conclusions and future work
      8. References
  11. Part V: Conclusion
    1. From talking and listening robots to intelligent communicative machines
      1. I Introduction
      2. II Looking for solutions
        1. A Beyond speech
        2. B Beyond words
        3. C Beyond meaning
        4. D Beyond communication
        5. E Beyond dialogue
        6. F Beyond one-off interactions
      3. III Towards intelligent communicative machines
        1. A Achieving an appropriate balance of capabilities
        2. B A consolidated perspective
        3. C Beyond human abilities
      4. IV Conclusion
      5. Acknowledgements
      6. References
  12. Index

Product information

  • Title: Robots that Talk and Listen
  • Author(s): Judith Markowitz, Sean Andrist, Guido Bugmann, Jonathan Connell, David Dufty, Francois Grondin, Francois Michaud, Nicole Mirnig, Roger Moore, Steve Mushkin, Bilge Mutlu, Fumiko Nazikian, Victor Ng-Thow-Hing, Sandra Okita, Allison Sauppé, Manfred Tscheligi, Alan Wagner, Jörg Wolf
  • Release date: December 2014
  • Publisher(s): De Gruyter
  • ISBN: 9781614519157