Interpersonal Coordination PDF

This book explores the fascinating area of interpersonal coordination in force production tasks, outlining the author’s extensive research to date and presenting stimulating new perspectives. The purpose is to provide a detailed exposition of current understanding of the science behind interpersonal joint action. Readers will find clear explanation of concepts from social cognition and neuroscience that are key to an understanding of the field, including the social brain hypothesis, the mirror neuron system, and joint action, as well as other relevant background information. The author then proceeds to present an overview of recent original studies on interpersonal movement coordination performed at his laboratory in Japan. These studies provide insights into such issues as complementary and synchronous force production in joint action, bidirectional transfer between joint and solo actions, and motor control hierarchy in joint action involving bimanual force. They also set the direction for integration of knowledge of physical properties and social cognition. The book will be of interest for researchers and graduate students in all areas of the biomedical sciences.

Autor Nobuyuki Inui | Ersan Kabalci | Yasin Kabalci | Masakazu Washio | Chikako Kiyohara

123 downloads 4K Views 7MB Size

Report

Download pdf

Recommend Stories

Empty story

Idea Transcript

Nobuyuki Inui

Interpersonal Coordination A Social Neuroscience Approach

Interpersonal Coordination

Nobuyuki Inui

Interpersonal Coordination A Social Neuroscience Approach

Nobuyuki Inui Laboratory of Human Motor Control Naruto University of Education Naruto-shi, Tokushima, Japan

ISBN 978-981-13-1764-4 ISBN 978-981-13-1765-1 (eBook) https://doi.org/10.1007/978-981-13-1765-1 Library of Congress Control Number: 2018955474 © Springer Nature Singapore Pte Ltd. 2018 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore

Preface

From the 1970s to the 1980s, I studied spino-olivo-cerebellar pathways in the cat using an evoked potential and a tracing method (HRP, horse radish peroxidase) in the Faculty of Medicine at Gifu University. After leaving Gifu, I tried studying the timing and force control of human movements on the basis of cerebellar functions. Concretely speaking, I first studied timing of serial reactions in a task of tracking serial light stimulation (Inui et al. 1995). Because I noticed that participants tapped a series of keys with a light keystroke as practice progressed, I tried to measure the force produced by the keystroke. Because it is too difficult for me to measure both movement time and force of a series of keystrokes, I measured both intertap interval and force in finger-tapping sequences (Inui et al. 1998). However, the force measured by the tapping task was not fingertip force but impact force. I then come to the study on the control of force and timing during periodic isometric force production. Such change in a task implies the change in the focus of the study from movement timing to the control of force production. Using periodic isometric force production, we first examined the control of force and timing during periodic isometric force production of the right index finger (Masumoto and Inui 2010). The important finding of this study is that decreasing isometric force to achieve the lower force target was markedly more variable than increasing isometric force to achieve the higher force target. Next, we examined the control of force and timing during bimanual periodic isometric force production (Masumoto and Inui 2012). Although without vision the correlation between the two finger forces was strongly positive over force level, with vision it changed from negative to positive with force level. The result with vision indicated that the strategy of the bimanual force control changed from force error compensation to force coupling and the available redundancy thus decreased with an increase in force. Finally, we reached the study on the control of force and timing during interpersonal periodic isometric force production (Masumoto and Inui 2013). Two participants simultaneously adopted both complementary and synchronous strategies exclusively when the target forces for the pair and a sum of the force produced by them were displayed on a monitor. This suggests that two people form representations of shared goals when the total force is visible. Surprisingly, the joint (interpersonal) action controlled force and timing more accurately than the individual action. v

vi

Preface

The direction and method of the study on body image in my previous monograph (2016) were provided by Dr. Simon Gandevia (Neuroscience Research Australia). On the contrary, the direction and method of the study on interpersonal coordination (joint action) involving this book are along the lines of our previous studies (Inui et al. 1995, 1998; Masumoto and Inui 2010, 2012) on the control of timing and force production in intrapersonal coordination. So this book is based on our original studies on interpersonal motor coordination. In an experimental setup of tracking serial light stimulation (Inui et al. 1995), six touch switches are placed beneath a six-light display. This setup itself perhaps makes participants produce serial reactions. Similarly, we use an experimental setup that two participants are seated on chairs at opposite ends of a table facing the load cell and computer monitor (Masumoto and Inui 2013). Although the setup is used to examine the information processing of interpersonal coordination, the setup itself seems to make the participants share intentionality or intersubjectivity. Both Drs. Hubel and Wiesel, who received the Nobel Prize in Medicine or Physiology in 1981, seemed to rewrite their manuscript more than ten times before they submitted the manuscript a journal. To make good movies, the Japanese famous movie director Ozu or Kurosawa also seemed to rewrite each scene of a scenario many times with his writer Noda or Hashimoto for several months in a regular room of a regular Japanese-style inn. I was inspired by these episodes, and I have aggressively published our studies with American and European journals as a result. Here, I present the fruits of our studies on interpersonal movement coordination in my laboratory at Naruto University of Education.

References Inui N (2016) Systematic changes in body image following formation of phantom limbs. Springer, Singapore Inui N, Yamanishi M, Tada S (1995) Simple reaction times and timing of serial reactions of adolescents with mental retardation, autism, and Down syndrome. Percept Mot Skills 81:739–745 Inui N, Ichihara T, Minami T, Matsui A (1998) Interactions: timing and force control of fingertapping sequences. Percept Mot Skills 86:1395–1401 Masumoto J, Inui N (2010) Control of increasing or decreasing force during periodic isometric force control. Hum Mov Sci 29:339–348 Masumoto J, Inui N (2012) Effects of force levels on error compensation in periodic bimanual isometric force control. J Mot Behav 44:261–266 Masumoto J, Inui N (2013) Two heads are better than one: both complementary and synchronous strategies facilitate joint action. J Neurophysiol 109:1307–1314

Naruto, Japan May 2018

Nobuyuki Inui

Acknowledgments

This book was supported by the Japan Society for the Promotion of Science (16K01598).

vii

Contents

1 Introduction�� 1 References�� 7 2 The Background of the Study on Interpersonal Coordination�� 11 2.1 The Social Brain �� 12 2.1.1 Dunbar’s Social Brain Hypothesis�� 12 2.1.2 Brain Regions Specialized for Social Interaction�� 16 2.1.3 Amygdala�� 16 2.1.4 Temporal Pole�� 17 2.1.5 The Brain’s Mirror System�� 18 2.1.6 Posterior Superior Temporal Sulcus�� 19 2.1.7 Medial Prefrontal Cortex �� 20 2.2 The Motor Cortex and Its Relation to Social Behavior�� 22 2.3 The Social Function of the Mirror Neuron System�� 26 2.3.1 Mirror Neuron System and Imitation�� 27 2.3.2 Effects of Sensorimotor Experience on the Observation of Others’ Actions�� 28 2.3.3 Applying Sensorimotor Experience to the Classroom�� 30 2.4 Imitation, Mimicry, and Its Relation to Social Behavior�� 33 2.4.1 Imitation and Its Relation to Social Behavior �� 33 2.4.2 Mimicry and Its Relation to Social Behavior�� 34 2.4.3 The Social Neuroscience of Mimicry�� 35 2.4.4 Interaction Between the Social Psychology and Cognitive Neuroscience�� 36 2.5 Joint Perception�� 40 2.5.1 Perception in a Social Context�� 41 2.5.2 Joint Perception �� 42 2.5.3 We Transcend Our Private Worlds by Responding to the Same Stimulus �� 44 2.6 Observational Motor Learning�� 45 2.6.1 Sensorimotor Adaptation �� 46 ix

x

Contents

2.6.2 Observational Motor Learning�� 48 2.6.3 The Neural Basis of Action Observation�� 49 2.6.4 The Motor System Accompanying Observational Motor Learning�� 51 2.6.5 Sensory Changes Accompanying Observational Motor Learning�� 52 2.7 The Effect of Action Expertise on Shared Representation �� 53 2.7.1 Effects of Expertise on Perception�� 53 2.7.2 Effects of Experimentally Induced Expertise on Perception �� 57 2.8 The Effect of Motor Expertise on Observational Learning in Sports�� 60 2.8.1 Effects of Action Observation on Motor Execution in Sport�� 61 2.8.2 Effects of Motor Expertise on Action Perception in Sport�� 62 2.8.3 Motor Experts Read Body Kinematics�� 64 2.8.4 Neural Systems Underlying Action Perception in Sport�� 66 2.8.5 Motor Expertise and Detection of Deception�� 67 2.8.6 Neural Bases of Deception Detection in Sport�� 68 2.9 The Effect of Shared Representation on Team Sports�� 70 2.9.1 The Effect of Shared Representation on Decision-Making in Team Ball Games�� 71 2.9.2 Shared Representation of Referees and Officials in Team Ball Games�� 74 2.10 The Effect of Shared Representation on Musical Ensemble Performance�� 79 2.10.1 Self-Other Integration and Segregation �� 80 2.10.2 Internal Models for Self, Other and Joint Action Outcome�� 83 2.10.3 Motor Simulation of Self and Other�� 84 2.10.4 Musical Synchronization and Social Interaction�� 87 2.10.5 Neural Mechanism of Synchronizing to Music�� 88 References�� 91 3 An Overview of the Study on Interpersonal Coordination�� 107 3.1 Unintentional Interpersonal Entrainment�� 108 3.1.1 Applying HKB Model to Human Coordination�� 108 3.1.2 Frequency Detuning�� 110 3.1.3 Unintentional Interpersonal Entrainment�� 112 3.1.4 Interpersonal Synergy Involving Intrapersonal Movements�� 114 3.1.5 Informational and Dynamic Constraints on Entrainment�� 118 3.1.6 Social-Psychological Variables�� 122

Contents

xi

3.2 Intentional Interpersonal Coordination �� 124 3.2.1 Perception–Action Matching�� 124 3.2.2 Shared Intentionality �� 126 3.2.3 Shared Representation �� 127 3.2.4 Coordination Strategy�� 129 3.2.5 Perceiving Others’ Abilities�� 129 3.2.6 Representing Others’ Task�� 130 3.2.7 Motor Anticipation�� 131 3.2.8 Communication Through Action �� 132 3.2.9 Interaction of Coordination Mechanisms Using Asymmetrical Joint Action Task�� 133 3.2.10 Representation of Self and Other Actions �� 134 3.2.11 Representation of Space in Relation to Self and Other �� 135 3.2.12 Distinguishing Self and Other �� 137 3.3 Development of Interpersonal Coordination�� 138 3.3.1 Adult-Infant Interaction�� 139 3.3.2 Infants’ Joint Action with Other Infant�� 143 3.3.3 Developmental Mechanisms for Interpersonal Coordination �� 144 References�� 146 4 Complementary and Synchronous Force Production in Joint Action�� 155 4.1 Two Heads Are Better Than One�� 156 4.1.1 Complementary Force Production�� 161 4.1.2 Synchronous Force Production�� 161 4.1.3 Accuracy and Variability of Force Production and Movement Interval�� 162 4.1.4 Complementary Strategy �� 165 4.1.5 Synchronous and Complementary Strategies Simultaneously Facilitate Joint Action�� 166 4.1.6 Two Is Better Than One�� 167 4.2 Is There Social Loafing in Joint Action that Consists of Four People? �� 168 4.2.1 UCM Analysis of Force�� 174 4.2.2 Principal Component Analysis of Force�� 175 4.2.3 Principal Component Analysis of Phase�� 175 4.2.4 Accuracy and Variability of Force and Movement Interval�� 178 4.2.5 Force-Sharing Pattern in Joint Action �� 179 4.2.6 Synchronization of Force Production in Joint Action �� 180 4.2.7 Four and Three-People Groups Performed Better Than Individuals�� 181

xii

Contents

4.3 A Leader-Follower Relationship in Joint Action�� 182 4.3.1 Complementary Strategy �� 184 4.3.2 Accuracy and Variability of Producing the Target Force �� 184 4.3.3 Onset Asynchrony of Force Production�� 185 4.3.4 Force Distribution�� 186 4.4 Effects of Speech on Joint Action �� 188 4.4.1 Complementary Strategy �� 190 4.4.2 Temporal Synchronous Strategy: Analysis of Frequency�� 191 4.4.3 Control of Force and Timing �� 192 4.5 Load Perturbation Facilitates Interpersonal Error Compensation During Joint Action�� 195 4.5.1 Interpersonal Compensation for Force Error�� 202 4.5.2 Accuracy and Variability of Force Production�� 204 4.5.3 Consistency of Force Variability Following Load Perturbation �� 205 References�� 207 5 Is There Bidirectional Transfer Between Joint and Solo Actions?�� 211 5.1 Accuracy of Force Production�� 216 5.2 Variability of Force Production�� 216 5.3 Complementary Force Production�� 217 References�� 219 6 Motor Control Hierarchy in Joint Action that Involves Bimanual Action�� 221 6.1 Complementary Strategy�� 229 6.2 Temporal Synchronous Strategy �� 231 6.3 Accuracy and Variability of Force Production and Movement Interval�� 234 6.4 Relations Between Motor Redundancy and Hierarchical Motor Control �� 235 6.5 Complementary Force Production Is Stronger and Synchronization Is Weaker Interpersonally than Intrapersonally�� 237 6.6 Control of Force and Timing�� 238 References�� 239 7 Conclusion�� 241 References�� 244

Chapter 1

Introduction

Abstract In synchronized swimming, dances, and piano duets, it appears as if synchrony among performances of some people is higher than that among movements of four limbs in individuals. Against the background of the social brain hypothesis, mirror neuron system, and theory of minds, interpersonal coordination has recently been studied using the term “joint action”. Many studies have examined mechanisms underlying coordinative behavior in two-person tasks. One type of the studies has mainly examined the mechanism of unintentional interpersonal coordination in cases of rhythmic behaviors, such as synchronized movements between people. Another type of the studies has investigated the mechanism of intentional interpersonal coordination in cases of non-rhythmic behavior, often discussed in context of imitative versus complementary movement. However, the study on joint action has to not only analyze the actions in time and space, but also the interaction of force produced by more than two people. This book thus reveals the information- processing of complementary and synchronous force production in joint action. In addition, the book addresses a leader-follower relationship in joint action, effects of speech or load perturbation on joint action, transfer between joint and solo actions, and motor control hierarchy in joint action that involves bimanual action. The complementary force production or force error compensation examined in this book consistently underlies the uncontrolled manifold hypothesis. This hypothesis quantifies variability in the dimension where performance is unaffected by changes in elemental variables (task-irrelevant dimension) and variability in the dimension where performance is affected by changes in element variables (task-relevant dimension). Keywords Joint action · Complementarity · Synchronization · Uncontrolled manifold hypothesis Dunbar and Schultz (2007) found a linear relationship between social group size and relative neocortex volume (indexed as the ratio of neocortex volume to the volume of the rest of the brain). From this finding, they propose social brain hypothesis that a necessity of high information-processing increases with an increase in group size, and results in the revolution of the brain in the primate by leads and bounds. © Springer Nature Singapore Pte Ltd. 2018 N. Inui, Interpersonal Coordination, https://doi.org/10.1007/978-981-13-1765-1_1

1

2

1 Introduction

They guess that group size is innately 150, and the 150 people may be innervated in a group without a hierarchy. Taking an opportunity of proposing the social brain, studies on interpersonal communication are noticed in the fields of cognitive science and neuroscience. In addition, theory of minds (Premack and Woodruff 1978), our ability to read the minds of others, and mirror neuron system in the premotor and parietal cortices (Rizzolatti and Craighero 2004), neurons that respond when we act and also when we see someone else act, are found, and perceptual-motor systems are thus studied corresponding with these findings. Although I briefly sum up the background of interpersonal coordination here, I am going to elaborate on its background in Chap. 2. Against the background of such hypothesis and finding, group action coordination has recently been studied using the term “joint action”, which is defined as a social interaction whereby two or more individuals coordinate their actions in space and time to bring about a change in the environment (Sebanz et al. 2006). Although the behavioral and neural processes underlying such joint action are still poorly understood, there are a few findings of joint action. Many studies have examined mechanisms underlying coordinative behavior in two-person tasks (see Knoblich et al. 2011 for a review). One type of the studies has mainly examined the mechanism of unintentional interpersonal coordination in cases of rhythmic behaviors, such as synchronized movements between people (Repp and Su 2013; Schmidt and Richardson 2008). I briefly sum up unintentional interpersonal coordination here, although I am going to review its previous studies in Sect. 3.1. When two people perform a rhythmic behavior, their behavior tends to synchronize automatically. Such automatic synchrony is known as entrainment (Schmidt and Richardson 2008). Because entrainment happens regardless of whether or not an intention to coordinate is present, it can be considered a form of emergent coordination (Knoblich et al. 2011). Entrainment occurs even when participants are instructed to intentionally maintain their preferred rhythms (Schmidt and O’Brien 1997) and when they are completely unaware that their behavior are influencing one another (Oullier et al. 2008; Richardson et al. 2007). In dynamic systems approach, entrainment is thought to reflect the coupling of rhythmic oscillators (Schmidt and Richardson 2008). It is observed in both mechanical systems, such as when two pendulum clocks placed on a wall become synchronized (Huygens 1673), and in biological systems, such as when the hand movements of a person performing bimanual actions synchronize (Haken et al. 1985). Ecological psychologists have studied entrainment in an interpersonal context to determine whether dynamical principles of intrapersonal coordination are extended to the interpersonal case (Marsh et al. 2009). Many studies on entrainment have shown that the joint movements of different people follow the same dynamic principles as the joint movements of an individual’s limbs. For example, when two people perform a rhythmic behavior such as swinging a pendulum (Schmidt et al. 1998), rocking in a rocking chair (Richardson et al. 2007), or walking side-by-side (van Ulzen et al. 2008), their behavior tend to syncvhronize automatically. In addition, entrainment occurs in a wide range of social contexts, including the s ynchronization of body sway during conversation (Shockley et al.

1 Introduction

3

2003), and in larger groups of people, such as a theatre audience clapping in unison (Neda et al. 2000). Clearly, entrainment contributes to joint action in which synchronization is a main part of the goal of joint action, such synchronized swimming or dancing. Richardson et al. (2007) found that people sitting in rocking chairs next to each other fell into synchrony even when the eigen-frequency of the two rocking chairs differed. This tendency to entrain may come at the cost of increased energy expenditure, and it seems to be preserved even under unfavorable conditions. Eaves et al. (2008) further asked participants to walk on a treadmill, finding that the participants spent more energy when they were visually coupled with the image of a body walking in anti-phase (i.e., contralateral arm and leg) than with the image of a body walking in phase (i.e., mirror image of ipsilateral arm and leg). Moreover, synchronization has been shown to have positive effects on other aspect of joint action including language comprehension (Richardson and Dale 2005) and the ability to cooperate (Vadesolo et al. 2010). Synchronization also seems to foster group cohesion. While participants were asked to perform a tapping task, the participants reported liking their partner more when they had performed a taping task in synchrony with their partner than when they had taped at a different tempo, even though they knew that external signals determined the tempo (Hove and Risen 2009). Wiltermuth and Heath (2009) also found that participants who had walked in step in groups of three people made more cooperative choices in a subsequent coordination game than participants who had not walked in step. This synchronized walking group reported feeling more connected and trusted each other more. These findings suggest that entrainment or synchronization plays an important role in increasing group cohesion. As an issue of learning transfer between joint and solo practices, entrainment in the context of joint practice has positive effect on the learning of individual tasks that require precise timing. At juggling, for example, a novice joining a group of more experienced jugglers was able to improve his performance at a faster rate than practicing alone (Huys et al. 2004). Appling this approach to field study on team sports, the attack-defense relationship between two players is recently studied in kendo as a martial art (Okumura et al. 2012) and a play-tag game in which pairs of participants have to remove a tag fastened to their partner’s hip (Kijima et al. 2012). As practice progresses in a play- tag game, the player’s step became synchronized in an anti-phase manner (Kijima et al. 2012). In kendo matches played by expert players, the change in interpersonal distance represented the switch in player’s stepping-toward and stepping-away movements from the anti-phase synchronization at distance less than 2.80 m and in players’ stepping toward/away from the in-phase synchronization at distances greater than 2.90 m (Okumura et al. 2012). In these studies, while players’ step toward-away velocity is the order parameter, interpersonal distance between two players is the control parameter. On the basis of Haken-Kelso-Bunz (HKB) model (Haken et al. 1985), the interpersonal distance is transformed to the relative phase (also see Yokoyama and Yamamoto 2011; Yamamoto and Yokoyama 2011).

4

1 Introduction

Another type of the studies has investigated the mechanism of intentional interpersonal coordination in cases of non-rhythmic behavior, often discussed in context of imitative versus complementary movement (Sebanz et al. 2006; Newman- Norlund et al. 2007; Masumoto and Inui 2013b). In coordination that relies on synchronization, people perform rhythmic behaviors, which combine to achieve the shared action goal. These motor behaviors are often identical, and hence symmetrical. Such symmetrical behaviors is the same as in imitative coordination, which requires people to imitate the action of their partner, such as when lifting a heavy object together. By contrast, in complementary coordination, while two people perform different behaviors, their behaviors link together so that the one compensates for the other to achieve the shared goal. Thus, I briefly sum up intentional interpersonal coordination here, although I am going to elaborate on its previous studies in Sect. 3.2. For example, when two people try to move a table together, they have to produce force complementarily, such that one person produce a strong force, while the other produce a weak force. Few studies have focused on complementarity as a coordination strategy (Bosga and Meulenbroek 2007; Newman-Norlund et al. 2007). Furthermore, there has been little study on both complementary and temporally synchronous strategies (Masumoto and Inui 2013b, 2014a, b; Vesper and Richardson 2014). However, the study on joint action has to not only analyze the actions in time and space, but also the interaction of force produced by more than two people. Bosga and Meulenbroek (2007) asked pairs of two participants to perform a virtual lifting task using continuous isometric force with two or four hands. The forces produced by two participants were negatively correlated when visual feedback of their forces was available, indicating the use of complementary forces to control a virtual bar. Neuroimaging studies in humans found that activities in the mirror neuron system areas were greater during complementary action than imitative action (Newman-Norlund et al. 2007). The areas were also more active in the joint condition than in the solo condition during the same virtual bar lifting task (Newman-Norlund et al. 2008). Furthermore, our studies asked pairs of participants to produce periodic unimanual (Masumoto and Inui 2013b, 2014a, b) or bimanual (Masumoto and Inui 2015) isometric forces such that the sum of forces they produced matched a target force that cycled between 5% and 10% of maximum voluntary contraction (MVC) with a target interval of 1000 ms. When the two participants were provided with visual feedback of the total force produced, they simultaneously adopted both complementary and temporal synchronous strategies (Masumoto and Inui 2013b, 2014a, b, 2015). Then, from error and variability of force production and timing, two people performed better than individuals. Although Koriat (2012) reports such two-heads-better-than-one (2HBT1) effect in a perceptual task, Masumoto and Inui (2013b) first find the effect in perceptualmotor behavior. Thus, this book reveals the information-processing of complementary and synchronous force production in joint action. In addition, the monograph examines a leader-follower relationship in joint action, effects of speech or load perturbation on joint action, transfer between joint and solo actions, and motor control hierarchy in joint action that involves bimanual action.

1 Introduction

5

Column 1.1: Uncontrolled Manifold Hypothesis Bernstein (1967) thought that the problem of motor redundancy in coordination was alleviated by uniting elementary variables into groups and then using one control variable in each group. When a group of variables changes simultaneously, the sum of the variation in each variable generally increases to decrease the variation in the group of variables. A computational method to identify and quantify such coordination has been developed within the framework of the uncontrolled manifold (UCM) hypothesis (Latash et al. 2002; Scholz and Schöner 1999). The UCM framework offers a method of quantifying coordination by comparing the amount of variance within the UCM and orthogonal to it. While the variance orthogonal to the UCM is called the “task-relevant variance”, the variance within the UCM is called the “task- irrelevant variance” (see Fig. 1.1a). Whereas the task-relevant variance introduces errors into performance, the task-irrelevant variance allows the system to be flexible without violating the task. Both have to be quantified per dimension in each of the sub-spaces to be compared quantitatively. If the task-irrelevant variance is significantly larger than the task-relevant one (see Fig. 1.1a), the data show an example of coordination stabilizing that performance variable. Such notion of coordination is experimentally supported by many recent studies. For example, consider a task of the periodic simultaneous force production with bimanual fingers (Masumoto and Inui 2012). Participants performed the task 60 times, and the forces of the two fingers were measured and plotted as points on the force-force plane. Because Masumoto and Inui (2012) plotted data points performed by ten participants, the data showed both intra- and inter-personal variations (Fig. 4.2, Masumoto and Inui 2012). The data points with vision is elongated approximately along the UCM for total force. Analysis of the data indicates significantly more the task-irrelevant variance than the task-relevant one, which can be interpreted as a two-finger coordination stabilizing total force. The data points without vision is oriented orthogonally to the UCM. Analysis of the data indicates significantly more the task-relevant variance than the task-irrelevant one, which can be interpreted as no two-finger coordination stabilizing total force. For another example, consider a task of the periodic simultaneous force produced by two participants (Masumoto and Inui 2013b). Similar to bimanual force production, two participants performed the task 60 times, and the forces of the two right fingers of the participants were measured and plotted as points on the force-force plane. Because Masumoto and Inui (2013b) plotted data points performed by ten groups, the data showed both intra- and intergroup variations (Fig. 4.3a–d). To show only the intra-group variation, Fig. 1.1c and d shows the data of three groups extracted from the data of Masumoto and Inui (2013b) individually. The left cloud of data points is elongated approximately along the UCM for total force when the target forces (continued)

6

1 Introduction

B

Force 1

Force 1

Ta s

k-

re

lev an t

A

sk Ta t an

lev re

-ir

Force 2

D Pair 1 Pair 2 Pair 3

8

Force of participant B

Force of participant B

C

Force 2

10

6 4 2 0

0

2 4 6 8 Force of participant A

10

10 8 6 4 2 0

0

2

4 6 8 Force of participant A

10

Fig. 1.1 An illustration of changes in two sources of force variability: the task-relevant and task-irrelevant dimensions. (Adapted from Latash et al. 2002) (a) The cloud of data points is elongated approximately along the task-irrelevant dimension. Dashed lines represent the taskirrelevant dimension (target force) and solid lines represent the task-relevant dimension (the sum of forces produced by two participants). (b) The cloud of data points is scatted over both the task-relevant and task-irrelevant dimensions. (c) The data points of three pairs extracted from the data of Masumoto and Inui (2013b) were elongated approximately along the taskirrelevant dimension when the target forces were displayed for the pair and a sum of the forces produced by the two participants on a monitor. (d) the data points of three pairs were scatted over both the task-relevant and task-irrelevant dimensions without visual information

were displayed for the pair and a sum of the forces produced by the two participants on a monitor. Analysis of the data indicates significantly more the task-irrelevant variance than the task-relevant one, which can be interpreted as a two-participant coordination stabilizing total force. The right cloud of data points without feedback is oriented orthogonally to the left one. Analysis of the data indicates significantly more the task-relevant variance than the task-irrelevant one, which can be interpreted as no two-participant coordination stabilizing total force. (continued)

References

7

The complementary force production addressed in this book consistently underlies the uncontrolled manifold (UCM) hypothesis (Latash et al. 2002; Scholz and Schöner 1999). Scholz and Schöner (1999) propose the UCM hypothesis to identify error compensation. This hypothesis quantifies variability in the dimension where performance is unaffected by changes in elemental variables (task-irrelevant dimension, Fig. 1.1a) and variability in the dimension where performance is affected by changes in element variables (task-relevant dimension, Fig. 1.1a). For example, in simultaneous force production by unimanual three fingers (Latash et al. 2001, continuous force production), by bimanual one or two fingers (Diedrichsen et al. 2003, continuous force production; Gorniak et al. 2007, continuous force production; Masumoto and Inui 2012, 2013a, periodic force production; Ranganathan and Newell 2008, continuous force production; Hu et al. 2011, continuous force production), or by two people with unimaual or bimanual index finger (Bosga and Meulenbroek 2007, continuous force production; Masumoto and Inui 2013b, 2014a, b, 2015, periodic force production; Solnik et al. 2015, continuous force production), participants produced the target force such that the sum of the force produced by each finger is a target force. These studies find that the nervous system uses a strategy that compensated for task-irrelevant variability in forces produced by individual fingers to reduce the task-relevant variability of the total force produced by two~four fingers. Figure 1.1c and d show raw data extracted from Masumoto and Inui (2013b). They asked participants to produce periodic isometric forces such that the sum of forces they produced was the target force cycling between 5% and 10% maximum voluntary contraction with an interval of 1000 ms. Figure 1.1c shows that the forces produced by three pairs were elongated along the task-irrelevant dimension when the total force was visible, indicating that the correlation between forces produced by the pairs was negative. In contrast, Fig. 1.1d shows that the forces produced by three pairs were scatted over both the task-relevant and taskirrelevant dimensions without visual information, indicating no correlation between forces produced by the pairs.

References Bernstein NA (1967) The co-ordination and regulation of movement. Pergamon Press, London Bosga J, Meulenbroek RG (2007) Joint-action coordination of redundant force contributions in a virtual lifting task. Mot Control 11:235–258 Diedrichsen J, Hazeltine E, Nurss WK, Ivry RB (2003) The role of the corpus callosum in the coupling of bimanual isometric force pulses. J Neurophysiol 90:2409–2418 Dunbar RIM, Schultz S (2007) Evolution in the social brain. Science 317:1344–1347 Eaves D, Hodges NJ, Willams AM (2008) Energetic costs of incidential visual coupling during treadmill running. Med Sci Sports Exerc 40:1506–1514

8

1 Introduction

Gorniak SL, Zatsiorsky VM, Latash ML (2007) Emerging and disappearing synergies in a hierarchically controlled system. Exp Brain Res 183:259–270 Haken H, Kelso JAS, Bunz H (1985) A theoretical model of phase transitions in human hand movements. Biol Cybern 51:347–356 Hove MJ, Risen JL (2009) It’s all in the timing: interpersonal synchrony increases affiliation. Soc Cogn 27:949–961 Hu X, Loncharich M, Newell KM (2011) Visual information interacts with neuromuscular factors in the coordination of bimanual isometric force. Exp Brain Res 209:129–138 Huygens C (1673/1986) The pendulum clock or geometrical demonstrateons concerning the motion of pemdula as applied to clock (trans: Blackwell RJ). Iowa State University Press, Ames Huys R, Daffertshofer A, Beek PJ (2004) Multiple time scales and multiform dynamics in learning to jugle. Mot Control 8:188–212 Kijima A, Kadota K, Yokoyama K, Okumura M, Suzuki H, Schmidt RC, Yamamoto Y (2012) Switching dynamics in an interpersonal competition brings about “deadlock” synchronization of players. PLoS One 7:e47911 Knoblich G, Butterfill S, Sebanz N (2011) Psychological research on joint action: theory and data. In: Ross B (ed) The psychology of learning and motivation, vol 54. Academic, Burlington, pp 59–101 Koriat A (2012) When are two heads better than one and why? Science 336:360–362 Latash ML, Scholz JP, Daion F, Schöner G (2001) Structure of motor variability in marginally redundant multi-finger force production tasks. Exp Brain Res 141:153–165 Latash ML, Scholz JP, Schöner G (2002) Motor control strategies revealed in the structure of motor variability. Exerc Sport Sci Rev 30:26–31 Marsh KL, Richardson MJ, Schmidt RC (2009) Social connection through joint action and interpersonal coordination. Top Cogn Sci 1:320–339 Masumoto J, Inui N (2012) Effects of force levels on error compensation in periodic bimanual isometric force control. J Mot Behav 44:261–266 Masumoto J, Inui N (2013a) Effects of movement duration on error compensation in periodic bimanual isometric force production. Exp Brain Res 227:447–455 Masumoto J, Inui N (2013b) Two heads are better then one: both complementary and synchronous strategies facilitate joint action. J Neurophysiol 109:1307–1314 Masumoto J, Inui N (2014a) A leader-follower relationship in joint action on a discrete force production task. Exp Brain Res 232:3525–3533 Masumoto J, Inui N (2014b) Effects of speech on both complementary and synchronous strategies in joint action. Exp Brain Res 232:2421–2429 Masumoto J, Inui N (2015) Motor control hierarchy in joint action that involves bimanual force production. J Neurophysiol 113:3736–3743 Neda Z, Ravasz E, Brechet T, Vicsek T, Barabast A-L (2000) Self organizing processes: the sound of many hands claping. Nature 403:849–850 Newman-Norlund RD, van Schie HT, van Zuijlen AMJ, Bekkering H (2007) The mirror neuron system is more active during complementary compared with imitative action. Nat Neurosci 10:817–818 Newman-Norlund RD, Bosga J, Meulenbroek RG, Bekkering H (2008) Anatomical substrates of cooperative joint-action in a continuous motor task: virtual lifting and balancing. NeuroImage 41:169–177 Okumura M, Kijima A, Kadota K, Yokoyama K, Suzuki H, Yamamoto Y (2012) A critical interpersonall distance switches between two coordination modes in kendo matches. PLoS One 7:e51877 Oullier O, De Guzman GC, Jantzen KJ, Lagarde J, Kelso JAS (2008) Social coordination dynamics: measuring human bonding. Soc Neurosci 3(2):178–192 Premack D, Woodruff G (1978) Does a chimpanzee have a theory of mind? Behv Brain Sci 1:515–526

References

9

Ranganathan R, Newell KM (2008) Motor synergies: feedback and error compensation within and between trials. Exp Brain Res 186:561–570 Repp BH, Su YH (2013) Sensorimotor synchronization: a review of recent research (2006–2012). Psychol Bull Rev 20(3):403–452 Richardson MJ, Dale R (2005) Looking to understrand: the coupling between speakers’ and listeners’ eye movements and its relationship to discourse comprehendsion. Cogn Sci 29:1046–1060 Richardson MJ, Marsh KL, Isenhower R, Goodman J, Schmidt RC (2007) Rocking together: dynamics of intentional and unintentional interpersonal coordination. Hum Mov Sci 26:867–891 Rizzolatti G, Craighero L (2004) The mirror-neuron system. Annu Rev Neurosci 27:169–192 Schmidt RC, O’Brien B (1997) Evaluating the dynamics of unintended interpersonal coordination. Ecol Psychol 9(3):189–206 Schmidt RC, Richardson MJ (2008) Dynamics of interpersonal coordination. In: Fuchs A, Jirsa VK (eds) Coordination: neural, behavioral and social dynamics. Springer, Berlin, pp 281–308 Schmidt RC, Bienvenu M, Fitzpatrick PA, Amazeen PG (1998) A comparison of intra- and interpersonal interlimb coordination: coordination breakdowns and coupling strength. J Exp Psychol Hum Percept Perfom 24:884–900 Scholz JP, Schöner G (1999) The uncontrolled manifold concept: identifying control variables for a functional tasks. Exp Brain Res 126:289–306 Sebanz N, Bekkering H, Knoblich G (2006) Joint action: bodies and minds moving together. Trends Cogn Sci 10:70–76 Shockley K, Santana MV, Fowler CA (2003) Mutual interpersonal postural constraints are involved in cooperative conversation. J Exp Psychol Hum Percept Perform 29:326–332 Solnik S, Reschechtko S, Wu Y-H, Zatsiorsky VM, Latash ML (2015) Force-stabilizing synergies in motor tasks involving two actors. Exp Brain Res 233:2925–2949 Valdesolo P, Ouyang J, Desteno DA (2010) The rhythm of joint action: synchrony promotes cooperative ability. J Exp Soc Psychol 46:693–695 van Ulzen NR, Lamoth CJ, Daffertshofer A, Semin GR, Beek PJ (2008) Characteristics of instructed and uninstructed interpersonal coordination while walking side-by-side. Neurosci Lett 432:88–93 Vesper C, Richardson J (2014) Strategic communication and behavioral coupling in asymmetric joint action. Exp Brain Res 232:2945–2956 Wiltermuth SS, Heath C (2009) Synchrony and cooperation. Psychol Sci 20:1–5 Yamamoto Y, Yokoyama K (2011) Common and unique network dynamics in football games. PLoS One 6:e29638 Yokoyama K, Yamamoto Y (2011) Three people can synchronize as coupled oscillators during sports activities. PLoS Comput Biol 7:e1002181

Chapter 2

The Background of the Study on Interpersonal Coordination

Abstract This chapter gives an overview of the background of previous studies on interpersonal coordination. While Dunbar’s social brain theory is directly related to human social behaviors, the social behaviors also involve Graziano’s action map view of the primary motor cortex. Prinz’s common coding theory proposes that perception and action share a common cognitive architecture. In other words, action experience changes not only domain-specific behavioral performance, but the neural basis of action observation. Because the representation of the action is also activated by observing action effects, the observation of action facilitates its performance. At the neural level, the mirror neuron system may provide the central nervous system for ideomotor mechanisms, and it is thus a candidate neural system underlying mimicry and imitation. When we observe others performing actions with which we are familiar, we experience increased motor resonance even when we have no intent to act. In addition, the extent to which an individual recruits sensorimotor processes during observation seems to be tightly linked to the individual’s ability to perform the action he is observing. The more familiar the observer is with a given action sequence, the greater the neural response magnitude in premotor and parietal areas seems to be. Representations of shared goals further facilitate interpersonal coordination such as musical ensemble performances in piano duet. In team sports, the intra- and inter-couplings of playing dyads has been proposed as the basis for space- time patterns. Intra-coupling refers to the linkage between two players from the same team and inter-coupling to the linkage between two players from opposing teams. The idea of coupling and layerings (coupling of coupling) predicts on the unifying principle of self-organizing dynamical principles. Keywords Shared representation · Mirror neuron system · Observational learning · Team sports · Musical ensemble

© Springer Nature Singapore Pte Ltd. 2018 N. Inui, Interpersonal Coordination, https://doi.org/10.1007/978-981-13-1765-1_2

11

12

2 The Background of the Study on Interpersonal Coordination

2.1 The Social Brain 2.1.1 Dunbar’s Social Brain Hypothesis The evolution of unusually large brains in primates has long been a puzzle. The brain’s running costs are about 8–10 times as high, per unit mass, as those of skeleton muscle. Harry Jerison (1973) first pointed this out, when he drew a distinction between the component of the brain required to meet the body’s physical needs and the component which could attend to tasks of a more cognitively complex nature. The second component of the brain has been increasing over evolutionary time across the birds and mammals, but fish and reptiles continue to thrive with brains of very modest size. Although it is easy to understand why brains have generally evolved, it is not obviously why the brains of birds and mammals have grown substantially larger than the minimum size required to stay alive. Jerison first plotted brain size against body weight (Dunbar 1996). This gives us a general relationship between brain size and body size that measured the amount of brain tissue required for basic bodily functions. Whatever was left was the spare capacity available for clever things like problem-solving. The distributions showed that whole groups of animals lay on higher levels than others. While the species values for dinosaurs and fishes lay below those for birds, those for mammals lay above the birds. Interestingly, the values for primates lay above those for other mammals, and similar distinctions could be made even within the mammals as a whole. At bottom of the mammal pile came the marsupials (e.g., kangaroons); above them lay the insectivores (e.g., shrews and hedgehogs), then the ungulates (e.g., sheep and deer), followed by the carnivores (e.g., cats and dogs), and finally the primates at the highest level. Even within the primates, prosimians (e.g., the lemurs of Madagascar and the galagos of mainland Africa) seemed to lie on a lower scale, with smaller brains for body size, than the advanced primates (e.g., monkeys and apes). Measured in this way, humans have a brain about nine times larger for body size than is usual in mammals in general. Although human brain is about 1600 cc in volume, a typical mammal of human body weight (55 kg) would have a brain of only 180 cc. This raises a fundamental question: why do some species have bigger brains than others? In particular, why do primates have bigger brains than mammals such as cats and dogs? In addition, why do some primates such as humans and chimpanzees have bigger brains for body size than other primates? Brain is extremely expensive to grow and to maintain. Although human brain accounts for only 2% of body weight, the brain consumes about 20% of all the energy humans take in as food. The fact that an organism has a large brain means that it really must need it very badly, otherwise the forces of natural selection will favor individuals with smaller brains simply because they are cheaper to produce. Conventional explanations for evolution of large brains in primates focused on ecological problem solving. Clutton-Brock and Harvey (1980) found that, among primates, fruit-eating animals like monkeys had larger brain for body size than

2.1 The Social Brain

13

l eaf-eating animals like sheep and cattle. Fruits are very patchy in their distribution, being here today and gone tomorrow. By contrast, there are always leaves in plenty. A fruit-eater thus needs a bigger brain to keep track of where these patchily distributed resources are than a leaf-eater whose foods tend to be more evenly distributed and more widespread. However, while a shift to a more fruit-based diet explains nearly why primates should have larger brains than other mammals, it does not explain why some fruit-eating primates should have bigger brains than others. As an alternative, Byrne and Whiten (1988) proposed the Machiavellian Intelligence hypothesis. Because the term “Machiavellian” was widely interpreted as implying deceit, manipulation, and connivance, this naming does not correspond to the content of their hypothesis. They pointed out that the unusually large brains of primates had something to do with their complex social behavior. Monkeys and apes are able to use sophisticated forms of social knowledge about each other, and they use this knowledge about how others behave to anticipate how they might behave in the future. And then they use these anticipations to structure their relationships. By contrast, other animals lack this capacity and instead make do with simpler rules for organizing their social lives. The larger species tend to be fruit-eating and have larger territories, as well as having bigger brain and living in bigger groups. Presumably, these species have large territories because they are fruit-eaters, and are fruit-eaters because they have large brains, which they need in order to hold large groups together. However, because four variables (brain size, body size, territory size and fruit-eating) were confounded, it was impossible to be sure that the correlation between any two of them was not simply a consequence of the fact that both had been correlated for quite distinct reasons with the third. Some way of testing between the various hypotheses was needed, to see which one correlated best with changes in brain size in primates independently of the others. Related to this testing, one point seems important. All the previous analyses had looked at total brain size. When we look at the history of primate evolution, it is not the whole brain that progressively increases in size from the smallest and most primitive species to the larger and more advanced forms. In order to correct the previous analyses, we have to refer to MacLean’s ‘triune brain’ model (1973). In addition, cognitive psychologist Gardner (1983) has pointed out that the mind does not work like an all-purpose computer that can use any of its bits to do any jobs. Rather, the mind consists of a number of separate modules, each designed to do a particular task. He divides human intelligences into seven modules: linguistic, musical, logic-mathematical, spatial, bodily-kinesthetic, interpersonal, and intrapersonal intelligences. Each of those modules might well be associated with different parts of the brain. According to MacLean’s ‘triune brain’ model (1973), the mammalian brain seems to consist of three main parts: the reptilian brain, old mammalian (paleomammalian) brain, and neocortex. The reptile brain forms the matrix of the upper brainstem and comprises much of the reticular formation, midbrain, and basal ganglia. It programs stereotyped behavior according to instructions based on ancestral learning and memories. In other words, its function involves instinct: the maintenance of an

Fig. 2.1 The relationship between social group size and relative neocortex volume (Dunbar and Schultz 2007, Redraw with permission from Science). In anthropoid primates, mean social group size increases with relative neocortex volume (indexed as the ratio of neocortex volume to the volume of the rest of the brain). Regression lines are reduced major axis fits

2 The Background of the Study on Interpersonal Coordination Monkeys

100 80 60

Apes

40 Mean Group Size

14

20 10 8 6 4 2 1 1

2

3

4

5

6

Neocortex Ratio

individual body and keeping a species. The paleomammalian brain corresponds to the limbic system, and its function is more complex than that of the reptilian brain. The neocortex is the cerebral hallmark of higher mammals and culminates in man to become the brain of reading, writing, and arithmetic. It is a rather thin layer, being a mere five or six nerve-cells deep (2–4 millimeters). Although the neocortex accounts for 30–40% of total brain volume in most mammals, in primates, it varies from a low of 50% among some prosimians to 80% of total brain volume in humans. Thus, we should be looking at the neocortex, not the whole brain. Not only was this the part of the brain that had expended dramatically in primates, but it was also the part where the activities we associated with intelligence (thinking and reasoning) seemed to go on. When Dunbar (1996) looked at the primates, it turned out that there was no correlation between the size of the neocortex of species and any of obvious ecological criteria: things like the percentage of fruit in the diet, the size of the territory, or the distance traveled each day while foraging. To the contrary, primates exhibit a positive correlation between brain size and many indices of social complexity, including social group size (Fig. 2.1), number of females in the group, grooming clique size, the frequency of coalitions, male mating strategies, the prevalence of social play, the frequency of tactical deceit, and the frequency of social learning (Dunbar and Shultz 2007). The measure of neocortex used here is the ratio of neocortex volume to the volume of the rest of the brain. This helps to correct for differences in neocortex size that are simply a consequence of differences in body size although the neocortex has disproportionately expanded in primates. The two axes are plotted on a log scale in order to represent the relationship between the two variables as a straight line. All these relationships held up even when the effects of differences in body size were statistically removed in order to overcome the problem of confounded variables were noted above.

2.1 The Social Brain

15

When Dunbar (1996) first discovered this relationship between group size and neocortex size, he assumed it was something unique to the primates. From personal communication between Dunbar and his colleague Barton (1996), however, Barton discovered that bats which live in stable social groups have larger neocortices than those that live in unstable groups. Dunbar (1996) further began to search for brain- size data on other groups of mammals, and he also found the positive correlation between group size and neocortex size for the carnivores. This finding suggests that there might be an underlying unity to the social lives of all animals. From developmental and comparative psychological points of view, Harrmann et al. (2007) set out to determine whether humans’ bigger brain meant the children were smarter than great ape. The three species were tested on spatial reasoning such as looking for a hidden reward, an ability to discriminate whether quantities were large or small, and an understanding of cause-and-effect relationships. It turned out that the toddlers and the chimpanzees scored almost identically on these tests, although orangutans did not perform quite as well. In contrast to these three tests, there was no contest when it came to social skills. Toddlers bested both chimpanzees and orangutans on tests that examined the ability to communicate, learn from others, and evaluate another being’s perceptions and wishes. Harrmann and colleagues discussed that human children are not born with a higher IQ but rather come equipped with a special set of abilities (cultural intelligence) that prepares them for learning later from parents, teachers and playmates. They further described that social cognitive abilities are the key skills that make us special in comparison to other animals. Furthermore, there is a plausible viewpoint suggested in Fig. 2.1. Dunbar (1996) finds that the ratio of neocortex volume to the volume of the rest of the brain is 4:1 in primates. If we plug this value into the Fig. 2.1, we can read off groups of about 150 as a predicted group size for primates. Our first impression to this size is disbelief because humans live in big cities like Tokyo, New York, and London, places where 10 million or more people live crowded together. However, the villages of modern horticulturelists in Indonesia and the Philippines, as well as those in South America, are also typically around 150 in size. In line with traditional tribal societies, Dunbar (1996) points out that most modern societies also have some kind of grouping of about this size. Although we do not convince whether 150 people in number is an intrinsic group size, it is clearly important for primates to form a group as a life style in order to adapt to social environment. Social groupings larger than 150 people become increasingly hierarchical in structure. With more people to coordinate, hierarchical structures are required. Social groups smaller than 150 people tend to lack structure of any kind, relying instead on personal contacts to oil the wheels of social interaction. Businesses with fewer than 150 people can be organized on entirely informal lines, relying on personal contacts between employees.

16

2 The Background of the Study on Interpersonal Coordination

2.1.2 Brain Regions Specialized for Social Interaction In advance of Dunbar’s social brain hypothesis, Brothers (1990) proposed that there was a set of brain regions that were involved in social cognition. She called this set of regions the social brain and listed amygdala, orbital frontal cortex, and temporal cortex as its major components. The evidence for her proposal came largely from studies of monkeys. In monkeys, lesions to the amygdala result in social isolation, and lesions to orbital frontal cortex also alter social behavior. Neurons in the superior temporal sulcus respond to aspects of faces such as expression and gaze direction. Findings from brain imaging support Brothers’ social brain. Frith (2007) further proposes that there are two major additions to the list of the social brain regions. First, the medial prefrontal cortex has a special role in the second-order representations needed for communicative acts when we have to represent someone else’s representation of our own mental state (Amodio and Frith 2006). Second, a ‘mirror’ system is found in the brain of monkeys and humans to share the experiences of others (Rizzolatti and Craighero 2004). This subsection thus addresses the function concerning the mirror system and the four brain regions: the amygdala, the temporal poles, the posterior superior temporal sulcus and adjacent temporal- parietal junction (TPJ), and the medial prefrontal cortex and the adjacent anterior cingulate cortex (ACC). The social brain allows us to interact with other people. Because we can do much better if we can anticipate what is going to happen next, the better we can anticipate what someone is going to do next, the more successful our interactions with that person will be. Frith (2007) proposes that the functional role of the social brain is to enable us to anticipate unconsciously during social interactions. The most important role of the social brain is that it allows us to anticipate about people’s actions on the basis of their mental states. The assumption that behavior is caused by mental states is called a ‘theory of mind’ (Premack and Woodruff 1978). The automatic process by which we read the mental states of others is called ‘mentalizing’.

2.1.3 Amygdala Early brain imaging studies found the fragmentation of emotion. There is no single brain system involved in emotion. Rather, each emotion has its own specific system. For example, while disgust is associated with activity in the insular (Phillips et al. 1997), fear is associated with activity in the amygdala (Morris et al. 1998). The amygdala thus plays a role on social interaction through its role in recognizing expressions such as fear. The amygdala is also activated by presentation of faces rated as untrustworthy (Winston et al. 2002). This is an example of prejudice because the faces were of people unknown to participants in the experiment. Amygdala activation is

2.1 The Social Brain

17

c onsistently found in association with the unconscious fear that is elicited by viewing the face of someone from another race. For example, Phelps et al. (2000) reported that activity was observed in the amygdala when white Americans were shown the faces of unknown black Americans. The magnitude of the activity in the amygdala correlated with implicit measures of race prejudice. However, Phelps et al. (2003) found that, although amygdala damage did not remove race prejudice, the magnitude of amygdala response did not correlate with explicit measures of race prejudice. The amygdala thus attaches emotional value to face, enabling us to recognize expressions such as fear and trustworthiness. The amygdala response to black faces was reduced when the faces were presented for 525 ms rather than 30 ms and, associated with this reduction, there was increased activity in areas of frontal cortex connected with control and regulation. (Cunningham et al. 2004). Such race prejudice is an example of stereotyping: associating mental attributes with a group of people and then applying this prejudice to individual members of that group. The amygdala is involved in this process owing to its role in fear conditioning. Using animals, Dolan (2002) has showed that the amygdala is part of a system that learns to associate value with stimuli, whether or not these stimuli are social (LeDoux 2000). This system operates on both positive and negative values. For example, the amygdala responds to objects that elicit fear owning to their association with punishment (negative value), but the amygdala also responds to objects associated with food and sex (positive value). In the experiments on race prejudice, the amygdala is responding to black faces in the same way as it responds to any object that has acquired a conditioned fear response (Buchel et al. 1998). The role of the amygdala in recognizing expressions of fear probably has the same origin. A fear expression is a signal (the conditional stimulus) that there is something fearful near at hand (the unconditioned stimulus), so that a fearful face will eventually elicit a fear response. The amygdala is involved in social cognition owning to its role in associating the value (positive or negative) with individual objects and classes of object. This system applies to people just as it does to objects. Our long-term prejudices about individuals and groups are built up through a conditioning process involving the amygdala, but this process is not specifically social.

2.1.4 Temporal Pole We learn facts about specific people: what they look like, where they live, whether they are trustworthy. We also learn facts about social situations: the moment-to- moment changes in behavior appropriate to the situations in which people frequently find themselves and also how feelings and dispositions affect the behavior of people in these situations. Funnell (2001) has reported that damage to the temporal poles impairs the ability to use this knowledge. In line with Funnell’s observation, Damasio et al. (2004) has suggested that the temporal poles are convergence zones, where simpler features from different modalities are brought together to

18

2 The Background of the Study on Interpersonal Coordination

define, by their conjunction, unique individuals and situations. Through this convergence of information, our understanding of an object can be modified by the context in which it appears. These processes instantiated in the temporal poles are important for mentalizing. They allow us to apply our general knowledge about social situations to the situation that currently confront us. They specify the kinds of thoughts and feelings most likely to occur in a particular context: the pride or embarrassment that we have felt or observed in similar situations in the past. However, situations are never exactly repeated. There is much to be learned by observing the moment- moment changes in expression and behavior in the person we are interacting with. This is the role for the brain’s mirror system.

2.1.5 The Brain’s Mirror System Our social brain has two problems to solve. First, it must read the mental state of the person we are interacting with. Second, it must anticipate future behavior on the basis of that mental state. These two problems are not independent. The same mechanism can be used for reading mental states as for reading hidden states of the world outside the social domain. For example, when a person reaches for an object, he has to estimate how heavy it is. On the basis of this estimate, he can control the appropriate muscle force for grasping and anticipate the consequences of his action. If his estimation of the hidden state of the object is wrong, his anticipation will be incorrect. If the object is lighter than he anticipated, his hand will move up faster than he anticipated. This error tells me that the object is lighter. Such an example of anticipatory action provides the basis for reading the hidden intentions of others during action observation. The idea that there is a mirror system in the brain arises from the observation that the same brain areas are activated when we observe another person experiencing an emotion as when we experience the same emotion ourselves (Wicker et al. 2003). The brain’s mirror system is identified by emotion earlier than by action (for a review, Rizzolatti and Craighero 2004). The mirror system also operates for touch and pain. Somatosensory areas are activated when we see someone else being touched (Keysers et al. 2004; Blakemore et al. 2005). Pain areas become active when we see someone receiving a painful stimulus (Morrison et al. 2004) or even when a symbolic cue tells us that someone is receiving pain (Singer et al. 2004). These mirror effects can occur for auditory as well as visual cues (Kohler et al. 2002). The brain’s mirror system is not tied to any particular brain region. The location of the activation depends on what is being observed. There must be some general mechanism by which sensory or symbolic cues can be converted into covert actions. According to Prinz’s ‘common coding principle’ (1997), the brain represents actions in the same way, whether perceiving them or planning them. Such representation does not specify who is performing the action and would be accessed when both perceiving and performing an action.

2.1 The Social Brain

19

Although we do not fully understand the mechanism, we can experience the emotional states of another person. For example, when we see someone smiling, we will automatically imitate that smile and feel happier ourselves. This phenomenon supplies the first step in mentalizing: the initial estimate of the mental state of the person we are interacting with. If we know that someone is afraid, we might expect that they will run, but we cannot expect where they will run unless we know what they are afraid of. Likewise, covertly performing the same movement as another is not sufficient to infer the goals and intentions behind that movement. Mitchell et al. (2005) further point out that, while the mirror system is suited for tracking the continually changing states of emotion and intention of the other, it can tell us nothing about the stable attitudes and predilections of the other.

2.1.6 Posterior Superior Temporal Sulcus How do we know what someone is afraid of? One way to find the cause of their fear is to observe where they are looking. The region of the brain at the posterior superior temporal sulcus (pSTS) and the adjacent temporal-parietal junction (TPJ) is a prime candidate for this process. From an fMRI study, Pelphrey et al. (2005) reported that this region was activated when participants observed someone moving their eyes and this activity was modulated by the context in which the eye movement occurred. For example, Pelphrey et al. (2004a, b) found that more activity was elicited in the pSTS if the actor moved his eyes away from, rather than towards, a flashing target. Presumably, the pSTS is concerned with anticipating the trajectory of movements and that greater activity is associated with anticipatory errors, i.e. when the movement is not anticipated. For example, while Saxe et al. (2004) showed participants a video in which an actor walked across a room, the actor was hidden behind a bookcase on some trials. If the actor paused behind the bookcase, he emerged later than anticipated, and greater activity was observed in the pSTS accordingly. Schultz et al. (2005) have further examined whether this anticipatory system is suitable solely to the anticipation of biological movements. Observing two balls that move in mathematically defined trajectories with no specifically biological appearance elicits activity in the pSTS as long as they appear to be interacting. Kawawaki et al. (2006) have reviewed that there is evidence that the pSTS is involved in anticipating complex movement trajectories of any kind. The trajectory to be anticipated needs to be complex, but not specifically biological to elicit activity in the pSTS. By looking at someone’s eyes, we can find where they are looking, but how do we know what they can see? At the simple level, we know that someone cannot see what we can see, as their line of sight is blocked by an obstacle. At a more complex level, we know that people looking at the same scene from different angles will arrive at different descriptions of the scene. From my point of view, the pole might be in front of the block, while from your point of view, the pole might be to the left of the block. From imaging studies, Zacks et al. (2003) and Aichhorn et al. (2005)

20

2 The Background of the Study on Interpersonal Coordination

observed activity in the TPJ when participants had to describe a scene from another viewer’s perspective. Using a more complex task than tasks employed by Zacks et al. (2003) and Aichhorn et al. (2005), Vogeley et al. (2004) were unable to observe the same activity as found by Zacks, Aichhorn, and colleagues. Knowing where a fearful person is looking and what they can see enable us to know what they looking at, and consequently we identify the cause of their fear. This ability to see the world from another’s perspective enable us to realize that other people can have different knowledge from us and may have false beliefs about the world. From both imaging (Saxe and Kanwisher 2003) and lesion studies (Apperly et al. 2004), there is evidence that the TPJ has a critical and more general role in the performance of tasks that depend on understanding that a person has a false belief about the world.

2.1.7 Medial Prefrontal Cortex Fletcher et al. (1995) and Goel et al. (1995) first found that mentalizing involved activity in the medial prefrontal cortex. Their observations were subsequently confirmed with a wide range of tasks (for a review, Amodio and Frith 2006). However, first, it is not clear whether this region needs to be intact for successful performance of mentalizing tasks. On one hand, several group studies showed that patients with damage to prefrontal cortex performed badly on mentalizing tasks and that this impairment was independent of problems with traditional executive tasks (Rowe et al. 2001; Gregory et al. 2002). On the other hand, Bird et al. (2004) reported that a patient with damage restricted to the medial prefrontal cortex was not impaired on performance of mentalizing tasks. Second, activation of the medial prefrontal cortex is often observed during rest or low demand tasks in comparison to high demand tasks. While activity may be seen in the medial prefrontal cortex when mentalizing is compared with a control task, this is not always the case when mentalizing is contrasted with rest. One plausible explanation for this phenomenon is frequently indulge in mentalizing or thinking about why they volunteered to participate in the experiment or what might be the real motives of the experimenter. Frith (2007) points out which kinds of task activate the medial prefrontal cortex. At least, three categories of task elicit activity in the medial prefrontal cortex: (1) mentalizing task, (2) person perception task, and (3) self-perception task. In mentalizing tasks, participants have to understand the behavior of characters in terms of their mental states. The tasks involve false beliefs and can be presented as stories or cartoons. However, the medial prefrontal cortex is also active when participants engage in real-time social interactions or even when they observe social interactions. These tasks involve anticipating people’s behavior in terms of their current beliefs and intentions. In person perception tasks, participants answer questions about long-term dispositions and attitudes. These can be general (e.g., can people be dependable?) or specific (e.g., is your mother talkative?) and need not apply only to people (e.g., can dogs be dependable?). In self-perception tasks, participants answer

2.1 The Social Brain

21

questions about their own long-term dispositions (e.g., are you talkative?) or about their current feeling (e.g., does this photo make you feel pleasant?). There is little evidence for any systematic differences in the location of the activity associated with these three kinds of task. The activity is diffusively located in the paracingulate cortex from the border of anterior cingulate cortex to medial prefrontal cortex. This region is called anterior rostral medial prefrontal cortex that has been labeled the emotional region of the medial prefrontal cortex and is more anterior and inferior to the region labeled cognitive (Steele and Lawrie 2004). Mitchell et al. (2002) found a different role of feelings and attitudes for the anterior rostral medial prefrontal cortex in comparison to the adjacent regions of medial prefrontal cortex. Participants were told about two target individuals who were described as having liberal or conservative views. They were then asked to anticipate the feelings and attitudes of these two individuals in various situations (e.g., would he enjoy having a roommate from a different country?). The analysis showed a different pattern when thinking about a similar or dissimilar other. Thinking about similar others involved activity in ventral medial prefrontal cortex, while thinking about a dissimilar other was associated with activity in a more dorsal region of the medial prefrontal cortex. Walter et al. (2004) and Grezes et al. (2004a) also observed functional differences between anterior and posterior rostral medial prefrontal cortex. Walter and colleagues asked participants to make inferences about private intentions in contrast to communicative intentions. Thinking about communicative intentions activated a more ventral region (anterior medial prefrontal cortex) than thinking about private intentions. Grezes and colleagues asked participants to infer whether the movements associated with the lifting of a box were intended to be deceptive because the actor was pretending that the box was heavier than it really was. Movements thought to be deceptive activated the ventral medial prefrontal cortex. In addition, Grezes et al. (2004b) asked participants to observe movements included unexpected adjustments because the box being picked up was lighter than the actor expected. Observing these unexpected adjustments was associated with activity in the posterior medial prefrontal cortex. These findings suggest that the anterior medial prefrontal cortex has a special role in handling communicative intentions. This is a more complex process than simply thinking about intentions, because we have to recognize that the communicator is also thinking about our mental state. This involves a second-order representation of mental state. We have to represent the communicator’s representation of our mental state. This is a form of triadic social interaction, such as joint attention. The observation of Mitchell et al. (2005) suggests that, when we think about the mental states of people with similar attitudes to ourselves, we automatically think in terms of our shared view of the word. Previous studies have also examined the anticipation and monitoring processes associated with the selection of action. Walton et al. (2004) observed activity in the posterior medial frontal cortex when participants monitored the outcomes of actions that were self-selected. Knutson et al. (2005) reported that the activity in the posterior medial prefrontal cortex was correlated with trial-by-trial variations in the

22

2 The Background of the Study on Interpersonal Coordination

anticipated probability of monetary gain. Coricelli et al. (2005) also reported that activity in a similar region of the posterior medial prefrontal cortex was associated with regret, i.e. discovering that an unselected action would have led to a better outcome. In addition, Brown and Braver (2005) reported that posterior medial frontal cortex activation was associated with anticipation of the probability of error. These findings suggest that this region is concerned with anticipating the probable value of actions of the self. However, the results of EEG studies show that this region is also involved when we observe the actions of others. A negative event- related potential component arising from the medial prefrontal cortex is seen not only when we make an error, but also when we receive delayed error feedback (Gehring and Willoughby 2002) or observe some else making an error (van Schie et al. 2004). The orbital region is inferior to the anterior medial prefrontal cortex, and the region is concerned with feelings rather than actions, in particular feelings relating to anticipated rewards and punishments. In orbital frontal cortex, while the value of offered goods is represented in monkeys (Padoa-Schioppa and Assad 2006), anticipated regret is represented in humans (Coricelli et al. 2005). This monitoring of feelings seems to apply to others as well as the self. Hynes et al. (2006) asked participants to make inferences about what other people were thinking (cognitive perspective taking) or what they were feeling (emotional perspective taking). Thinking about people’s feelings was associated with activity in the medial orbital cortex, while perspective taking in general was associated with activity in more dorsal regions.

2.2 The Motor Cortex and Its Relation to Social Behavior Since the discovery of the motor cortex by Fritsch and Hitzig (1870), three main views of the primate motor cortex have been proposed over about 150 years. These views are not incompatible, and rather the preceding view is included by the following view (Graziano 2016). In the first homunculus view (Penfield and Boldrey 1937), the motor cortex functions as a rough map of the body’s musculature. In the second population-code view (Georgopoulos et al. 1986), populations of broadly- tuned neurons combine to specify hand direction or some other parameters of movement (i.e., force and speed). In the recently proposed action map view (Graziano et al. 2002), common actions in the movement repertoire are emphasized in different regions of cortex. To understand the action map view, it is necessary to study the structure and complexity of the movement repertoire and understand how that statistical structure is mapped onto the cortical surface. In this subsection, related to joint action, my consideration is restricted to the action map view. In the action map view, the function of the motor cortex is not to decompose movement into constituent muscles and joints (Penfield and Boldrey 1937) or into elemental movement parameters such as direction and speed (Georgopoulos et al. 1986), but instead to help produce some of most complex components of the

2.2 The Motor Cortex and Its Relation to Social Behavior

23

ovement repertoire. Graziano et al. (2002) first pointed toward an action map m applying microstimulation to the motor cortex of monkeys. Although the function of the motor cortex was traditionally studied by stimulating on a short timescale, such as for 50 ms or less, Graziano et al. (2002) involved stimulation for half a second, roughly matching the timescale of a monkey’s reaching movement. Speculatively speaking, this stimulation interval of half a second (0.5 s) is an important significance for the production of motor behavior. In a domain of perceptual-motor skills, to examine an idea that the sequencing of movements represents in the central memory structure as a ‘motor program’, researchers examined serial information processing dependent motor control using a task of tracking a serial light stimulation in 1970s. As a seminal study, Restle and Burnside (1972) asked participants to press six buttons placed beneath a six-light display. Lights came on every 0.7 s. To examine the relationship between motor program and rhythmic time structure, Summers (1975) asked participants to press the corresponding nine response keys to a repeating nine-light pattern. The interval between one response and the next light (inter-stimulus interval) was either 0.5 or 0.1 s, and some rhythmic patterns were made by the combination of two inter-stimulus intervals. Inui et al. (1995) further examined serial information processing dependent motor control in adolescents with mental retardation, Down syndrome, and autism using a task of tracking a serial light stimulation with an inter-stimulus interval of 0.5 s. Although Restle and Burnside used the inter-stimulus interval of 0.7 s, Summers and Inui and colleagues used inter-stimulus interval of 0.5 s, suggesting that half a second may be a basic or fundamental duration of an elemental movement of human serial actions, such as a simple reaching movement. Stimulation of the monkey motor cortex on a behavioral timescale evoked complex movements that resembled components of the animal’s normal repertoire (Graziano et al. 2002, 2005). Different movements were evoked from different sites in an ‘action map’ (see Fig. 2.2). For example, when sites within one region of the map were stimulated, a hand-to-mouth movement was evoked (Graziano et al. 2002, 2005). The movement included a closure of the hand into an apparent grip, a turning of the wrist and forearm to direct the hand toward the mouth, a rotation of the elbow and shoulder bringing the hand through space to the mouth, an opening of the mouth, and a turning of the head to align the front of the mouth with the hand. An occurrence of the movement depended on each stimulation trial and could be replicated even when the monkey was anesthetized. If a lead weight was hung on the hand, the movement compensated and pulled the hand to the correct height to reach the mouth. But the movement was somewhat stereotyped. For example, if a barrier was placed between the hand and the mouth, the hand did not move around the barrier as in normal. Instead, the hand crashed into the barrier and remained pressing against it until the stimulation stopped. Electrical stimulation in the region of the map thus appeared to generate a stereotyped, average version of a common movement. The polysensory zone in the motor cortex contains a high proportion of neurons that respond to tactile, visual, and auditory stimuli (Graziano et al. 1997; Rizzolatti et al. 1981). Each multimodal neuron has a tactile receptive field on the skin and

24

2 The Background of the Study on Interpersonal Coordination

Fig. 2.2 Action zones in the motor cortex of the monkey (Graziano 2016, Redraw with permission from Cambridge University Press). These categories of movement are evoked by electrical stimulation of the cortex on the behaviorally relevant timescale of 0.5 s. Images are traced from video frames and each image shows the final posture obtained at the end of the stimulation-evoked movement. Within each action zone in the motor cortex, movements in similar behavioral categories are evoked

also responds to visual stimuli in the space near the tactile receptive field. Some auditory neurons also have auditory response to sound near the body. Electrical stimulation of these cortical sites evoked a movement that appears to protect the body surface in the area of tactile receptive field (Graziano et al. 2002, 2005). For example, if a site in the cortex responds to touching the left cheek and to visual stimuli near or approaching the left cheek, then stimulation of that site evokes a squint, a folding back of the left ear, a rightward tuning of the head, a lifting of the left shoulder, and a rapid lifting and lateral movement of the left arm as if to block a threat. Another region of the map in the motor cortex, following electrical stimulation, resulted in reaching movements of the arm into distal space with palm facing

2.2 The Motor Cortex and Its Relation to Social Behavior

25

o utward and the hand shaped as if to grasping something (Graziano et al. 2005). To compare the effect of electrical stimulation and the response profiles of neurons, Aflalo and Graziano (2006, 2007) conducted a study in which monkeys was restrained in a chair but free to move its arm spontaneously, grasping, reaching, scratching, and so on. These movements of the arm were tracked in three dimensions at high resolution by a set of lights fixed to key points on the arm. Using regression analysis, each neuron could be matched to a preferred posture of the arm, defined not by hand position in space, but by an 8-domentional joint space. If the arm moved toward that preferred posture, the neuron became more active during that movement. In contract, if the arm moved toward other posture, the neuron was less active. Further studies of cortical action maps have found action categories in distinct cortical zones (Stepniewska et al. 2005), different synergies in different adjacent regions of cortex (Overduin et al. 2012), and social gestures by stimulating the insular cortex of monkeys (Caruana et al. 2011). Van Acker at al. (2013) found that intracortical microstimulation of the monkey motor cortex evokes complex movements of limbs, including hand-to-mouth movements. Desmurget et al. (2013) found that electrical stimulation of the human motor cortex evokes complex behaviorally relevant movements. A similar organization has been also found in the rodent motor cortex. Stimulation of different regions in the rat motor cortex evoked different behaviorally relevant whisking actions, including exploratory whisking from one cortical region and defensive-like whisker retraction and squinting from another cortical region (Haiss and Schwarz 2005). Stimulation of different zones in the rat motor cortex evoked different kinds of forepaw movements (Ramanathan et al. 2006). Lesion of the reaching zone in the rats resulted in the disability to reach. When the recovered rats were mapped again, their cortex showed a new zone, near the lesion site, from which reaching movements were evoked, and the size of the new reaching zone correlated with the extent of the rat’s behavioral recovery. To confirm the effect of electrical stimulation on cortical action maps in the rat motor crtex, Harrison et al. (2012) compared it to the effect of optogenetic stimulation, which is more precise because it specifically induces action potentials in cell bodies in a small target area. They found complex, multi-joint movements of the limbs to specific postures. The more precise optogenetic stimulation was consistent with the results of electrical stimulation at same sites. Bonazzi et al. (2013) systematically mapped the rat motor cortex using long-train electrical stimulation. They found complex, multi-joint movements of limbs that matched the rat’s behavioral repertoire and that were arranged across the cortical surface in an apparent action map. These findings indicate that the motor cortex is organized at least partly as an action map. Although the majority of the findings come from electrical microstimulation studies, the studies are now corroborated by optogenetic stimulation, single neuron physiology, chemical inhibition, and disinhibition, lesion and recovery from lesion, studies of the natural movement repertoire, and computational studies (Graziano 2016). Previous studies showed that, while the motor cortex represents a map of the bodily musculature, neurons in the cortex represent movement parameters. However,

26

2 The Background of the Study on Interpersonal Coordination

recent studies suggest that the motor cortex is organized as an action map. To understand the action map, we have to understand the movement repertoire of animals. The movement repertoire is complex and multidimensional. Actions vary in terms of body parts involved, location in space to which actions are directed, and behavioral significance such as defending the body surface or acquiring objects. Because the multidimensional movement repertoire is squeezed onto the two-dimentional cortical surface, we are yet unable to fully understand the principle of cortical organization. In social implication of an action map in the motor cortex, Graziano (2016) noticed a similarity between standard primate defensive movements and many of the actions in human social communication. In other words, many movement repertoires represented in an action map are contained in the defensive-like movements. The social gestures and expressions such as smiling, laughing and crying appear to evolve from something as specific as defense of the body surface from impending collision. Darwin (1872) and Andrew (1962) early proposed the hypothesis that defensive actions gave rise to many social displays. The action map of the motor cortex has a large zone related to defensive behavior. Neuron in that zone monitor the space around the body using their sensory receptive fields, and monitored space shares a notable resemblance to personal space (Fogassi et al. 1996; Graziano et al. 1994; Rizzolatti et al. 1981). Some other neuroanatomically connected brain regions, such as the ventral intraparietal area, have similar properties and may be part of a larger network that helps to maintain a margin of safety (Graziano and Cooke 2006). This network of brain areas presumably contributes to social behavior. Socially relevant stimuli such as face have an especially strong influence on these neural mechanisms, and the same mechanisms may be involved in judging the margins of safety around other people’s bodies (Brozzoli et al. 2013; Teneggi et al. 2013). Such speculation shows that social behavior, defensive reactions and the action map in motor cortex may overlap in a meaningful way.

2.3 The Social Function of the Mirror Neuron System Mirror neurons were first observed in the premotor cortex of macaque monkeys and fire both when a monkey performs an action and when it observes another monkey performing the same action (Gallese et al. 1996). Thus, neurons that were previously though to be purely ‘motor’ also seem to contribute to action perception and, potentially, action understanding (Buccino et al. 2007; Knoblich and Sebanz 2006). Although the original discovery of mirror neurons occurred in monkeys, the presence of a similar system in humans has been identified by neuroimaging and brain stimulation studies using functional magnetic resonance imaging (fMRI) and transcranial magnetic stimulation (TMS) (Buccino et al. 2007; Iacoboni and Mazziotta 2007).

2.3 The Social Function of the Mirror Neuron System

27

The existence of mirror neurons provides the neural basis of ideomotor theory. The ideomotor theory proposes that actions are centrally represented in terms of the effects they produced (Knuf et al. 2001). Because the representation of the action is also activated by observing action effects, the observation of action facilitates its performance. At the neural level, the mirror neuron system may provide the central nervous system for ideomotor mechanisms, and it is thus a candidate neural system underlying mimicry and imitation (Iacoboni 2008; Prinz 2005).

2.3.1 Mirror Neuron System and Imitation We very often and ambiguously use the term ‘imitation’, and the term is thus used to include several behavioral phenomena that could require different degrees of cognitive processing and intentionality and different corresponding neurophysiological mechanisms. All the phenomena in which an agent exhibits a certain behavior similar in terms of goals or motor appearance to that previously or contingently exhibited by another individual share the capacity of recognizing the observed motor events. This capacity is identified with the property of the observer’s motor system to resonate with that of the observed agent by the activation of the mirror neuron system (Rizzolatti et al. 2002). Such recognition of other’s behavior would render it possible for an observer to replicate the observed patterns of movements or the achievement of the same goals. The involvement of mirror neuron system in imitation is demonstrated by a series of brain imaging studies. Using the functional magnetic resonance imaging, Iacoboni et al. (1999) scanned participants while they lifted a finger in response to the same action presented on a screen (imitation), to a symbolic cue, or to a spatial cue. The activation in the pars opercularis of the left inferior frontal gyrus, the right anterior parietal region, the right parietal operculum, and the right superior temporal sulcus was stronger during imitation during the other motor conditions. Using positron emission tomography, Grezes et al. (1998) also found the importance of Broca’s area when action to be imitated had a specific goal. Using magnetoencephalography, Nishitani and Hari (2000) examined imitation of grasping action, and confirmed Broca’s area for imitation. In addition, Nishitani and Hari (2002) asked participants to observe still pictures of verbal and non-verbal (grimaces) lip forms, to imitate them immediately after having seen them, or to make similar lip forms spontaneously. During lip forms observation, cortical activation progressed from the occipital cortex to the superior temporal region, the inferior parietal lobule, Broca’s area, and finally the primary motor cortex. The activation sequence during imitation of both verbal and non-verbal lip forms was the same as during observation. Moreover, using transcranial magnetic stimulation, a technique that transiently disrupts the functions of the stimulated area, Heiser et al. (2003) confirmed a functional role of the pars opecularis of inferior frontal gyrus in imitation. They used the same task as the study by Iacoboni et al. (1999). Following stimulation of both left and right inferior frontal gyrus, there was significantly impairment in imitation of

28

2 The Background of the Study on Interpersonal Coordination

finger movements although the effect was not observed when finger movements were made in response to spatial cues. These data clearly show that the basic circuit underlying imitation coincides with the circuit activated during action observation, although there are some minor discrepancies among studies. The data indicate that a mapping of observed action on its motor representation take place in the posterior part of inferior frontal gyrus (Broca’s area). In addition, the mirror neuron system in human has not only an important role of imitation but also a direct effect on the peripheral motor system (nerve and muscle). Borroni et al. (2005, 2008) found that the observation of hand simple movements and actions induced specific muscles subliminal activations as measured with H-reflexes and transcranial magnetic stimulation. These activations are modulated subliminally and reproduce the motor commands needed to execute the observed movement.

2.3.2 E ffects of Sensorimotor Experience on the Observation of Others’ Actions Deliberate practice produces physiological changes (Ericsson et al. 1993). The changes occur not only in skeleton muscles but also in the brain. Neural changes underlying performance are thought to separate experts from less skilled individuals. Maguire et al. (2000) have found changes associated with experience in brain structure in the case of London cab drivers. London cab drivers develop extensive knowledge of the intricate metropolitan streets and how to navigate them. The posterior hippocampus, a cortical area important for navigating and recalling complex routs, is enlarged in London cabbies compared with non-drivers. The neural change is the significant positive correlation between the size of a cab driver’s posterior hippocampus and number of years spent behind the wheel. Similarly, Draganski et al. (2004) have found that mastering the art of juggling results in an increase in grey matter density. After several months of practice at juggling, participants showed an increase in grey matter (where cell bodies of neurons are housed) density in areas of the brain involved with motion perception. When the participants stopped their intensive juggling practice, density in those motion perception regions decreased. Such neural plasticity depends on sensorimotor experience immediately before changes in the brain. The plasticity is thus termed ‘use-dependent change’. Practice-related effects on neural plasticity have been also studied extensively in musicians. Pantev et al. (1998) have found that the auditory cortex in highly skilled musicians is enlarged by about 25% compared with people who have never played an instrument. Similar to London cab drivers, enlargement is correlated with the age at which musicians began to practice, indicating that the reorganization of the auditory cortex is use-dependent. Besides the sound-processing part of the brain that changes in musicians, the regions of the brain that control movement and touch also

2.3 The Social Function of the Mirror Neuron System

29

changes as a function of use. Because professional violin players use the fingers of their left hand to play the strings of their violin, their left hand fingertips are stimulated often for several hours a day. Gaser and Schlaug (2003) have found that the left-hand representation in the right hemisphere of the brain is enlarged in many string players compared with people who have never played a stringed instrument. In contrast, the finger representation is not enlarged in the left hemisphere of these violet players because the right-hand moves the bow and there is much less movement and stimulation of the right-hand fingertips. These experience-dependent neural changes not only support the execution of actions, but also play an important role in how expert performers understand and react to the actions of others. In other words, the neural changes result in a high level of both motor and perceptual skills. Experience performing a specific action appears to change the neural activity underlying observation of that action. The works of Calvo-Merino et al. (2005, 2006) are briefly cited from a viewpoint of the relationship between mirror neurons and ideomotor theory here, although their works will be elaborated from a viewpoint of the effects of motor expertise on action perception in sports in Sect. 2.8. Calvo-Merino et al. (2005) recruited participants with a range of experience performing classical ballet and capoeira, a Brazilian art form that combines elements of dance and martial arts. Using an fMRI, Calvo-Merino and colleagues examined differences in neural activation when participants watched actions in which they were skilled compared with actions in which they were not skilled. First, participants passively observed videos of the two movement styles in the scanner. Next, neural activity when participants observed their own dance style was compared with neural activity when they watched the other, unfamiliar dance style (e.g., ballet dancers watching ballet versus ballet dancers watching capoeira). When experts watched only the familiar movement, a network of brain regions thought to support both the observation and production of action elicited greater activation (e.g., bilateral activation in premotor cortex and intraparietal sulcus, right superior parietal lobe and left posterior super temporal sulcus). Calvo-Merino et al. (2006) further examined whether the specific experience of doing the action within a domain was responsible for effects. Because male and female ballet dancers train together, they have extensive experience seeing many gender-specific movements. Thus, ballet dancers represent an ideal group to test the relationship among motor experience, observation, and action perception. The analysis showed greater premotor, parietal, and cerebellar activity when dancers watched movements from their own repertoire than movement performed by the opposite gender. Previous production of specific actions affected the way the dances subsequently perceived those actions, supporting that systems involved in action production can also underlie action perception. In addition to the studies of Calvo-Merino et al. (2005, 2006) with expert dancers, Cross et al. (2006) examined the formation of a motor simulation by training a specific set of movements and testing neural activity in response to watching those movements before and after training. At initial testing, expert dancers’ brain activity was recorded while they watched novel sequences of dance movements. Over the next 5 weeks, these dancers practiced some of the novel sequences of movements

30

2 The Background of the Study on Interpersonal Coordination

but not others. After training, participants showed greater motor resonance when observing trained movement sequences than untrained movement sequences. Neural activity was further correlated with the dancers’ ratings of their own ability to perform the various sequences. These results thus support the idea that our previous action experience impacts our ability to perceive the actions of others through a re- activation of some of the same sensorimotor brain areas we have used to act. Casile and Giese (2006) further provided another example of the impact of doing on the experience-dependence of observational effects. Although human gait patterns are characterized by a phase difference of about 180° between the two opposite arms and the two opposite legs, participants were asked to perform an unusual gait pattern: arm movements that matched a phase difference of 270°. Participants were trained while blindfolded, and given only minimal verbal and haptic feedback from the experimenter. Before and after training, participants conducted a visual discrimination task in which they were presented with consecutive pairs of point- light walkers. The participants were asked to determine whether the gait patterns of the point-light walkers were the same or different. Within each pair, one of the walker’s gait patterns corresponded to a phase difference of 180°, 225°, or 270°. The other point-light walker either matched the first exactly, or moved with a phase difference shifted slightly away from each of the three prototypes. Before training, whereas participants conducted at a high level of accuracy on the 180° discriminations, their discrimination ability was poor for the two unusual gait patterns (225° and 270°). After training, while participants’ performance on the 180° and 225° discrimination remained unchanged, discrimination performance significantly improved for the 270° displays. Furthermore, the more accurately participants had learned to perform the 270° gait pattern during training, the better their post-test performance increase on the 270° discrimination trials. These results thus indicate a direct influence of learning a motor sequence on the ability to perceive slight variations in the sequence. Brief motor training suggests the reorganization of participants’ representation of one specific gait pattern, integrating sensorimotor experience within the representation and visual discrimination performance in the sensorimotor systems of the brain. These findings show that areas of the brain associated with sensorimotor skill execution are recruited when people are observing. When we observe others performing actions with which we are familiar, we experience increased motor resonance even when we have no intent to act. In addition, the extent to which an individual recruits sensorimotor processes during observation seems to be tightly linked to the individual’s ability to perform the action he is observing.

2.3.3 Applying Sensorimotor Experience to the Classroom Such idea that action experience changes not only domain-specific behavioral performance, but the neural basis of action observation, is applied to observational learning in perceptual motor skill learning. Furthermore, the idea suggests that

2.3 The Social Function of the Mirror Neuron System

31

experience acting in the world may facilitate understanding of abstract conceptual information. For example, neural processes which underlie our ability to act also come to subserve our ability to understand using action-related language. For the potential educational value of an embodied cognition framework, Glenberg et al. (2004, 2007) examined language comprehension in young children in both formal and informal learning environment. Participants were divided in action and non-action reading groups. In the action group, each participant read a sentence aloud and then acted out the events of the sentence using objects mentioned in the text. In the non-action group, participants simply read the sentence aloud and then repeated it. On a subsequent comprehension test, the action group exhibited significantly greater understanding and retention than the non-action group. The participants who had acted out the sentences’ events facilitated understanding and retention of the learned information. From an embodied cognition perspective, participants’ experiences acting out the events in each sentence allowed them to call upon a rich set of sensorimotor experiences when they were later being tested for comprehension. This mechanism lies behind enhanced understanding and retention of the material in the action reading group. To examine the transfer of embodied effects to language comprehension, Beilock et al. (2008) recruited expert and novice hockey players for a behavioral and neuroimaging study. In the scanner, participants passively listened to sentences depicting hockey-specific actions (e.g., the hokey players finished the stride) and everyday actions (e.g., the individual pushed the cart). Later, all participants conducted a comprehension test outside the scanner that gauged their understanding of the sentences they had heard. The comprehension task involved the auditory presentation of each sentence followed by the presentation of a picture. Participants were asked to judge whether the actor in the picture had been mentioned in the sentence. In some pictures, the actor performed the action described in the sentence. In some, the actor performed an action not mentioned in the sentence. And, in some, the actor pictured was not mentioned at all in the sentence. If participants comprehend the actions described in the sentences they hear, they should be faster to correctly identify a match between the actor in the picture and the actor mentioned in the sentence when the actor is pictured performing an action that matches the action described in the sentence. Regardless of hockey experience, all participants performed well on the everyday action sentences. However, Beilock et al. (2008) showed that behavioral performance for the hockey-specific action sentences was correlated with hockey experience such that the expert exhibited greater comprehension than novice. Neuroimaging data showed that activity in the left dorsal premotor cortex full account for the relationship between action experience and comprehension. The more extensive an individual’s hockey experience, the greater the neural activation in the left dorsal premotor cortex for hockey-specific language and the better an individual’s hockey specific language comprehension. Therefore, our understanding of language describing action is driven by experience-dependent activation of the left dorsal premotor cortex. This region is thought to support the selection of well- learned action plans and procedural knowledge (O’Shea et al. 2007).

32

2 The Background of the Study on Interpersonal Coordination

On the basis of the finding of Beilock et al. (2008), Kontra et al. (2012) have speculated that specific sensorimotor experience supports learning in the classroom. In a college-level course in physics, for example, introductory physics courses traditionally begin with topics in mechanics, and students encounter the challenging concept of vector quantities such as velocity, force, torque, and angular momentum. Because these quantities exhibit a salient relationship between concept and action, sensorimotor experience has an important role of understanding the quantities in the classroom. Kontra et al. (2012) examined the idea in the laboratory using mechanical physics. Undergraduates who did not learn college-level physics took part in pairs. Each participant was randomly assigned to either the ‘actin’ or the ‘observation’ group. During a 10-minute training session, while the action group felt a series of trials involving two bicycle wheels on an axle, the observation group saw it. The wheels were used to demonstrate various factors that change the angular momentum of a physical system, and thus affect torque when the wheels on an axle is tilted through space. As the action group tilted the axle of the spinning wheels, the group felt a resistance that indicated the magnitude and direction of torque for each trial. The observation group received visual information about the magnitude and direction of torque for each trial via a laser pointer mounted in the axle. Before and after training, participants conducted a computerized torque judgment task to assess comprehension. The participants were asked to compare videos of two avatars, each tilting a set of bicycle wheels on an axle just like the one used during training. Factors such as the moment of inertia, angular speed, direction of spin and rate of tilt were varied from trial to trial. The participants had to determine which of the two avatars was experiencing more torque, or alternatively if the two avatars were experiencing the same amount of torque. Although both groups did not differ in accuracy on the pre-test of the torque judgment task, at post-test the action group showed a 10% improvement in performance above the observation group. This improvement seems to have been driven mainly by increased comprehension of the concept of vector cancellation, or the idea that angular momentum and torque are vector quantities with both magnitude and direction and can thus cancel and sum to zero. The participants of the action group seemed to be integrating the specific sensorimotor experience with the bicycle wheels with the subsequent performance on our physics comprehension task. After 10 min of specific action experience linked to the concepts of torque and angular momentum, participants were better able to make judgments about the magnitude of torques within a physical system. Beyond implication for performance and the observation of other’s actions, these findings support the notion that sensorimotor experience can impact the high level of cognitive activities such as comprehension of action language or physics concepts. In the future study, we need to examine which concepts will benefit most from sensorimotor training, and what aspects of sensorimotor experience are most effective in driving learning. Such findings therefore facilitate an application of embodied cognition to the field of education.

2.4 Imitation, Mimicry, and Its Relation to Social Behavior

33

2.4 Imitation, Mimicry, and Its Relation to Social Behavior The term ‘imitation’ is used when an individual reproduces the action of another (Thorpe 1956). However, there is no simple answer to the question, ‘what is imitation?’ The main reason is that imitation is not a unitary skill. Imitation has been conceptualized as a multifaceted psychological faculty whose function is to adaptively and flexibly copy others’ knowledge and responses. This faculty has two main functions: the ability to copy familiar skills that are recalled from long-term memory and the ability to copy novel skills that may be generated in working memory (Subiaul et al. 2016). While the ability to copy familiar responses represents the primitive state of imitation, the ability to copy novel response represents a derived state of ‘primitive’ imitation. There are specialized imitation mechanisms for copying different types of stimuli: motor (i.e. actions), vocal (sound), and cognitive (unobservable abstract representation) stimuli. On the other hand, the term ‘mimicry’ is used when an individual subconsciously reproduces the mannerisms or gestures of a model (Chartrand and Bargh 1999). Social mimicry is the ubiquitous tendency to copy the bodily movements, expression, postures and speech patterns of an interaction partner (Obhi 2016). Behavioral mimicry includes copying the verbal expressions, accents, and speech tones of interaction partners.

2.4.1 Imitation and Its Relation to Social Behavior Monkeys are able to copy familiar motor rules. For example, Voelkl and Huber (2007) and van de Waal and Whiten (2012) demonstrated that marmosets and vervets use the same opening technique to open a sealed can. Ferrari et al. (2006) also found that macaques are able to copy some familiar facial expressions such as lip-smacking. However, monkeys are unable to imitate novel motor actions although they are capable of novel cognitive imitation or imitation learning (Subiaul et al. 2004). In contrast to monkey, apes are able to copy novel motor actions (Myowa- Yamakoshi and Matsuzawa 1999; Whiten 1998). Novel motor imitation thus represents a highly advanced cognitive skill that may be unique to apes. Novel motor imitation is not only critical for learning and copying social rules but for learning which foods are palatable, perhaps due to dietary pressures favoring tool use in apes and resulting in unique cognitive and neural specializations in the parietal lobe. To the contrary, familiar imitation represents a general yet essential skill for social animals that have to conform to strict social hierarchies and coordinated group activities such as feeding and territory defense. Such pressures common in social animals favored a basic imitation whose primary skill is to copy familiar rules and responses expressed by conspecifics. As the saying goes, “do in Rome as the Romans do.”

34

2 The Background of the Study on Interpersonal Coordination

2.4.2 Mimicry and Its Relation to Social Behavior Since the 1990s social psychologists have studied mimicry intensively, and social neuroscientists have recently began to study this phenomenon. In particular, mechanisms that have been studied in tasks such as action observation and automatic imitation have been assumed to play a role in social mimicry. Related to joint action, my consideration is restricted to mimicry of bodily movements. Mimicry of bodily movements is usually assessed via video recordings of social interactions between two individuals. The mimicry is defined as the repetition, by one member of a dyad, of a bodily movement made by the other member of the dyad. The repeated movement is identical or similar to the original movement. The repeated movement occurs in close temporal proximity to the original movement, and in any case, not longer than 3–5 s afterwards (Chartrand and Lakin 2013). If awareness of mimicry occurs during an interpersonal interaction, this awareness usually arises after the performance of a repeated movement. Thus the mimicry itself occurs automatically and is not consciously initiated. The study of behavioral mimicry is facilitated by the seminal paper published by Chartrand and Bargh (1999). They used the term ‘chameleon effect’ to describe behavioral mimicry. Just as a chameleon changes its color based on its environment, people change their movement based on their current social environment. Chartrand and Bargh (1999) asked participants to engage with a confederate who made distinct target movements. The confederate sometimes touched their own face, whereas they shook their foot at other times. The participants increased face-touching behavior after the confederate touched their own face, and more foot-shaking behavior after the confederate shook their foot. This important finding indicates such mimicry despite the fact that participants and confederates do not know each other, the confederates do not perform any welcoming or affiliate behaviors, and participants are unaware of any behavioral patterns on the part of the confederate. Following the seminal paper, Obhi (2016) recently reviews that many scientists identify many moderates that affect the degree of mimicry displayed. For example, friends tend to mimic each other more than strangers (McIntosh 2006). Participants with a conscious or unconscious affiliation goal mimic more than those with no affiliation goal (Lakin and Chartrand 2003). Individuals primed with interdependent selfconstrual mimic more than those primed with independent self-construal (van Baaren et al. 2003). Further, individuals who are socially excluded from an in-group tend to mimic members of that in-group more after the exclusion experience, thinking to be due to exclusion triggering motivation to re-affiliate (Lakin et al. 2008). High selfmonitors mimic powerful interaction partners more than powerless partners (Cheng and Chartrand 2003). Similarity between interaction partners also increases mimicry whether that similarity refers to opinions (see Obhi 2016), group membership (Bourgeois and Hess 2008), or even a name (Gueguen and Martin 2009). Surprisingly, non-Christian women mimic the face-touching behavior of obviously Christian confederate less than non-Christian confederates (Yabar et al. 2006).

2.4 Imitation, Mimicry, and Its Relation to Social Behavior

35

On the other hand, social mimicry has many consequences for individual information processing. Being mimicked makes people more field dependent, such that they are better at detecting stimuli embedded in a complex background (van Baaren et al. 2003). Being mimicked also encourages more convergent thinking, whereas not being mimicked leads to more divergent thinking (Ashton-James and Chartrand 2009). In addition, feelings of trust increase after being mimicked and lead to greater disclosure of personal information to a stranger (Gueguen et al. 2013). Mimicking an out-group member reduces scores on explicit and implicit measures of prejudice compared to mimicking an in-group member (Inzlicht et al. 2012).

2.4.3 The Social Neuroscience of Mimicry If mimicry involves the production of a identical or similar movement to an interaction partner at very short latency, movement of an interaction partner is first registered by the sensory systems of the observer and the motor systems in the observer’s brain are then activated in such a way as to produce the same or a similar action in short succession. Mimicry fundamentally requires some kind of translation of a sensory event into an identical or nearly identical motor event. Crucial processes are thus involved in action observation, action production and the production of specifically imitative responses. Chartrand and Bargh (1999) proposed the existence of a perception–action link to understand the occurrence of mimicry. As described in Sect. 2.3, the ideomotor theory suggests that observing action effects activates the representation of the action, and then facilitates its performance (Knuf et al. 2001). Ideomotor theory provides a plausible conceptual mechanism of imitation and mimicry. At the neural level, the mirror neuron system may provide the central nervous system for ideomotor mechanisms, and it is thus a candidate neural system underlying mimicry and imitation (Iacoboni 2008; Prinz 2005). TMS is an electromagnetic tool that can non-invasively stimulate neurons in cortical motor areas (Hallett 2007). When body part representations in the motor cortex are stimulated by TMS, a muscle response measured with electromyography can be recorded in the corresponding muscle of the body in the form of a motor-evoked potential (MEP). MEP amplitudes find to increase when someone watches another person performing an action, and this increase is specific to the body part representations in motor cortex corresponding to the observed actions (Fadiga et al. 1995; Naish et al. 2014). MEPs thus reflect the excitation of motor representations in an observer’s brain, providing an indicator of the degree of ‘motor resonance’, and by inference the level of activity in the mirror neurons. However, Heyes (2011) points out that the links between the mirror neuron system and mimicry or imitation have not been empirically demonstrated.

36

2 The Background of the Study on Interpersonal Coordination

2.4.4 I nteraction Between the Social Psychology and Cognitive Neuroscience There is a technical limit for studying the neural basis of natural social mimicry. Researchers interested in the neural mechanisms underlying mimicry are mostly unable to use fMRI due to the difficulty of participating in a social interaction in a constrained environment, and due to the signal corruption that occurs because of excessive movement. They have thus used other approaches. One way to address the issue of whether processes involved in automatic imitation and action observation tasks are also involved in social mimicry, is to determine whether known moderators of mimicry also affect dependent variables measured in these tasks. This ‘common-moderators’ approach has been taken in a few studies. For example, Obhi et al. (2011) employed an action observation task in which participants viewed a model’s hand squeezing a rubber ball between the index finger and thumb. Under the interdependent condition, the video had an interdependence prime word superimposed over them such as ‘together’ whereas under the independent condition prime words like ‘individual’ that referred to independence were used. In the control condition, no word was superimposed, and participants simply saw a fixation cross. Under each condition, TMS-evoked MEPs were elicited to assess the degree of motor cortical excitability. Motor cortical excitability was higher in the interdependent condition than in both the control and independent conditions. In the field of social psychology, van Baaren et al. (2003) found that individuals primed with interdependence exhibited more behavioral mimicry than individuals who were not primed or who were primed with independence. A specific moderator variable thus affects social mimicry and motor resonance in a parallel fashion. This result demonstrates that motor resonance as indexed by MEPs underlies natural social mimicry, suggesting that mechanisms underlying social mimicry and action observation are at least partially common. In line with Obhi et al. (2011), in social psychology domain, Cheng and Chartrand (2003) have previously shown that powerful individuals do not mimic the less powerful, whereas powerless individuals do mimic the more powerful. On the basis of this result, Hogeveen et al. (2014) examined the effect of power on motor resonance in an action observation task. Hogeveen and colleagues first primed individuals to feel powerful or powerless by asking them to recall an episode from their own lives where they had power over others or where others had power over them, and then engaged these participants in an action observation task. A third group who were simply asked to write about what they did yesterday was also run through the action observation task. Motor cortical excitability was lowest for the powerful group and highest for the powerless group, consistent with the findings of Cheng and Chartrand (2003) for the effects of power on social mimicry. Using video-based paradigms to address important questions about the factors that drive the rapport-building and liking effects of mimicry, Catmur and Heyes (2013) asked whether it is the maching of motor actions or simply the contingency between the ‘mimcker’s’ action and the other person’s action that are important for

2.4 Imitation, Mimicry, and Its Relation to Social Behavior

37

pro-social outcomes. Catmur and Heyes point out that describing mimicry as an imitative phenomenon emphasizes that reproduction of the original action is key, finding that perceiving an action in response to an action was associated with enjoyment of the task and feeling of closeness to others, irrespective of whether the perceived action was motorically identical or not. On the other hand, Sparenberg et al. (2012) found that mere effector matching between a participant’s movement and observed movements of on-screen avatar was enough for participants to indicate high pleasantness ratings for the avatar. If this result is generalized to real-life social interactions, it suggests that mimicry- induced liking may be due to simply observing an interaction partner moving the same effector, and that imitation of the type of movement may not be important. To address the potential neural correlates of the positive effects of mimicry, Kuhn et al. (2010) asked participants lying in an fMRI scanner to watch an interaction between two individuals under some conditions. In one condition, one member of the dyad mimicked the other, and in a second condition, they did not mimic the other. Crucially, participants see the interaction from over the shoulder camera shot of one of the interaction partners. Reward- and emotion-related brain regions such as the medial orbitofrontal cortex and ventral prefrontal cortex were more active in the mimicry condition. In addition, the increase in activity in these two areas in mimicry versus non-mimicry conditions was positively correlated with ratings of closeness to the interaction partner. Being mimicked seems to be more intrinsically rewarding than not being mimicked, suggesting a first step in understanding the reasons why mimicry might be so ubiquitous. All of the above-mentioned studies employed video-based paradigms, and then offer important clues as to which aspects of mimicry drive its consequences and how the activity of the brain’s reward system is involved. In future study, researchers on the neural mechanisms underlying mimicry will combine these approaches to manipulate specific aspects of mimicry like motor similarity, effector similarity or temporal contingency within an fMRI environment to ascertain which aspects of mimicry result in the highest levels of reward-related activity. In sum, these studies indicate that moderators that affect mimicry affect dependent measures of assumed component processes like motor resonance. This suggests that the degree of imitative matching may not be at the level of exact movements, but rather effectors or movement goals. Using the automatic imitation paradigm, Leighton and Heyes (2010) have shown that imitation is not wholly effector dependent and that movement types are imitated even by a different effector, although to a lesser degree than when the same effector is used. A detail analysis of the frequency of effector matching, movement matching and timing in social mimicry occurs are useful in guiding thinking about underlying mechanisms. On the other hand, to address the issue of common mechanisms in tasks such as action observation and social mimicry on the other, it is necessary to engage the same participants in both tasks within a single experiment. Hogeveen and Obhi (2012) examined whether engaging in social interaction had any effect on the degree of motor resonance occurring in a subsequent action observation task. They asked participants to take part in a social interaction with a confederate after the participants

38

2 The Background of the Study on Interpersonal Coordination

engaged in an action observation task in which MFPs were recorded. Another group of participants did not engage in a social interaction before they did the action observation task. The participants who had engaged in the social interaction showed significantly more motor resonance than those who had not engaged in the social interaction. In addition, the participants who had shown more mimicry in social interaction showed more motor resonance than those who had showed less mimicry in the social interaction. This study first examined social mimicry and motor resonance during action observation in the same participants, suggesting that there is at least some common resonance involved. From unpublished data, however, Obhi (2016) points out that, whereas prior mimicry induces more motor resonance in an action observation task, prior participation in an action observation task does not induce more mimicry. Using mu-suppression, an index of mirror activity in conjunction with a mimicry manipulation (Pineda 2005), Hogeveen et al. (2014) recently assessed mu- suppression over left sensorimotor cortices during an action observation task, before and after engaging in asocial interaction with a confederate, or with a computer. In social interaction, some participants were further mimicked by the confederate whereas others were not. After the intervening social interaction, only the participants who were mimicked by the confederate showed more mu-suppression than in the pre-test. Interestingly, the participants who engaged with a computer showed less mu-suppression in the subsequent action observation task, suggesting that mirroring is dependent on the recent experience of the individual (Press et al. 2007). Using videos of a model performing subtle face-touches, van Ulzen et al. (2013) also have shown that observers exhibit motor resonance while engaged in a series of clerical tasks. Furthermore, researchers are selectively interfering with the normal processing of a node in the processing chain underlying a task and determining the effect on some performance measure. Such interference results in reducing excitation in brain area using low-frequency TMS approaches (van Honk et al. 2002; Obhi et al. 2002), or increasing excitability using anodal transcranial direct current stimulation (tDCS). Hogeveen et al. (2014) delivered anodal tDCS to the right inferior frontal cortex, which was implicated in mirroring by previous studies (Heiser et al. 2003; Iacoboni et al. (1999). They found that increasing excitability of this area resulted in a subsequent increase in social mimicry in asocial interaction with a confederate, suggesting a potential link between the human mirror neuron system and the manifestation of social mimicry. In this subsection, I address the potential mechanisms that underlie social mimicry from an interaction or integration of social neuroscience and social psychology approaches. However, many questions remain as follows: questions about how other signals interact with mirror mechanisms and the motor system to modulate the degree and type of mimicry exhibited in any given situation. Kuhn et al. (2010) found that being mimicked is rewarding. Kouzakova et al. (2010) further showed that a lack of mimicry is associated with increased levels of salivary cortisol, suggesting that not being mimicked raises stress levels and is perhaps akin to being

2.4 Imitation, Mimicry, and Its Relation to Social Behavior

39

socially excluded. Related to these results, future study needs to examine the rewarding and stressful effects of mimicry or lack of mimicry. In addition, Bardi and Brass (2016) try to identify self-other control mechanisms, and the neural correlates of these mechanisms in the context of imitation control and using the automatic imitation task. Their studies found a potential limitation control network with key nodes in the anterior fronto-median cortex and the temporoparietal junction (rTPJ). Using an fMRI, Qin and Northoff (2011) support the idea that rTPJ may play a key role in the control of imitation and medial prefrontal cortex is a well-known neural correlate of self-related processing. An important feature of social mimicry is that it fluctuates within and across social interaction for a given individual, suggesting that mimicry is unconsciously regulated. To examine this feature of social mimicry, Dalton et al. (2010) proposes that mimicry is schema driven, and that such schemas contain information about the degree of mimicry that can be across different social scenarios. The fact that social mimicry is regulated means that some actions are mimicked while others are not. Mimicry is thought as a decision or selection problem for brain. In other words, mimicry is selected by action to produce in response to social input from a partner. Such action selection is presumably achieved via anticipated reward values of all current motor possibilities. Action selection based on anticipated reward is proposed to involve the orbitofrontal cortex that is active when observing mimicry (Kuhn et al. 2010). In addition, what is the role of sensorimotor anticipation in mimicry? Presumably, internal forward modes are involved in anticipating the sensory consequences of pending actions, and these anticipations might include the expected reactions of an interaction partner (Wolpert and Flanagan 2001). In addition, real-time changes in goals perhaps exert an effect on action selection. For example, new information about a social partner that arises during an interaction could change the goal state from an original ‘affiliation goal’ to a ‘disaffiliation goal’. Then, the change in goal state should have an effect on activated motor possibilities. Thus, the brain may judge a non-mimicked action as one associated with the highest reward, and then making it less likely that a matching action is emitted. To date, we have not been conducted experiments examining these kinds of ideas in social contexts. Finally, Obhi’s review (2016) points out that there are other issues that should be addressed to understand what aspects of an action are mimicked in social scenarios. For example, is mimicry related to the end-point of movements, the movements themselves, or the effector used? What are the conditions that might change what aspects of an action are mimicked? Is the temporal relationship between actions and mimicry of the actions important? What are similarities and differences between social mimicry in natural interactions and automatic imitation in laboratory tasks, and what patterns of brain activity are specific to both? Therefore, there are many remaining questions surrounding how social mimicry arises for social neuroscientists and psychologists.

40

2 The Background of the Study on Interpersonal Coordination

2.5 Joint Perception As Sartre (1958) described in the famous book ‘being and nothingness’, how we perceive the word and what it has in store changes as soon as the other enters the scene. A review of Zimmermann and Richardson (2016) show that something of Sartre’s phenomenology can be experimentally measured in eye movements. For example, Richardson et al. (2012) found that the knowledge that somebody else is looking at the same stimulus as themselves changes participants’ eye movements and focus of attention. Richardson and colleagues developed the joint perception paradigm to examine the effect of a minimal social context. While participants’ gaze was tracked, they looked at images in the belief that they were looking alone, or with the belief that another person was currently looking at same stimuli. In the joint perception paradigm, participants did not see each other or interact in any stimuli. In contrast to the joint perception paradigm, two people can see each other or interact in some way in studies on gaze coordination, gaze following and the effect of eye contact. When researchers think about perception, they have traditionally examined it in individual terms. They thus studied perceiving something as a very private and subjective experience. In other words, they have predominantly studied individual cognition and perception in isolation and removed from any social cues. However, human cognition in real world often takes place in a social context. In line with a social context of human cognition, in recent decades, many researchers have adopted the view that cognition is socially situated and needs to be studied accordingly. Biological, cognitive and social scientists have shown that the brain is sensitive to social information. Cacioppo and Cacioppo (2013) point out that the social aspects of human life do not only manifest themselves in higher-order processes such as language, reasoning or decision making, but are at work in neural, hormonal, cellular and genetic mechanisms. Senju and Johnson (2009) have reported that, from the day we are born, we attend to social cues in our environment. Because we are restricted in our possibilities to experience the world when we are young, we make use of the people around us to learn and pay attention to what might be relevant (Striano and Reid 2006). Infants are fascinated by eyes and very quickly comes to find that the two eyeballs can be used to access information, to learn the name for objects in the world and to understand other minds (Frischen et al. 2007). Striano and Reid (2006) have reported that, by following the eye gaze of caregivers, infants direct their attention to objects in the environment to learn about the physical world. Simultaneously, infants develop social cognition through the mechanisms of joint attention. In adults, we continue to be highly sensitive to the gaze of others. Direct eye contact with another is means to signal attention (Mason et al. 2005), establish dominance (Dovidio and Ellyson 1982) and to cue an observer’s attention (Kuhn et al. 2009). Frischen et al. (2007) have also reported that, when we see other people looking in a particular direction, this serves as a powerful cue to direct our own visual attention. In conventional gaze-cueing experiments, participants are looking

2.5 Joint Perception

41

at the face of another person. Although face-to-face contact is the most immediate and powerful way to experience social context, it is not the only way that it can arise. Experiments that show participants the face of another person conflate the direct response to a social stimulus with the effect of the presence of another.

2.5.1 Perception in a Social Context Eyes are inherently social. Before we can engage in any form of joint action, we use our eyes and eyes of others for communication, for receiving information, and for expressing social affinities (Tomasello et al. 2005). The human visual system can detect information from only a small region of the visual field, about 2° (Levi et al. 1985). Using not only their own visual system but also the eyes of those around them, humans are able to exploit each other’s visual systems by being attentive to where conspecifics are looking. The white of the sclera in social mammals has evolved so that our gaze direction can be easily perceived by conspecifics (Kobayashi and Kohshima 1997). Social attention underlies the exploitation of another’s visual system, and it enables humans to learn from others about the environment (Richardson and Gobel 2015). There are many studies about how our visual perception is shaped by the presence of other people. As a classical study, Sherif (1937) used a phenomenon called the autokinetic effect to examine how people use other people as a source of information when they make judgments under ambiguous circumstance. Individual participants were asked to seat in a dark room and they focused on a dot of light 15 feet away. They then reported how much they thought the light moved in inches. Although the light was not moving at all, all participants were able to arrive at stable estimate over time. Next, participants were divided into three groups and told to say their judgment out loud. Although the autokinetic effect was experienced differently by every person, people reached a common estimate and every member of the small group conformed to that estimate. These results do not show that the individuals’ actual perception of the seeing moving dot of light was changed through the presence of others, but they suggest that the interpretation of the incoming sensory information was altered. In the past decade, a number of studies have reported that the mere presence of another person changes an individual’s perception of the physical world. For example, Schnall et al. (2008) have showed that the geographical slant of a hill is perceived as less steep whenever a supportive other is imagined or actually present, suggesting that social support lightens an individual’s load when facing a physical challenge. Doerrfeld et al. (2012) also asked participants to judge the weight of the boxes while expecting to either lift the box alone, together with a healthy co-actor or together with an injured co-actor. The participants perceived the box as lighter when they intended to lift it up with a non-injured co-actor. Thus, while actual physical properties of objects are taken into account, human perception is very susceptible to social factors.

42

2 The Background of the Study on Interpersonal Coordination

Moreover, the presence of others influences not only how we interpret the world, but also where we look. Milgram et al. (1969) early asked groups of people of different sizes to stop and stare into a building window on a crowed street in New York, finding that, with an increase in the size of the stimulus group, the gaze-following responses of passersby increased. Gaze direction seems to be subject to social influence and can be contagions. Gallup et al. (2012a, b) have recently reported the dynamic of visual attention and information transfer between people in crowed environments. The movements and gaze-following behaviors of about 3,000 pedestrians were quantified in a shopping street in Oxford in the United Kingdom. The mixed-gender stimulus groups varied in size (1, 3, 5, 7, 9, 12 or 15) and were looking up at the camera, which was filming the area, for 60 s without moving (Gallup et al. 2012b). Thirty percent of the passersby adopted the gaze direction of the stimulus group and 14.2% even stopped walking to lookup. The proportion of pedestrians copying the gaze direction and the proportion of the individual time spent looking both increased as a function of stimulus group size. Individuals behind and to the side of the group were more likely to follow gaze than individuals who were in front of and within the gaze of the group. Thus, gaze-following also depended on the spatial location of the pedestrians in relation to the stimulus group. Gallup et al. (2012a) further found the directional flow of visual information transfer in a natural environment. An attractive stimulus with a hidden camera inside was placed in a trafficked corridor at a university. No confederates were sent out to increase gaze following, but the visual orientation dependence of the passersby was recorded. While the baseline rate of gazing at the stimulus was 28.4%, the rate significantly increased to 49.4% whenever another pedestrian had looked at the stimulus within the previous 3 s. Individuals were more responsive to changes in the visual orientation of others walking in front of them (57.1%), while attention towards the stimulus even diminished (to 19.8%) in comparison to the baseline rate, when oncoming pedestrians had looked at the stimulus. Human beings thus monitor changes in visual orientation from behind and align with it frequently to be able to relate to cues in an environment that they will shortly experience themselves. We are accustomed to looking at the same things simultaneously when we watch TV, go to a museum, look at lecture slide, posters or shop windows. While we talk about the things we observe, the degree of gaze coordination between a speaker and listener anticipates the success of their communication (Richardson and Dale 2005). People engaged in a conversation further coordinate their gaze to accommodate difference in the knowledge and the visual context that they share (Richardson et al. 2007). These studies suggest that perception in a social context is fundamentally different from perception that takes place under solitary circumstances.

2.5.2 Joint Perception We have seen that social context can influence how we move our bodies and how we interpret the world around us. It is not clear, however, exactly what constitutes a social context. To examine these questions, Richardson et al. (2012) asked pairs of

2.5 Joint Perception

43

Fig. 2.3 Total looking time when looking at a picture with a positive or negative valence in solo and joint conditions. (Richardson et al. (2012), redraw from von Zimmermann and Richardson (2016) with permission from Cambridge University Press)

participants to sit back to back in opposite corners of a laboratory room. Their gaze tracked sets of four pictures presented on screen for a short time. On each trial, two of the pictures were neutral fillers, one picture had a positive valence and another one had a negative valence, as rated by the international affective picture system norms (Lang et al. 2005). Before the trials in the joint condition, participants heard the experimenter’s voice say, “You will both be looking at the same set of pictures.” In the alone condition, they heard, “You will be looking at pictures. Your partner will be looking at symbols.” There were also filler trials in which the participants were looking at symbols while they were told that their interaction partner was looking at pictures. The conditions were randomized within participants. In Experiment I, when participants believed that their partner was also looking at the pictures, they looked significantly more at the negative images than at the positive image (Fig. 2.3). Although participants could not see or interact with each other and had no knowledge of each other’s attentional focus, their eye movements were systematically shifted over trials. The mere belief that another person is looking at the same images at a given moment changes an individual’s gaze patterns. In Experiment II, participants in the joint condition performed better on a memory test for the negative images, finding that when they look at stimuli together those that are shared are made more ‘psychological salient’ (Shteynberg 2010), and thus more memorable. From previous studies on joint action, we know that people only form shared representations and joint action effects only occur when the interaction partner is engaging in the same task (Atmaca et al. 2011). Consistent with such shared intentionality, it was predicted that joint perception effects would be stronger when participants believed that they were acting jointly, rather than passively sharing an experience (Richardson et al. 2012). To test this in Experiment III, participants received instructions for two tasks. In a memory task, the participants were asked to remember the pictures viewed for a later test, which did not actually take place. In a search task, the participants were asked to find a translucent X superimposed on one of the images and press a mouse button upon detection. The participants were told that their partner would always look at the same pictures, but that their individual tasks could change from trial to trial. At the beginning of every trial, the participants were told what their task would be and a large icon appeared at the top

44

2 The Background of the Study on Interpersonal Coordination

Fig. 2.4 Total looking time in both memory and search tasks. (Richardson et al. (2012), redraw from von Zimmermann and Richardson (2016) with permission from Cambridge University Press)

of the screen, representing the task. In Fig. 2.4, a small icon below showed their partner’s task. In additional to the visual information, the participants heard a voice telling them to search/memorize and what their partner was supposed to do. As predicted, participants only showed a robust preference for negative images when believed that they and their partner had been assigned to do the same task. These results indicate that the effects of joint perception can only be observed when people believe that they are engaged in the same task. Whenever participants are made aware of the fact that they are doing something different to their partner, the phenomenon of joint perception diminishes or disappears. Like joint action, joint perception requires people to believe that they are interacting with an individual who shares similar or the same intentions in a mutually engaging task or environment.

2.5.3 W e Transcend Our Private Worlds by Responding to the Same Stimulus In joint perception, Richardson et al. (2012) found that people spent more time looking at the image with negative valence than at those with positive or neutral valence. Why do people look at images of a negative valence in joint perception? One plausible explanation is that, when people collaborate in groups, they tend to align with the group emotion and attitudes (Barsade 2002, Hatfield et al. 1993). Human beings have developed a learnt or evolved priority to detect threats in the

2.6 Observational Motor Learning

45

environment, which results in a general bias for negative or bad information and stimuli (Rozin and Royzman 2001). In joint perception, gaze patterns and focus of attention changes as soon as people think that they are looking together. The gaze- coupling is observed between people taking part in the joint perception condition. Another possible explanation is that the joint perception effect is driven by salience. The term ‘salience’ is usually used to denote the remarkable ability of people to resolve ambiguous reference during conversation and social interactions. Our behavior depends on our knowledge about the context and assumptions that we have in common to coordinate our behaviors effectively and to avoid social disorientation. In the absence of any actual communication between people, however, some of the mechanisms of coordination may have still been tuned on by the simple knowledge that images were viewed together (Richardson et al. 2012). When people thought that they were looking at the same images, they may have paid more attention to those they thought would be more salient to their partners. Because salience is driven by the valence of the pictures used in the experiment of Richardson et al. (2012), paying more attention to the most salient pictures meant to pay more attention to the negative pictures. Although the finding people look at images of a negative valence in joint perception sounds novel, it is not surprising that the phenomenon takes place when they think that they are looking at the same images. As joint perception effect takes place when they think that they are looking at the same images, the effect depends on shared representations. The review of Zimmermann and Richardson (2016) points out that many researchers have examined shared representations under many concepts as flows: empathy and mood contagion, perspective taking, theory of mind, embodied synchrony and mirroring, common ground and interactive alignment in conversation, socially distributed knowledge, and shared cognition in groups. Joint action perception seems to be an additional mechanism that provides a basis for and serves the emergence of the shared representation. People seem to transcend their private worlds by responding to the same stimulus in such a way that the foundation for intersubjectivity and common ground is laid (For a review, Thompson and Fine 1999). However, this does not happen under all circumstances, but only whenever there is the prospect of collaborative interaction and the pursuit of a shared goal. Shared intentionality seems to be a blinding force for joint perception because the phenomenon of joint perception is stronger when people are engaging in the same task at the same time (Richardson et al. 2012). The effects diminish or disappear significantly when people are doing a different task.

2.6 Observational Motor Learning Our capacity to move accurately in external environment lies in the brain’s ability to flexibly modify our motor behavior. For example, holding a heavy or light object in the hand changes the arm’s dynamic environment. In order to skillfully manipulate the object, the brain has to change motor commands to compensate for the object’s

46

2 The Background of the Study on Interpersonal Coordination

weight and achieve a desired movement. Subsequent movements improve performance because the brain formulates the relationship between repetitive motor commands and sensory information produced by movements. Consequently, perceptual-motor representation or schema is formed in the brain. This process is called motor learning. While we acquired many motor skills through active physical practice, we can also learn how to make movements by observing others. This is referred to as observational motor learning.

2.6.1 Sensorimotor Adaptation Although there are many categories of motor learning, this section addresses sensorimotor adaptation (for a review, McGregor and Gribble 2016). In a typical experiment of sensorimotor adaptation, participants are provided with altered sensory inputs or motor outputs and have to modify their movements in order to regain a normal level of performance. Even before moving, the brain anticipates how the intended movement should look and feel. If sensory inputs or motor outputs are altered, the actual sensory outcomes of a movement do not match the brain’s anticipation. Sensory feedback informs the brain of how the executed movement differs from the intended movement and guides the brain’s modification of motor commands during the planning of subsequent movements (Shadmehr et al. 2010). In a visuomotor adaptation task, participants are provided with altered visual feedback such that there is a discrepancy between the actual trajectory of the hand and the visual consequence of movement. For example, visual feedback can be shifted laterally using prism lenses (Martin et al. 1996), inverted using mirrors (Imamura et al. 1996), and rotated in a virtual environment (Krakauer et al. 2005). If we first make the novel visuomotor movements, executed movements do not result in the anticipated visual outcome. Based on the anticipatory errors, the brain implicitly updates its motor anticipations to account for the visual perturbation. Once adapted, participants aim their movements in the opposite direction of the visual perturbation and successfully complete the task. Krakauer et al. (2005) instructed participants to guide an on-screen cursor to the target. Under the baseline conditions, the cursor corresponds to the actual position of the hand and executed movements result in the anticipated visual outcome. Visual feedback is then altered such that the cursor’s position is rotated 30° counterclockwise to the hand’s trajectory, and moving the hand to the target now results in visual error. As practice progresses, the brain incorporates the visual rotation into its anticipations and movements are aimed 30° clockwise to the target so that the cursor hits the target. Executed movements again match the anticipated visual outcome as a result. This result suggests that, if there is a mismatch between the anticipated and actual visual feedback of a movement, the brain will adapt its planning of movement direction in order to bring the motor anticipation and actual visual outcomes back into alignment.

2.6 Observational Motor Learning

47

In a task of fingertip force adaptation (for reviews, Johansson 1996; Flanagan 1996), participants learn to adjust their force output when lifting objects of varying weights. Johansson and Westling (1984, 1988) have shown that, before lifting an object, the participants anticipate the object’s weight based on its visual characteristics such as its size or material. This anticipation influences their lifting behavior in terms of the force applied by their fingertips and the force with which they lift the object (Gordon et al. 1991). For example, individuals tend to grip and lift objects with slightly more force than is necessary to keep an object from slipping. Participants also apply vertical load force that slightly exceeds the weight of the object in order to achieve lift off and a smooth lift to the intended height. Immediately before lifting the object, participants increase their grip force and load force in parallel according to the anticipated weight of the object. If the anticipation is inaccurate, participants will generate inappropriate forces and the object may be not lifted smoothly. In such cases, grip force and lift force are quickly adjusted based on sensory feedback (Johansson and Westling 1984). The brain’s motor anticipations are rapidly updated, fully adapting force to the object’s actual weight in as little as one trial (Johansson and Westling 1988). Flanagan et al. (2003) instructed participants to grip an object with novel dynamic properties and move it along a straight path. The participants rapidly learned to scale their grip and load forces well before they were able to accurately move the object along the desired path, again emphasizing the role of anticipation in learning. These studies on fingertip force adaptation indicate that the brain anticipates the force requirements for lifting objects and, if the anticipation is inaccurate, more commands for force production are rapidly modified such that the object can be lifted in the intended manner. Flanagan and Beltzner (2000) asked participants to grip the object-mounted handle between the thumb and index finger. The object handle contains force sensors to measure grip force and load force. In a version of the object lifting task, participants are instructed to lift large and small cubes that have identical weights. Based on its size, participants anticipate that the smaller cube will be lighter. This anticipation is incorrect and, when lifting the smaller object, the applied load force is insufficient to lift the object at the anticipated time. Based on this mismatch between anticipated and actual sensory feedback, the motor system increases its load force output until the object is lifted as intended. Knowledge of the smaller cube’s weight is used to update the motor anticipation for lifting that object in the future. Furthermore, in a task of force field adaptation which participants adapt their reaching movements to a novel force environment, they grasp the handle at the end of robotic arm and are instructed to perform straight reaching movements to on- screen targets (Shadmehr and Mussa-Ivaldi 1994). The robotic arm can apply force to the hand during movement, creating a novel dynamic environment (force field). For example, in a leftward force field, the robotic arm pushes the hand to the left as it is moved. Participants’ initial movements in a force field are highly curved, but based on the anticipatory errors, the brain quickly acquires a representation of the novel force environment. As practice progresses, the brain updates its motor anticipations to account for the external forces imposed by the robot. Repetitive motor commands are modified so as to generate compensatory muscle force patterns that

48

2 The Background of the Study on Interpersonal Coordination

will result in straight and accurate reaches. These studies on force field adaptation indicate that the brain anticipates the force requirements for arm movements and, if inaccurate, it can rapidly learn complex muscle force patterns in order to execute the desired movements.

2.6.2 Observational Motor Learning Motor learning can be achieved not only through physical practice, but also by observing the movements of another individual. In similar to motor learning with physical practice, observational motor learning is also driven by anticipatory errors. The observer generates anticipations about the sensory consequences of the tutor’s movements. These anticipations are presumably generated under the assumption that the tutor intends to move his hand to a target location in a straight path (Morasso 1981) or to lift an object in a smooth manner (Johansson and Westling 1984). Any deviation from these anticipations would indicate a movement error. Observers are not provided with multisensory feedbacks associated with physical practice, and they thus relay solely on visual information of kinematic errors. Observers could compare the actual visual outcome of the tutor’s movement with their own anticipation and use systematic errors to update their own motor anticipations for use in subsequent performance (Blakemore and Decety 2001). Action observation has been shown to facilitate the learning of novel visuomotor environments. Ong and Hodges (2010), Ong et al. (2012), and Lim et al. (2013) asked participants to observe a tutor learning to reach for targets in a environments in which the cursor was rotate 30° clockwise with respect to the coach’s actual hand position. In subsequent experiment of the same rotated visual environment, observers’ movements were more accurate in guiding the cursor to the targets compared to participants who had not previously observed a tutor. While the observer’s movements did not fully compensate for the visual rotation, their physical performance was significantly facilitated by the observation of the tutor. This indicates that individuals can learn how to adapt to a novel visuomotor environment through observation. Action observation can also facilitate the adaptation of forces when lifting objects. Reichelt et al. (2013) asked participants to repeatedly lift an object in turn with a tutor such that the participants observed the tutor lifting the object before lifting it themselves. In some trials, the objects’s weight was unexpectedly changed. For example, if the object became lighter, the tutor overestimated the load force and overshot the target lift height. Participants acquired information about the object’s new weight by observing just one of the tutor’s lift. Participants were able to use this new weight information to adapt their own load force when subsequently lifting the new weight. Buckingham et al. (2014) similarly examined the consequences of observing a tutor making load force errors when lifting small and large objects of identical weights. Participants watched either a novice tutor making lifting errors or an expert tutor who performed fully adapted, error-free lifts. After observation, the

2.6 Observational Motor Learning

49

participants lifted the objects seen in the videos. The participants who had observed the novice’s lift outperformed those who had observed the expert’s lifts. In particular, they were less likely to commit load force overestimations when lifting the larger object and needed smaller force adjustments during their lifts. Based on observing the novice’s errors, participants were able to update their motor anticipation and thus more accurately scale their fingertip forces when they lifted the objects themselves. Therefore, observers are able to use visual information of others’ movement errors in updating their own motor anticipations and adapt their fingertip force output. As the saying goes, “correct your conduct by observing that of others.” Moreover, action observation can facilitate the learning to move in the novel force environments. Mattar and Gribble (2005) asked participants to watch a video of a tutor performing reaching movements in a force imposed by a robotic arm. Participants who observed the tutor gradually adapting their movements to a force field performed better movements when they later encountered that same force field compared to participants who did not observe. While this study assessed the generation of anticipatory force from the participants’ kinematic performance, this was later confirmed through the direct measurement of participants’ generated forces after observing motor learning. Mattar and Gribble (2005) further found that the beneficial effect of observing motor learning persisted even if participants performed a cognitive distractor task while watching the video, indicating that observational motor learning is likely not dependent on the use of explicit cognitive strategies. However, observational learning was impaired if the motor system was engaged with an unrelated movement task during observation, suggesting that the availability of the observer’s motor system plays a key role in this effect. These results indicate that observational learning of novel force environment occurs through implicit engagement of the observer’s motor system.

2.6.3 The Neural Basis of Action Observation Because the mirror neurons in the premotor and parietal cortices of the monkey fire while it performs a goal-directed action and while it observes a similar action being performed by another individual (Rizzolatti et al. 1996), the neural mechanisms underlying observational motor learning come from the mirror neurons. Neuroimaging studies report a common temporo-parieto-premotor circuit (mirror neuron system) activated by both action execution and action observation (Gallese et al. 2002). This putative human mirror neuron system is part of a broader ‘action observation network’ (AON), which is involved in the visual processing of actions. The AON consists of many of the same brain areas involved in action execution, including the supplementary motor area, premotor cortex, primary somatosensory cortex, primary motor cortex, superior and inferior parietal lobules, and visual area V5/MT (Caspers et al. 2010). Transcranial magnetic stimulation studies have shown that the observer’s motor system is activated during action observation. Strafella and Paus (2000) applied

50

2 The Background of the Study on Interpersonal Coordination

TMS pulses to the primary motor cortex while participants observed hand movements and arm movements. While watching hand movements, the MEPs recorded from the observer’s hand muscle were larger whereas, while watching arm movements, the MEPs recorded from the observer’s arm muscles were larger. This indicates that the excitability of the observer’s primary motor cortex increases in a muscle-specific manner during action observation. Alaerts et al. (2010) further asked participants to watch tutors lifting objects of different weights while TMS pulses were applied over primary motor cortex and MEPs were recorded from various hand muscles. Alaerts and colleagues showed that primary motor cortex excitability was greater when observing heavy objects compared to light objects, in particular for the hand muscles that were most involved in the lifting movement. Thus, watching the movements of others activates the observer’s motor system in a muscle-specific manner in accordance with the anticipated forces requirements of the task. Action observation can also affect motor memories encoded in the primary motor cortex. Stefan et al. (2005) asked participants to perform repetitive thumb movements and then they observed a video of a tutor performing thumb movements in the opposite direction. After observation, the experimenter applied TMS to the observers’ primary motor cortex and found that the elicited thumb movements were biased in the direction of the observed movements. This finding suggests that motor memories encoded in the primary motor cortex are subject to modification via action observation. Rizzolatti et al. (2001) hypothesize that the activation of cortical sensorimotor areas during action observation reflects the engagement of the observer’s motor representation as it simulates the observed movement. There are numerous proposed roles of covert simulation within the AON, including action understanding, action anticipation, inferring intentions, imitation and various functions related to social cognition such as communication, emotional empathy and perspective taking (Rizzolatti and Craighero 2004). In particular, the AON’s proposed function in action anticipation is important to observational motor learning. To examine covert motor simulation and anticipation during action observation, Flanagan and Johansson (2003) tracked participants’ eye movements both while they performed a block-stacking task and while they observed a tutor performing the task. When watching the tutor, participants’ eye movements were very similar to those produced when they performed the task themselves. Their eye movements were proactive in both conditions, fixating on the grasp sites of the blocks before the hand picked them up and fixating on the landing sites before the blocks were placed. This gaze-hand coordination suggests that participants were not simply assessing the tutor’s movements at a visual level, but rather were anticipating the tutor’s upcoming movements through activation of their own motor representations. These results indicate that observed actions are represented by the motor system of an observer in such a way that can contribute to the anticipation of others’ actions. Such a neural mechanism linking action observation and action execution may also underlie motor learning through observation.

2.6 Observational Motor Learning

51

2.6.4 T he Motor System Accompanying Observational Motor Learning There is growing evidence that observing motor learning engages the observer’s motor system. Using TMS, Brown et al. (2009) have found that the primary motor cortex (M1) plays a key role in observational motor learning. When TMS is applied repetitively, neurons beneath the TMS coil are temporarily hyperpolarized, dampening excitability so as to create a ‘virtual lesion’ of the target brain area. TMS was repetitively applied to M1 after participants observed a video of a tutor learning to reach in a force field. Observational motor learning was attenuated for participants who received repetitive TMS to M1 such that their subsequent performance in the observed force field was comparable to participants who had not observed a tutor. By contrast, observational motor learning was not affected by the application of repetitive TMS to a control brain area not involved in motor function. These findings suggest that action observation engages learning mechanisms in the observer’s primary motor cortex. In addition, Malfait et al. (2010) have reported that observing movement errors during motor learning engages a network that is also involved in processing our own movement errors. Malfait and colleagues asked participants to undergo fMRI while observing videos of a tutor performing reaches in a force field, which depicted the typical progression from curved to straight movements during learning. During observation, activity in the intraparietal sulcus, dorsal premotor cortex and cerebellum was modulated by the magnitude of the tutor’s reach errors. This network quite overlaps with a network engaged in processing one’s own movement errors during physical practice. This result indicates that the observer’s motor system is activated when processing visual information about movement errors, in particular when the tutor commits large reach errors. This finding supports the idea that the observer’s motor system covertly simulates observed action as a means of detecting movement errors and updating motor anticipation. Using resting-state fMRI, Albert et al. (2009) and McGregor and Gribble (2015) assessed the neural basis of observational motor learning. Resting-state fMRI is a neuroimaging technique in which the blood-oxygen-level-dependent (BOLD) signal is measured while a participant is in a state of wakeful rest. After performing a task, the brain areas engaged in that task show changes in activity while at rest. McGregor and Gribble (2015) assessed changes in resting-state activity after observing motor learning in order to gain insight into the brain networks underlying this phenomenon. As we learn a motor skill, the learning process is accompanied by changes in task performance. Conventional task-based fMRI paradigms cannot distinguish the changes in brain activity that are due to learning from the changes that are due to such performance differences. Because there is no task performed during resting-state MRI scans, any changes in brain activation are due to learning itself. McGregor and Gribble (2015) asked participants to undergo resting-state MRI both before and after observing a video of a tutor learning to reach in a force field. Participants then performed reaches in a force field and a motor learning score was

52

2 The Background of the Study on Interpersonal Coordination

calculated based on the straightness of their movements. The analysis found a new network consisting of visual area V5/MT, which is involved in motion perception (Zeki et al. 1991), the cerebellum, dorsal premotor cortex, primary motor cortex and somatosensory cortex. Activity in this network was modulated by the participants’ motor learning scores, such that participants who learned more through observation showed greater functional changes in this network. These findings indicated a link between visual systems involved in motion perception and sensory-motor circuits involved in motor learning. This network forms the basis by which visual information of other’s movements is transferred to the sensory–motor system for learning new motor skills.

2.6.5 S ensory Changes Accompanying Observational Motor Learning Similar to changes in motor performance and neural plasticity within motor circuits with motor learning, somatosensory changes have been reported following physical practice (Ostry et al. 2010) as well as following the observation of motor learning. Bernardi et al. (2013) examined somatosensory function before and after participants observed a video of a tutor learning to reach in a force field. Somatosensory function was assessed by having participants grasp the handle of a robotic arm, close their eyes and make judgments about the position of their arm. Consistent with previous studies, observing motor learning facilitated participants’ motor performance when they alter encountered the observed force environment. The participants who watched a tutor learning to reach in a rightward force field subsequently perceived their arm to be positioned more rightward than it actually was. By contrast, the participants who watched a tutor learning to reach in a leftward force field subsequently perceived their arm to be positioned more leftward than it actually was. These results indicate that observational motor learning promotes plasticity not only in the motor system, but also in the sensory system. In conclusion, the brain can learn to alter its movement planning and force output through observation of other’s movements. However, we know little regarding the duration and generalizability of observational motor learning. Studies on the behavioral and neural bases of observational motor learning are interesting not only from a research perspective, but also for its potential clinical applications. Current stroke rehabilitation strategies require patients to undergo intensive physical practice aimed at promoting plasticity and reorganizing damaged sensorimotor networks. This strategy may not be an effective option for stroke patients with poor or absent voluntary movement control. Although observational motor learning may serve as a supplement or alternative to conventional rehabilitation techniques, it may be able to promote adaptive plasticity in sensorimotor circuits and restore motor function in stroke patients. Garrison et al. (2010) report that the effectiveness of this technique has yielded mixed results using inconsistent methods. In the future study, we should

2.7 The Effect of Action Expertise on Shared Representation

53

assess whether observational motor learning would be an effective and feasible stroke rehabilitation technique.

2.7 The Effect of Action Expertise on Shared Representation When we watch dance, we feel that we are in a relationship between dancers and ourselves. If the dancers jump highly and turn rapidly, we might also image to jump and turn together. If the dancers play the role of a bird, we could also image to play the role of the bird. By dancing or watching a dance, as we sympathize with others, and play the role of others, we extend the relationship among people. Such relationship between dancers and observers suggests that they share the representation of a dance performed by dancers and watched by observers. It is expected that there is a difference between expert and novice dancers for shared representations. This section thus addresses the relationship between an actor and an observer’s motor ability in a dance and sports, and how expertise impacts this relationship. The idea that perception and action share a common cognitive architecture dates back at least to the time of William James (1890). In the recent decades, since the discovery of mirror neurons within the premotor and parietal cortices of rhesus macaques, psychology and cognitive and social neuroscience have witnessed a surge in interest in the mechanisms supporting coupling between action and perception (Gallese et al. 1996). The discovery of mirror neurons in the monkey brain provided critical evidence in support of a direct matching or common coding account of how the primate brain navigates between perception and action (Gallese et al. 2011; Prinz 1990).

2.7.1 Effects of Expertise on Perception How is longstanding motor expertise reflected at behavioral and neural levels? The concept of expertise can be described as in-depth knowledge of a particular field. The storage of all motor knowledge an individual has acquired during his lifetime is known as a motor repertoire. A motor repertoire is like a vocabulary of actions. While each person’s motor repertoire depends on by the movements he has learnt, an individual’s motor learning is constrained by two factors. First, motor learning is constrained by limitations of human anatomy. We are mainly guided by the flexion and extension of the joints known as the degree of freedom of movement. Second, our motor repertoire depends on our own individual physical experience. Most people naturally acquired the ability to perform common motor patterns, such as walking, running, or throwing, through motor learning. On the basis of basic actions, furthermore, we can train to perform more complex and accurate actions. For example, a figure skater has practice to execute motor sequences on ice for a long time,

54

2 The Background of the Study on Interpersonal Coordination

and a set of motor commands related to this action is store in a neural network that composes her motor repertoire. Neuroimaging Evidence: By the discovery of mirror neurons, we know that overlapping neural codes are recruited whether watching or performing the same action. We further know that there is a shared neural system for action observation and execution in the primate brain. Early studies on action observation have been conducted with non-primates involving observation of simple goal-directed actions such as grasping and reaching using an fMRI. These studies repeatedly confirm activation in the ventral premotor cortex, parietal cortex and superior temporal sulcus. However, these early works could not discriminate between a general sensorimotor response during observation and a specific internal motor simulation of the observed action. Calvo-Merino et al. (2005) examined the extent to which brain activation during action observation reflects resonance between an observed action and one’s specific motor repertoire. They designed a series of studies employing longstanding expertise. Their expertise model is based on the individuality of one’s motor repertoire depended on individual learnt movements, and the similarity of the motor repertoire of those individuals who received similar training. They chose two groups of dancers who were trained in different movement vocabularies that were kinematically similar: classical ballet and capoerira. Employing two expert groups was important to ensure that putative effects were not due to a simple experience effect. Using an fMIR, Calvo-Merino and colleagues compared brain activation of ballet and capoeira dancers and control participants when all participants watched three- second video-clips of ballet and capoeira movements (Fig. 2.5). Participants were asked to watch the videos and perform a dummy task to ensure they were paying attention to the stimuli. The brain activation was compared when watching familiar movement with when watching unfamiliar movement. There was an interaction between group (ballet, capoeira and controls) and type of observed movement (ballet, capoeira), showing a specific effect of expertise. This was clear as an effect of watching familiar movement compared to activations while watching unfamiliar movement. In particular, this interaction showed significant activation in the premotor cortex, superior parietal lobe, intraparietal sulcus and posterior superior temporal sulcus. The control group did not show differentiated responses within the described areas while watching ballet or capoeira moves. These results suggest that neural activation of these regions during observation of familiar actions may provide access to a form of shared action representation between the observer and the performer. However, the work of Calvo-Merino et al. (2005) raised a question which component of an action representation is retrieved during observation. Does action observation predominantly engage motoric mechanisms over the visual representation or sematic knowledge of the action? In order to answer this question, it is necessary to examine an observer’s experience associated with different components of an action representation. Calvo-Merino and colleagues were aware that classical ballet features gender-specific movements and gender-common movements. Female

2.7 The Effect of Action Expertise on Shared Representation

55

Fig. 2.5 The relationship between longstanding dance expertise and brain engagement (Redraw from Cross and Calvo-Merino (2016) with permission from Cambridge University Press). (a) schema of 2 × 3 design using expert observes. The expertise effect is determined by interaction group (observer: ballet dancers, capoeristas, non-dancers) and type of observed movement (ballet movements, capoeira movements). (b) parameter estimates for the expertise effect during action observation in left precentral gyrus/dorsal premotor cortex and left intraparietal sulcus. BB: ballet dancers viewing ballet, BC: ballet dancer viewing capoeira, CB: capoeira dancers viewing ballet, CC: capoeira dancers viewing capoeira. (c) schema of 2 × 2 × 2 design using experts with visual and motor familiarity and only visual familiarity. (d) schema of standard brain activations significant at (1) ventral premotor, (2) dorsal premotor, (3) SPL, (4) IPS, and (5) pSTS

dancers trained in classical ballet acquire motor training of female specific moves, and vice versa for male dancers. However, as female and male dancers train and perform together, both genders acquire visual familiarity and sematic knowledge about all the movements, regardless of gender specificity. Calvo-Merino et al. (2006) subsequently designed an experiment that enabled the dissociation of visual and motor familiarity, to test for brain regions that might respond to an internal simulation of the action in specifically motor terms, over any associated visual or sematic representations. Using an action observation task, they asked female and male classical ballet dancers to watch three-second video-clips of gender-specific dance movements. A female dancer and a male dancer dressed in black clothes performed these movements (Fig. 2.5). The dancers also watched a set of dance movements commonly performed by both genders to rule out any possible effects related to observing a female or a male dancer. In order to dissociate motor and visual representations during observation of gender movements, only classical ballet dancers needed to train specifically in their respective gender movement vocabulary participated in the study. To ensure that dancers’ prior motor training adhered to gender-defined conventions, all dancers completed a preliminary questionnaire enquiring about how often they performed and watched the individual movements used in the experiment in their professional training. The questionnaire showed that male dancers were visually familiar with both male and female move-

56

2 The Background of the Study on Interpersonal Coordination

ments, but only motorically familiar with the male-specific movements, and the opposite was true for the female dancers. This control was in particular important as it is becoming increasingly common for the ‘rules’ of classical dance to be broken in order to create novel performances. In order to find areas turned by motor resonance with the observed action rather than visual or sematic knowledge, Calvo- Merino and colleagues compared brain activity related to gender-specific movement controlling for visual or sematic knowledge with classical ballet movements common to both genders in two female and male dancers and a control group. They found that, during observation of motorically familiar movements, brain activity was stronger in three brain regions: the premotor cortex in the left hemisphere, and the bilateral superior parietal lobe and cerebellum. In both studies of Calvo-Merino et al. (2005, 2006), brain activities shared a common set of areas classically identified as core nodes of the human mirror system. These regions, jointly with the cerebellum, are involved in making motor responses and also coding observed actions. In addition, although activation of the STS is observed during observation of familiar movements, the STS does not appear to participate when truly motor resonance is isolated. Thus, the activation of the STS must be related to features of the action such as visual or sematic familiarity rather than strictly motor familiarity. Using a similar expertise paradigm, Orgs et al. (2008) also found that alpha/beta event-related desynchronization was reduced when expert dancers watched familiar movements compared to non-dancers observing the same stimuli. These studies indicate that neural activation in the mirror neuron system during observation of familiar actions may provide access to a form of shared action representation between the observer and the performer. Behavioral Evidence: There are many examples of expert observers’ ability to spot nuances in skilled performers’ actions such as judges of diving, gymnastics or figure skating in the Olympic games. Is Olympic judges’ visual sensitivity enhancement by experience due to the fact that many of them have been practitioners of the same sport they are judging, or is this simply a matter of extensive visual practice (also see Urgesi and Makris (2016))? In order to address this question, Calvo-Merino et al. (2010) compared dance experts’ performance in a simple visual discrimination task of dance movements. This experiment consisted of four groups: female and male expert dancers, and female and male non-expert dancers. As described in the previous subsection, when participants observed two types of dance movement in the classical ballet, the gender factor was important. In addition, to facilitate a broad spectrum of performance across expertise levels on the behavioral task, a set of stimuli was made using point- light displays (PLDs). This technique is used to study biological motion, and points of lights are attached to the main joints of a performer while actions are recorded in a dark room (Johansson 1973). The task consisted of watching pairs of dance videos depicted as PLDs and judging whether the pairs depicted the same or different videos. Video pairs always depicted the same movements, which were performed by either the same dancer or two different dancers. This task was difficult for classical ballet dancers because they are trained such that their performance minimizes any

2.7 The Effect of Action Expertise on Shared Representation

57

possible representation that could be used to differentiate between the dancers. The control group performed the task significantly worse than experts, and no interaction with gender was found. Both female and male dancers performed the task with a similar accuracy. These results indicate that visual sensitivity to others’ action improves with expertise. However, once the visual sensitivity reaches a specific level of expertise, there appears to be little room for behavioral improvement. From a different perspective, Loula et al. (2005) addressed the effect of expertise on visual perception. They compared performance on action recognition and agent identification tasks while participants observed PLDs of strangers and friends. Performance was significantly better during observation of a friend performing the movements as compared to the other conditions. This visual familiarity effect represented in the ability to recognize our friends’ actions highlighted the effect that mere visual exposure may have on action discrimination. In addition, Casile and Giese (2006) ruled out whether motor experience made an additional unique contribution to action perception. Participants were blindfolded while they were asked to undergo motor training, showing significantly better visual discrimination of physically trained motor action without vision.

2.7.2 E ffects of Experimentally Induced Expertise on Perception Another way to examine how motor experience with complex action shapes perception is to induce expertise in an experimental context. When examining the effect of motor action on perception in the laboratory, the merit is that the actual amount of time a participant has spent rehearsing or watching an action can be manipulated and measured. Naturally, this approach is not without its limitations because there is a quite difference between the times required for achieving a skilled motor performance in the laboratory and over a lifetime. Ericsson et al. (1993) reported that at least 10,000 h are required for achieving skilled professional status in a motor skill. However, in tandem with studies examining perception of expert over a lifetime of deliberate practice, the studies in a laboratory presumably produce a more complete picture of how visuomotor experience shapes perception. Neuroimaging Evidence: Employing expert contemporary dancers, Cross et al. (2006) asked the dancers to learn a new 25-minute work of dance across a six-week rehearsal period. While the dancers were invited into the laboratory each weekend across the rehearsal period, they underwent fMRI when watching short video segments of the choreography they were rehearsing as well as kinematically similar movements that they never physically rehearsed. The dancers’ task during the scanner was to watch each movement and imagine themselves performing it, and at the end of each video assign a rating based on how well they thought they could reproduce the particular movement segment at present. Across all scanning sessions when the dancers watched rehearsed movement compared to the kinematically

58

2 The Background of the Study on Interpersonal Coordination

similar non-rehearsed movement, Cross et al. (2006) found a pattern of activity comprising parietal, premotor and superior temporal cortices similar to the report of Calvo-Merino et al. (2005). The new finding of Cross and colleagues is that, when the dancers’ ratings of their own performance ability were added to the neuroimaging data as parameter modulators, brain regions showed increasing levels of activity the better the dancer could perform the observed movement. This analysis showed activity increases in the two core mirror system regions, namely, the left ventral premotor cortex and inferior parietal lobule, finding that more adept an observer becomes at performing an action, the more he simulate that action when observing it. Shortly after Calvo-Merino et al. (2006) reported on the effect of visual compared to physical experience on perception among expert ballet dancers, Cross et al. (2009) examined how visual and physical experience with complex, full-body actions impacted brain and behavior. Novices at dance were trained to dance a number of sequences over 5 days of training using a popular dance video game similar to ‘Dance Dance Revolution’. During each day of physical practice, participants spent an equivalent amount of time watching a different but similar set of dance sequences that they never physically practiced. A third set of dance sequences remained untrained. Participants underwent fMRI scanning on the first day of the study (before the starting of training) and after the fifth and final day of training concluded. During scanning, participants watched and listened to the soundtracks of each of the sequences from the physically trained, observationally trained and untrained sequences. After all scanning and training procedures were completed, participants returned to the laboratory to perform all sequences with the dance video game, which enabled objective scoring of physical performance across all training categories. After the 5 days of physical and observational training, participants performed the physically practiced sequences the best, whereas they performed the untrained sequences the poorest. Their performance for the observed sequences was at an intermediate level between practiced and untrained sequences. Participants were never told to try to learn the sequences they observed during daily training, and were not told until the final day of the study they would be asked to perform these sequences. Participants were told to sit and watch a few sequences in between physical training bouts to reduce their heart rate. Thus, performance gains from visual experience represent incidental learning from passive observation. The analysis was performed to find brain regions that respond to practiced compared to untrained sequences and observed compared to untrained sequences (Fig. 2.6), showing that two sensorimotor brain regions emerged: the left inferior parietal lobule and the right premotor cortex. After the 5 days of training, neither of these brain regions discriminated between sequences that were physically practiced or visually experienced. Contrary to the report of Calvo-Merino et al. (2006), Cross et al. (2009) indicated that, in novices at dance, a week of physical practice with one set of movements and visual experience with another produced similar responses within parts of the parietal and premotor cortices. However, in the direct contrast of physical > observational practice, Cross

2.7 The Effect of Action Expertise on Shared Representation

59

Fig. 2.6 Effects of visual and physical experience on brain response and behavior (Redraw from Cross and Calvo-Merino (2016) with permission from Cambridge University Press). (a) Left ventral premotor cortex (PMv) and inferior parietal lobule (IPL) showed an increasingly robust response when dancers watched movements they were most expert at physically performing. (b) brain regions emerging from an analysis evaluating overlap between danced > untrained sequences (top left) and watched > untrained sequences (top right) from the training study performed with novice dancers. The parameter plots beneath the brains show the response within the left inferior parietal lobule (bottom left) and right premotor cortex (bottom right) during the pre-training scans when participants observed music video-clips from the different training sequences during fMRI

60

2 The Background of the Study on Interpersonal Coordination

et al. (2009) found greater activity in the right dorsal premotor cortex, indicating a role for dorsal premotor cortex in specially physical action experience. Neurophysiological and Behavioral Evidence: Laboratory-based training experiments have shown how experience shapes perception in a number of ways. Arbib and Rizzolatti (1999) noticed that mirror neurons seemed to generalize their responses to actions performed by non-biological effectors, such as tools. Ferrari et al. (2005) trained monkeys to perform actions with hand, arm or mouth, and to observe actions performed by a hand, mouth or tool. After 2 months of training, a certain subset of neurons in premotor area F5 responded most strongly when the monkeys observed actions performed by a tool. This result suggests that the training experience enabled the monkeys to extend their action understanding capacity to actions for which they lack a strict corresponding motor representation. Similarly, employing human participants, Casile and Giese (2006) asked blindfolded participants to learn to perform a novel movement of the upper-body with only verbal and haptic feedback. After this non-visual learning, the participants were asked to visually identify non-visually learned actions. Not only were participants able to identify the visual test pattern of non-visually learned movement after training, but also the accuracy with which participants could execute the performed movement correlated positively with visual recognition performance. This study further indicated that changes in one’s motor repertoire result in changes in perception. In addition, Kirsch et al. (2013, 2015, 2016) have recently reported the effect of training manipulations on an observer’s affective response to a perceived action. In these studies, participants learned to perform complex dance sequences in a video game context that used whole-body motion tracking to quantify performance. These studies accessed how participants’ affective responses when watching these dance sequences changed after they had spent time physically practicing them, simply observing them or only listening to the music that accompanies them. Participants enjoyed watching dance movements after they had spent time either physically practicing them or passively observing them. However, listening to the soundtrack only had no effect on the enjoyment participants derived from watching the movements. These studies indicate that an observer’s affective response to watching others’ action presents another experimental paradigm for examining the effect of laboratory-induced training experience on perception.

2.8 T he Effect of Motor Expertise on Observational Learning in Sports In the past few decades, sport performance has been studied for many different research fields including cognitive psychology and cognitive neuroscience. Although these studies try to examine the cognitive and neural basis of expert

2.8 The Effect of Motor Expertise on Observational Learning in Sports

61

performance in sport, neuroscience research is constrained by laboratory-based experiments that do not assess how sport skills acquired over a long period of time. Thus research on sport experts has mainly focused on behavioral experiments. However, the review of Urgesi and Makris (2016) points out that important aspects of expert performance are related not only to their ability to execute complex actions, but also to superior perception of the actions of opponent or confederate players. The recent interest of researchers thus facilitates the research stream related to the neural bases of perception and action coupling. While the action observation network includes both visual (occipitotemporal) and motor (frontoparietal) areas and seems to be responsible for the observation and simulation of perceived actions (Rizzolatti and Craighero 2004), the activity of the network depends on the familiarity of actions. In other words, the more familiar the observer is with a given action sequence, the greater the neural response magnitude in premotor and parietal areas seems to be (for a review, Cross and Calvo-Merino 2016). In line with this notion, while the recognition and simulation of ongoing actions depends on subjective experience (Aglioti et al. 2008; Urgesi et al. 2012), covert simulation of actions is crucial for both imitative and no-imitative motor learning (for a review, Obhi 2016). Sport performance thus becomes an important model in cognitive neuroscience to study how perception and action interact and what neural mechanisms support such interaction.

2.8.1 E ffects of Action Observation on Motor Execution in Sport The notion of common representation for executed and observed actions is interesting in applicative fields like motor learning in sports. Similar to motor imagery, action observation may offer the possibility to acquire new motor skills (McGregor and Gribble 2016). In contrast to motor imagery, on the other hand, action observation does not require active effort by trainee to image the movements. Compared to motor imagery, another benefit of action observation is that observation can be more easily controlled by the trainer. Some behavioral studies in the field of motor control and learning have shown that observation learning of movements can lead to subsequently improved performance (Ashford et al. 2007), although the extent of improvement may be lower compared to physical practice. Observation learning involves the action observation network, and this is corroborated by TMS evidence that interference with primary motor cortex activity disrupts the consolidation of motor memories acquired after both physical (Muellbacher et al. 2002) and observational (Brown et al. 2009) learning. These results suggest that both types of training induce changes of movement representation in the primary motor cortex. Importantly, not only is the general coordination pattern of movements acquired during observational practice, but also acquired are the movement parameters such

62

2 The Background of the Study on Interpersonal Coordination

as timing and force scaling. In other words, observational practice does not only specify what must be performed, but also how to perform the movements. In keeping with the effects of physical training, moreover, the observation-related improvements can be transferred to a different motor task. However, the mechanism of motor skill acquisition during physical and observational practice seems qualitatively different. The combination of physical and observational practices results in a greater advent in transfer tasks (Shea et al. 2000), compared to either kind of training in isolation. For example, Gruetzmacher et al. (2011) reported that physical practice induced a greater advent in the transfer to tasks when using the opposite limb to control the same pattern of muscle contractions and joint angles (when the motor coordinates of the actions were maintained). By contrast, observational practice induced a greater advantage in the transfer to tasks when using the opposite limb to control movements with the same direction in the external space (when the motor coordinates of the movements were changed, but the visuo-spatial coordinates were maintained). Similarly, although adapting to the repeated observation of reaching movements in a perturbed environment did not induce any after-effects, adaptation to the repeated execution of the same movements induced after-effects (Ong and Hodges 2010). These results suggest that observational learning may facilitate the development of a representation of visuo-spatial coordinates of a given action but not of its specific motor codes in terms of joint angles and activation patterns. The specific motor codes require direct motor experience. Taken together, although action observation can trigger activation of not only visual but also motor areas, and induce the information of motor memory, the extent of involvement of visual and motor areas and cognitive processes underlying observational and physical learning do not completely overlap.

2.8.2 Effects of Motor Expertise on Action Perception in Sport Motor expertise also affects action perception, in particular anticipatory simulation of action sequences. In a constantly changing environment, the full sequence of an action is rarely visible and missing information needs to be completed. The perceptual system formats anticipatory representations of observed motion sequences on basis of internal modes of the rules that dictate these actions (Hubbard 2005). This kind of top-down modulation is responsible for optimal interaction with moving objects or living creatures. In sport, because performers have to plan their actions based on anticipations of the future of perceived motion sequences executed by their opponents and confederates, anticipatory representations of observed actions is essential. This view has been confirmed by behavioral evidence from different sports that elite athletes are equipped with a unique ability to make accurate anticipations of the outcome of observed sport actions. Previous studies on sport performance and the relationship between expert perception and action have focused on whether this perception is direct or indirect (for a review, Craig 2013). From a viewpoint of indirect

2.8 The Effect of Motor Expertise on Observational Learning in Sports

63

process, expertise is based on sensory information and internal representations that are stored in memory and recalled during action execution to influence choice and performance (Handford et al. 1997). When the indirect process comes to action anticipation, experts outperform novices in making better and more accurate anticipations of the outcome of action sequences (Williams et al. 1999). This notion has been confirmed by studies in different sports, such as soccer (Savelsbergh et al. 2002), volleyball (Kioumourtzoglou et al. 2000) and Rugby (Jackson et al. 2006). For example, Ripoll et al. (1995) examined the specific visual search strategies, information processing and decision-making mechanisms of expert boxers, intermediates and novices. All participants were tested in a virtual environment replicating a boxing field and had to response to maneuvers of an on-screen opponent. In addition, the complexity of the environment was divided into two categories: simple and complex. Behavioral data and eye-tracker recordings showed that expert boxers were better at anticipating the maneuvers of the opponent, as well as accurately responding to these, as compared to intermediates and novices. This was clear in the complex environment. In their visual strategies, furthermore, expert boxers exhibited unique spatial and temporal characteristics of visual search activity. Similarly, Williams (2000) reviewed that skilled soccer players are better at recalling and recognizing patterns of play compared to the less-skilled players. Moreover, in controlling eye-movement patterns for seeking and choosing the most important sources of visual information, skilled soccer players are provided with advent of making more accurate anticipations of the outcome of their opponents’ actions. However, an indirect approach to the relationship between action and perception is criticized by the fact that it cannot account for cases of consistent skilled performance under extreme conditions that would not allow memory recall (Craig 2013). Thus, a direct approach to the relationship between expert action and perception has been proposed. The ecological psychology approach to perception indicates that action processes are deeply based on vision and perception, and that the relationship between vision and action is dynamic and mutual (Gibson 1979). If that is the case, the relationship between the athlete and the environment, the information with which the environment is constantly bombarding the athlete and the ways the athlete actively react to it should be focused upon (Handford et al. 1997). In dynamic sports, the environment undergoes continuous change, as relevant actions are constantly altered, and athletes have to account for the uniqueness and variability of any given scenario (Craig et al. 2011). In addition, Craig et al. (2009) examined how dynamic on-line changes of ball trajectory affected soccer players’ anticipations of the ball’s future arrival position. Unlike previous studies that address anticipatory judgments in sports, this study examined how expert and novice soccer players are influenced by dynamic visual stimuli such as a ball’s moving trajectory. Experts recalled more previous information for making accurate anticipation than novices. The experts are also better attuned to visual information invariants than the novices. In line with soccer players’ anticipations (Craig et al. 2009). Correia et al. (2011, 2012) reported that expert rugby players are better than novices at attending to or tuning into on-line actionrelevant information within the environment, and achieving superior performance.

64

2 The Background of the Study on Interpersonal Coordination

Fig. 2.7 Schematic representation of a temporal occlusion paradigm in basketball free shots at different intervals from onset. (Aglioti et al. (2008), redraw from Urgesi and Makris (2016) with permission from Cambridge University Press)

2.8.3 Motor Experts Read Body Kinematics Using a temporal occlusion paradigm, Aglioti et al. (2008) and Urgesi et al. (2012) have been found that expert athletes provide not only more accurate but also earlier anticipations of the outcome of sport actions. Aglioti et al. (2008) asked elite basket athletes, expert observers (coaches) and novices to anticipate the probability of success in basket free shots (i.e., ball in or out of the basket) whose presentation was interrupted at different time intervals (Fig. 2.7). Professional basket players were better at making accurate anticipations at shorter video presentation times, compared to both expert observers and novices. For example, professional basket players were able to anticipate the outcome of the action for presentation times as short as 426 ms. Expert players were more able to make accurate anticipations by basing their judgments on the initial body cues of the model player executing the shots. These results indicate a direct evidence of the role of motor experience in simulating observed actions and anticipating their outcome. Subsequently, Urgesi et al. (2012) confirmed the finding of Aglioti et al. (2008) by applying similar experiment paradigms in volleyball. Expert volleyball players, expert observers (volleyball tem supporters) and novices were asked to anticipate the probability of success in volleyball floating services. In previous temporal occlusion paradigm studies (Aglioti et al. 2008; Farrow and Abernethy 2003), video

2.8 The Effect of Motor Expertise on Observational Learning in Sports

65

presentation was interrupted at different instants after the beginning of the action. The presentation of body- versus ball-related cues was confounded with increasing viewing time and cumulative presentation of more information available on the action. Thus, the presentation could not be established whether the superior perceptual abilities of motor experts reflected faster processing speed and need of less information to make a decision or were specifically related to their capacity to read body kinematics. Urgesi and colleagues modified a previous occlusion paradigm, and used only initial body movements or only ball trajectory of volleyball floating serves. This modified paradigm allowed participants to highlight the complementary roles of motor and visual expertise on the representation of body action and object motion. Similar to the results of Aglioti et al. (2008), both expert players and observers are better than novices at making accurate anticipations on the basis of the ball trajectory. In addition, only expert players can depend on the anticipations of body kinematics. Urgesi et al. (2012) indicate that expert athletes, but not expert watchers, are equipped with fast and generally accurate perceptual mechanisms that allow them to read the kinematics of others’ body movements in order to anticipate the future course of observed actions. However, this difference between expert athletes and watchers may reflect either the greater extent of athletes’ visual experience with domain-specific sport actions or the additive combination of visual and motor experience. Although comparing different groups of individuals with acquired domain- specific expertise is seldom used in longitudinal studies, it enables researchers to test the effects of extensive practice periods. However, such an approach is unable to clearly disentangle the specific roles of motor and visual experience in action perception. In addition, comparison of different groups does not take into account possible pre-existing inter-individual differences that may have determined superior abilities of athletes in sport as compared to novices or watchers. In order to clarify the specific roles of motor and visual expertise on anticipatory perceptual abilities, Urgesi et al. (2012) conducted a follow-up experiment in volleyball using a longitudinal design. Participants consisted of adolescents who practiced volleyball and were divided into three groups: (1) those receiving physical training for execution of floating services, (2) those being trained with repeated observation of floating service executed by expert player, and (3) those receiving control training by watching videos of volleyball defense actions, in which the floating services were cut out. Participants assigned to the physical training condition improved their ability to anticipate the outcome of volleyball floating services by reading the initial bodily cues of the opponent. Participants assigned to the observational training improved in their ability to understand the ball trajectory. However, participants trained in watching nonspecific volleyball actions did not show any improvement. These results indicate the selectivity of the training effects, highlighting the distinct and complementary contributions of physical and observational experience to the development of superior action-anticipation skill in elite athletes.

66

2 The Background of the Study on Interpersonal Coordination

2.8.4 Neural Systems Underlying Action Perception in Sport In basketball free shots, Aglioti et al. (2008) showed that professional players anticipate the outcome of the action more accurately and earlier than expert observers and novices. Aglioti and colleagues further conducted a second experiment combining a temporal occlusion paradigm with measures of corticospinal excitability by means of single-pulse TMS to study motor facilitation during sport action observation. During observation of the videotaped basket free shots, single-pulse TMS-induced corticospinal reactivity of the muscles associated with the observed action of athletes, watchers and novices was measured at different time intervals from action onset. In line with their behavioral findings, results of the TMS showed increased motor excitability in expert players and watchers more than in novices when they were anticipating the outcome of free basketball shots. However, this increase in motor responses was not replicated when the same players and watchers anticipated the outcome of soccer kicks as a different sport. By contrast, the motor cortex of novices was comparably facilitated during observation of basketball and soccer actions. On the other hand, greater facilitation of the little finger motor representation was obtained during observation of erroneous compared to correct throws. This modulation was only obtained when viewing videos that the model player could exert final control over the ball trajectory using distal movements of the fingers. This finding thus suggests that excellent performance in sport may be related to the fine-tuning of specific anticipatory mechanisms that allow for an earlier and more accurate anticipation of future of others’ actions, supporting the role of motor expertise in anticipating the future of observed actions. Different nodes of the action observation network may be more affected by motor and visual familiarity. While the activity of motor and premotor areas is more dependent on motor experience, that of visual (temporal) areas are more dependent on visual experience. This complementary role of motor-premotor and temporal areas in action expertise is in keeping with single-cell recordings in the monkey’s brain, in particular the superior temporal sulcus and motor and premotor areas. Neural responses in the superior temporal sulcus are influenced by previous action perception more than by execution (Rizzolatti and Craighero 2004). By contrast, neural responses in the premotor cortex seem to occur both during action observation and execution, indicating the role of previous motor experience in anticipating the outcome of ongoing actions (Avenanti et al. 2013a, b). These results suggest that, while neural activity in the superior temporal sulcus uses visual information and perceptual experience to form a representation of ongoing actions, activity in the premotor cortex may function as an internal forward model, basing judgments on and anticipations for observed actions on motor expertise. In a recent neuroimaging study, activation of new brain areas is found in superior perceptual-motor skills of expert basketball players. Abreu et al. (2012) devised an fMRI experimental paradigm, according to which expert basketball players and novices determined the outcome of free shots performed by model players. Video- clips of shots were manipulated in such a way that either forward or backward

2.8 The Effect of Motor Expertise on Observational Learning in Sports

67

moving conditions were presented, allowing the experimenters to distinguish pure attentional monitoring from anticipatory action skills. Results of blood-oxygenation- level dependent (BOLD) response showed that areas associated with the frontoparietal action observation network were equally activated in both basket experts and novices whenever they had to anticipate an action. In athletes, a higher activation of the extrastriate body area was also observed associated with the observation and reading of the model’s action kinematics in the video-clips, as well as an involvement of the inferior frontal gyrus and the anterior insular cortex following response errors in the task. Furthermore, although anticipation of correct action in expert athletes induced higher activity in the posterior insular cortex, this type of activation in novices was observed in areas of the orbitofrontal cortex. These new findings suggest that neural bases of athletes’ superior perceptual and motor abilities cannot be searched in one area or even a set of areas related to similar function, but are related to complex interactions between multiple brain structure.

2.8.5 Motor Expertise and Detection of Deception Deceiving the opponent and detecting others’ deceptions are considered important components or skills in an athlete’s repertoire of sport actions. For example, in a 1 vs. 1 situation in rugby, Brault et al. (2010) described deception in sports as an exaggeration in body-related cues that induces others to make incorrect action anticipations and delays in postural cues that may inform opponents of sudden changes. Previous studies examining the detection of action deception in sports have described it as the ability to identify incongruence between honest and deceiving body- kinematic cues and flexibility of updating ongoing action representations on the basis of upcoming information. However, the previous studies have reported mixed results concerning the role of experience in detecting deceiving actions and accurately responding to them. Jackson et al. (2006) examined the ability to detect deceiving actions both in expert and novice rugby players. The type of detection was a change in the observed body direction with and without deceptive movement. All participants had to anticipate the direction of the change. Experts’ responses were less susceptible to deceptive cues, as compared to those made by novices. In addition, expert players were more confident than novices in anticipating the outcome of the direction change in the case of deception trials. Canal-Bruland et al. (2010) also examined deception in handball players and novices, finding that experts outperformed novices in anticipating the outcome of true or fake shots by a penalty-taker. In their results, however, neither the degree of motor experience nor visual familiarity could account for successful recognition of deceptive actions. Using a different approach, Dessing and Craig (2010) examined deception in soccer. Participants consisted of novice and expert goalkeepers, and they were asked to judge the outcome of free kicks. Instead of looking at how soccer players try to deceive their opponents, they were asked to detect the bending of free kicks and

68

2 The Background of the Study on Interpersonal Coordination

gravitational acceleration of the ball. Thus they had to decide on the final direction of the ball in relation to the goal line. Even experts were frequently deceived when the ball followed a bending trajectory. Despite extensive visual and motor experience in defending ball kicks, the experts could not account for the spin-induced visual acceleration of the ball. However, it is difficult to generalize the deception in the case depended on the ball trajectory rather than the body kinematics of the players. To overcome this limitation, Tomeo et al. (2013) examined how incongruent body kinematics may affect judgments for the outcome of soccer penalty kicks in expert goalkeepers, outfield players and novices. Video-clips of a model player executing penalty kicks were presented using a temporal occlusion paradigm. The videos were interrupted at three phases, manipulating the congruence between the kick direction suggested by the initial body movements (the direction indicated by the foot-ball contact) and the initial ball trajectory. At the end of each video-clip, participants had to indicate whether the ball would end up at the right or left side of the goal post. In the congruent condition, as expert goalkeepers and outfield players were better in perceiving the body kinematics of the model player and anticipating the fate of the kick, they outperformed novices. Compared to goalkeepers and novices, outfield players were more susceptible to being caught out by the deception created by the incongruence between the initial body kinematics and the final direction of the kick indicated by the ball trajectory. This finding thus indicates that motor experts cannot refrain from representing actions and by reading body kinematics even when these last cues are incongruent with upcoming contextual cues.

2.8.6 Neural Bases of Deception Detection in Sport In addition to behavioral evidence on detection of deception in soccer, Tomeo et al. (2013) examined the neural correlates of the behavioral finding. Similar to previous experiment, participants consisted of goalkeepers, outfield players and novices, and videos showing soccer penalty kicks presented them. Video-clips were interrupted at two phases: after the foot-ball contact or the initial ball trajectory. For half of the trials, incongruent cues were introduced, and the corticospinal motor correlates of the participants for anticipating congruent versus incongruent kicks were determined by means of single-pulse TMS. The TMS technique was used to probe excitability of muscles associated with soccer (lower leg and forearm). The analysis showed significantly differences among three groups in the corticospinal facilitation of the associated muscles. In particular, electromyography measures from lower limb muscles exhibited different levels of facilitation during the observation of incongruent trials as compared to the congruent trials. During the observation of incongruent kicks, whereas goalkeepers reduced the corticospinal facilitation of the associated muscles, novices increased it. However, outfield players showed the comparable facilitation between incongruent and congruent actions. Figure 2.8 showed that, the greater the motor facilitating during observation of incongruent

2.8 The Effect of Motor Expertise on Observational Learning in Sports

69

Fig. 2.8 Correlation between the facilitation of the corticospinal representation of leg muscles during observation of deceiving soccer penalty kicks and accuracy of outfield players, goalkeepers and novices in anticipating their actual outcome (Tomeo et al. (2013), redraw from Urgesi and Makris (2016) with permission from Cambridge University Press). The greater is the motor resonance for deceiving actions, the lower is the accuracy of outfield players in anticipating the action outcome

kick, the lower the accuracy of outfield players in anticipating the actual outcome of the ball. These results indicate that neural makers of motor simulation processes were correlated with poorer performance. Furthermore, Makris and Urgesi (2015) examined the roles of visual and motor action representations in expert athletes’ ability to anticipate the outcome of soccer actions. Similar to the experiment of Tomeo et al. (2013), expert goalkeepers, outfield players and novices were asked to anticipate the fate of penalty kicks interrupted at specific time intervals. Videos of penalty kicks executed by a model player were interrupted at foot-ball contact and contained or did not contain incongruent body kinematics. At the end of each video-clip, participants were instructed to anticipate the direction of the ball. Moreover, in order to examine the causative roles of visual and motor areas of the action observation network in the action anticipation task, repetitive TMS trains were applied over the superior temporal sulcus (STS) area and the dorsal premotor cortex (PMd) of the participants during the observation of the action. Although STS-repetitive TMS impaired performance in both experts and novices, this effect was more marked in goalkeepers with more visual experience in the task. Most importantly, PMd-repetitive TMS disrupted performance only in expert goalkeepers who exhibit strong motor expertise in soccer actions. While both experts and novices use visual representations in the posterior temporal cortex, only direct motor experience endow the onlooker’s brain with spe-

70

2 The Background of the Study on Interpersonal Coordination

cific motor-resonant mechanisms that allow for the creation of anticipatory representations of ongoing actions. In other words, while experts and novices can access visual action representation in the STS, only experts are equipped with and use internal motor representations to anticipate others’ behavior. From this finding, the review of Urgesi and Makris (2016) suggests that, although we need to embody others’ actions in order to anticipate their future actions, when facing deceptive actions, we need to flexibly inhibit such embodied representations to favor a more abstract aspect of social perception based on visual models of others’ actions.

2.9 The Effect of Shared Representation on Team Sports Traditional cognitive theory has attempted to understand decision-making skill as a normative rational process (Mellers et al. 1998). In contrast to the cognitive theory, from a dynamic systems perspective, skilled behavior consists of intentional adaptation to the interacting constraints imposed by the environment during task performance (Newell et al. 2001). From this standpoint, the role of cognitions and intentions is viewed as leading to the emergence of self-organized behavior, not explicitly controlling such processes during movement coordination (Davids et al. 2001). Self-organization of movement systems is constrained but not determined by important cognitive processes. In ball games, Davids et al. (2001) point out that skill acquisition may be seen as the emergence of movement solutions based on a better tuning to the constrains on action. Cognitive factors related to game intelligence such as anticipation, decision- making and creativity constrain player’s intentions, guiding their search for optimal task solutions. Thus, practice should provide the opportunity to search for solutions to the movement problem in a perceptual-motor workspace that is generated by the combined constraints of the learner, task and environment. Bourbousson et al. (2010) notice that the intra- and inter-couplings of playing dyads has been proposed as the basis for space-time patterns in many sports (e.g., McGarry et al. 2002). Intra-coupling refers to the linkage between two players from the same team and inter-coupling to the linkage between two players from opposing teams. Whereas sports of one vs. one comprise a single inter-coupling between two components, team sports of many vs. many offer the possibility of multiple dyads comprising both intra- and inter-coupling. The idea of coupling and layerings (coupling of coupling) proposed by McGarry et al. (2002) predicts on the unifying principle of self-organizing dynamical principles.

2.9 The Effect of Shared Representation on Team Sports

71

Fig. 2.9 Changes in radial distance from central point of court (‘T’) in squash rally performed two players. (McGarry et al. (2002), redraw from Lebed and Bar-Eli (2013) with permission from Routledge)

2.9.1 T he Effect of Shared Representation on Decision- Making in Team Ball Games According to Davids et al. (2001), McGarry et al. (2002) examined whether emergent processes characterize decision-making performance in specific sport activities such as dribbling in team ball games. Sport competition in general, and team ball sports in particular, can be considered as a dynamic system composed of many interacting parts such as players, ball, referees, and court dimensions. Macroscopic patterns of behavior from such a system spontaneously emerge from nonlinear interactions of various components at a more microscopic level of the organization. Discontinuous changes in the macroscopic order of a system, induced by continuous changes in value of a control parameter, are called a phase transition and are based on a symmetry-braking process (Kelso 1995). For example, a control parameter is a variable that can act as an important source of information for a particular system and exerts considerable influence over system stability. McGarry et al. (2002) argued that sport competition should be described in terms of control and order parameters and exhibit a general tendency to stability. McGarry et al. (2002) focused on dyadic sports such as squash. Figure 2.9 shows ‘on-field’ mutual oscillations of opponents during a match. They interpret these oscillations as ‘in-phase’ – ‘anti-phase’ changes of order parameters in critical moments of play. On the other hand, this mutual oscillation can be interpreted not

72

2 The Background of the Study on Interpersonal Coordination

Fig. 2.10 Oscillating dynamics of two competing basketball players. (Bourbousson et al. 2010, redraw from Lebed and Bar-Eli (2013) with permission from Routledge)

as the attribute of a dynamical system but as a tactical move learned by all players. After a shot, a player has to return to the central point of court in order to occupy a more universal position before receiving the next ball. Simultaneously, the receiving ball player must always move away from the central point according to the ball’ rebound. The inconsistency between anticipatory and actual placement of the ball cause the players’ oscillations. A ball rebounding properly from a wall provokes a perturbation because it is moved suddenly and far from the central point of court. This is a good example of contesting complex systems. Bourbousson et al. (2010) examined the space-time coordination dynamics of basket player dyads. They analyzed the coupled movements of dyads of defenders and attackers. The analysis showed oscillatory movement patterns in both the longitudinal and lateral direction, with each dyad traversing the court in lockstep fashion for the most part, particularly in the longitudinal direction as the game proceeds from basket to basket. The five basketball players on offense in coupling oscillation with five on defense constitute an example that applies only when man-to-man-zone defense is used in basket. However, Fig. 2.10 showed that one dyad deviated markedly from this lockstep fashion (about 15 s), with one player not returning with his opponent to the other basket. Araujo et al. (2002) examined an example from basketball to understand the application of these ideas to the study of decision-making processes in team ball games. Consider the relative positioning of an attacker with the ball and a marking defender near the basket. Such a one versus one sub-phase of invasive team ball games is very common and can be referred to as a dyad. The dyad formed by an attacker and defender comprises a system. The aim of the attacker is to destroy the stability of this system. When the defender matches the movements of his opponent and remains in position between the attacker and the basket, the form or symmetry of the system remains stable. When an attacker dribbles past an opponent, near the

2.9 The Effect of Shared Representation on Team Sports

73

Fig. 2.11 Starting position of a dyad (in grey), based on the ‘attack system 1:2:2’, in basketball. (Araujo et al. (2004) redraw from Williams and Hodges (2004) with permission from Routledge)

basket, he creates a break in the symmetry of the system. Such explanation is a typical description of interpersonal coordination from a dynamic systems approach. In basketball, Araujo et al. (2002) regarded the distance between the median point of dyad and the basket as an order parameter, and they predicted that the order parameter would change dramatically as a consequence of the attacker successfully exploring the constraints on deciding when to attack the basket. Ten players took part in the experiment and formed five dyads of attackers and defenders. Each dyad started on the free throw line, with the other members of both teams located on court based on the attack system 1:2:2 (Fig. 2.11). Instruction constraints were for the attacker to score and the defender to prevent a score, within the rule of basketball. The eight other players were passive players, only participating in the play 5 s after the beginning of the task, and keeping the positions described in Fig. 2.12. To examine symmetry breaking, the trials chosen for analysis were those where the attacker did not shoot immediately after receiving the ball, and instead, tried to dribble past the defender. The two-dimensional trajectory of the mass center of each player in the dyad was recorded by one digital camera placed behind and above the dyad’s position on the court. This study mainly examined whether phase transitions occurred in the interpersonal system formed by the attacker-defender dyads. Dyad transitions were assumed to occur at emergent decision points, identified as the moment when the attacker attempted to dribble past the defender. To illustrate this point, data contained a number of typical situations in basket dribbling. Figure 2.13 showed that, although the attacker-defender-basket system exhibited initial symmetry during the first seconds, it was broken during transition to a new state just before 4 s at a specific value of the control parameter (interpersonal distance). In other words, while the attacker was trying to dribble past the defender, the defender was attempting to maintain the initial system state. The attacker increased the number of dribbling actions (fluctuation) in order to create information on the emergence of a system transition (a decision ‘when to go’). Suddenly, the decision emerged in the ‘intention-perceiving-acting cycle’ (Kugler et al. 1990). The emergence of the decision was a result of the breaking of symmetry between the dyad. Figure 2.13 showed that very early the attacker abruptly broke the symmetry. In this case, the symmetry lasted for a very short period compared to Fig. 2.12. After being passed, the defender could not follow the progression of the

74

2 The Background of the Study on Interpersonal Coordination

Fig. 2.12 Changes in distinction between attacker and defender’s distance to basket, showing a slight attacker’s advantage. (Araujo et al. (2004) redraw from Williams and Hodges (2004) with permission from Routledge)

attacker towards the basket. By contrast, Fig. 2.14 showed that, where the defender had supremacy, the system maintained its symmetry.

2.9.2 S hared Representation of Referees and Officials in Team Ball Games In both team sports and for individuals with specific roles such as referees and match officials, shared representations and coordinated action have a big effect on performance outcomes in a wide range of sporting domains. This subsection will address these cases separately. First, in team sports, the need for shared representations and shared thinking is considered in cooperative environments. Cooperative environments are those where the outcome performance of both individual and environments is dependent on the actions of others. In such situations, performers have to see things in similar ways

2.9 The Effect of Shared Representation on Team Sports

75

Fig. 2.13 Changes in distinction between attacker and defender’s distance to basket, showing attacker’s supremacy. (Araujo et al. (2004) redraw from Williams and Hodges (2004) with permission from Routledge)

and make similar decisions if performance is to be optimized. In particular, consistency of shared representation is important for performance in team sports. By contrast, if team members see things differently, performance in team sports is worsened by different representation among teammates. Second, in refereeing of sports, an individual or small group has to apply a shared representation. For example, in soccer, rugby or tennis, a refereeing team consists of a referee, touch judges, or linesmen, the refereeing team has to apply a shared representation consistently. Here, there are explicit and implicit representations for a refereeing team. While explicit representation is guided by rules, laws or formally issued guidance notes, implicit that is generally accepted as the appropriate sanction for a particular penalty offence, depending on game circumstances, score, ‘temperature’ of the match, and so on. In addition, we have to discuss how these shared mental models are developed, monitored and modified. Third, in team sports, coaches or administrators have to give many decision- makings according to the proceeding of a game or the change in weather. In ball games, for example, a coach has to make a decision of changing members according to the proceeding of a game. A coach also has to change tactics according to the

76

2 The Background of the Study on Interpersonal Coordination

Fig. 2.14 Changes in distinction between attacker and defender’s distance to basket, showing defender’s supremacy. (Araujo et al. (2004) redraw from Williams and Hodges (2004) with permission from Routledge)

proceeding of a game. On the other hand, an administrator has to give a decision- making of whether the baseball game is on, off, or suspended because of rain. Shared mental models are essential for such a decision or selection. All team games involve cooperation between team members. The team games contain invasion games, team net games and some other sports. For example, while invasion games contain soccer, rugby and hockey, team net games contain volleyball. There are curling and relay running as examples of other sports. In all team games, shared representations are apparent in two broad but interlinking factors: perception and decision-making. A mental simulation of a teammate’s actions presumably produces these factors. In other words, players anticipate the outcome of a teammate’s action or his perception and they further compare their action and perception in the future with his action and perception. This thus links to the ideas of ‘mirror system.’ Perception is the most researched area in the psychology of team sports. What players look at and the difference between experts and novices in visual perception are an important theme of study on the relationship between perception and team sports. As a typical study of this area, Williams and Davids (1998) used a life-size

2.9 The Effect of Shared Representation on Team Sports

77

projection of soccer player, dribbling towards participants, and examined eye tracking to detect differences in focus. Naturally, experts showed different points of focus compared to novices, enabling them to more quickly and accurately anticipate the direction of play. The point here is a focus on what each player actually perceives and interprets in the display rather than merely what features he attends to. This interpretation process involves an implicit ‘weighting scale’ that not only dictate which features might convey more or less information than others but also what these particular display features suggest is going to happen. Based on this approach, coaches will seek to tease out shared meaning of particular features of the display. A process is achieved through a structured questioning approach. Richards et al. (2009, 2012) asked players to dictate critical moments in field hockey and their perceived key factors. Through the use of such reflective practice approaches, the coach can develop a shared representation as the players themselves have generated the information. As for decision-making, on the other hand, the building of a shared mental model for a team involves a subtle but crucial combination of fast and slow thinking (Kahneman 2011). The first step involves the use of the critical moments approach. This enables both coach and players to reach consensus on what are key challengers they are likely to meet, what is important to monitor, and the relative weighting scale of these factors in facilitating anticipation. What follows is a repeated cycle of fast action simulation drills, combined with slower, often video-facilitated, debrief and refinement. Given the ubiquity of team meetings, video debriefs and key point summaries in professional team sports, the suggested process for developing a team shared mental model holds at least face validity. This cyclic process generates rapid, accurate and consistent decision-making across the team. Players’ decision-making is improved as a consequence of using this approach. As the other approach to team sports, the dynamic system approach does not pay attention to the shared representation (Araujo et al. 2006; Silva et al. 2013) but to a commonality in shared affordance. Although detail descriptions of the differences between the cognitive and dynamic systems do not produce a fruitful perspective, consideration of these competing views raise the key question of how slow, considered thoughts are internalized into rapid, quick-fire, on-field decision-making in the case of team sports. Silva et al. (2013) describes the common affordances from common goals from an ecological dynamics approach to team coordination in sports. From a cognitive approach, on the other hand, Blickensderfer et al. (2010) in table tennis, Bourbousson et al. (2012) in basketball, and Gershgoren et al. (2013) in soccer indicate that shared expectations and concerns co-act with team experience and team chemistry to effect on implicit coordination. As the different components exert different influences on aspects of the team sports, distinguishing between different aspects of the shared mental model will be essential from hybrid forms of dynamics and cognitive approaches (Butterfill 2018). Because team processes are related to the knowledge structures, how shared mental models are measured is an important consideration for both research and practice (DeChurch and Mesmer- Magnus 2010).

78

2 The Background of the Study on Interpersonal Coordination

Another important factor in team performance is the concept of cognitive readiness. Cognitive readiness is described as “the state of processing the psychological (mental) and sociological (social) knowledge, skills and abilities, and attitudes needed to sustain consistent, competent professional performance and mental well- being in dynamic, complex, and unpredictable environments” (Schmorrow et al. 2012). The psychosocial construct is clearly a shared representation or shared attitude, which is crucial for high-level performance. The construct regards readiness as adaptability. In other words, the team’s capacity to be self-adjusting meets the novel demands of any task (Fiore et al. 2012). An effective team has shared representations on both the task at hand and the processes employed, with team members committed to a specific style of interaction (Eccles and Tenenbaum 2004; Eccles and Tran 2012) and team leaders showing high levels of coherence in terms of where the team is heading (Miles and Kivlighan 2008). As a particular example, referee or match official use shared representations. The referee presents and applies a consistent weighting scale to decision-making based on the rule of the game. This consistency is apparent in both the formal decision and the informal game management systems which play an essential part in producing an attractive and following game. Researches on shared representation for referees and officials have focused on the patterning and employment of visual skills, highlighting differences between elite and non-elite officials (Ghasemi et al. 2011). The role of shared representations or shared skills are examined using the flash lag effect (Helsen et al. 2006). Studies on the flash lag effect examined the comparison between two moving objects as in decisions on offside in soccer (Put et al. 2013). The flash lag effect may be more due to cognitive than perceptual reasons (Catteeuw et al. 2009). In other words, a shared understanding of its implications and how these may be countered underpin the observed effects. The complexity of these effects shows the importance of perceptual-cognitive training in developing an appropriate internal representation (Catteeuw et al. 2010). Although shared representations work in both rule-based situations and game management, Mascarenhas et al. (2005b) points out that, employing England’s top rugby referees, this game management factor is both ill-defined and acknowledged as crucial for top-end performance. An increase in the consistency of a shared representation between refereeing teams, and within each referee/touch judge/television match official team, is an important facet in raising consistency in performance. This consisted in large part of exposing the decision weighting scale demonstrated by experts, then developing this in intermediates (Mascarenhas et al. 2005a). The outcome deliverable from this process clearly is faster-flowing games, less open disagreement from players and coaches, and greater satisfaction for the paying fans. Of particular relevance to the shared representation construct, coherence in leading referees is raised from less than chance to better than 90% consistency (Mascarenhas et al. 2005c). The training did the job based on in this case on video representations from the touch judge’s perspective. In a similar fashion, offside accuracy has been improved by web-based exemplars with the weighting scale and logic presented to develop appropriate perceptual-cognitive skills in a shared mental model (Put et al. 2013).

2.10 The Effect of Shared Representation on Musical Ensemble Performance

79

Shared representations come from natural processes as well as training. In judo referees, for example, consistent but different judging of certain phases is apparent in referees with actual physical experience compared to those who do not (Dosseville et al. 2011). Similarly, Calvo-Merino et al. (2005, 2006) showed the contrasting effects on expert dancers of watching dance versus similar movements in capoeira, indicating that previous personal experience acts to develop an internal representation shared with other participants. MacMahon and Mildenhall (2012) point out that development of the official’s decision-making skills is driven from more information in a reflective manner.

2.10 T he Effect of Shared Representation on Musical Ensemble Performance Musical ensemble performance constitutes a refined form of joint action that involves the non-verbal communication of information about musical structure and expressive intentions via co-performers’ sounds and body movements. Joint action on musical performance has been often studied using piano duet (Keller et al. 2016), constituting a most progressive domain of joint action study. From a psychological perspective, ensemble performance needs precise yet flexible interpersonal coordination at the level of sensorimotor, cognitive, emotional, and social processes. Once shared goal representations, performance plans and cues are established, pianists interact with online sensorimotor and cognitive processes that facilitate precise yet flexible interpersonal coordination by allowing co-performers to anticipate, attend and adapt to each other’s actions in real time. This section thus addresses how such interpersonal coordination is facilitated by representations of shared performance goals, which are consolidated during preparation for interpersonal musical performance. Using behavioral and neural methods, previous studies provide evidences for three functional characteristics of shared musical representations. First, shared representations involve the integration of information related to one’s own part, other’s parts and the joint action outcome, while maintaining a distinction between self and other. Second, self, other, and joint action outcomes are represented in predictive internal models. Third, internal models recruit the motor system to simulate self-and other-produced actions at multiple hierarchical levels. Shared musical representations thus facilitate interpersonal coordination by dynamically embodying intended action outcomes related to the self, others, and the ensemble as a whole. To examine musical interpersonal coordination, we need to understand fundamental elements of music such as rhythm, pitch, and intensity (loudness) across individuals. Rhythm refers to the temporal patterning of sequential events. This patterning is determined by the durations of intervals between sound onsets, which form ratios such as 2:1, 3:1 and 4:1. Pitch is related to the perceived fundamental frequency of complex tones raging from low to high.

80

2 The Background of the Study on Interpersonal Coordination

2.10.1 Self-Other Integration and Segregation Although shared goals are represented and used to guide performance, a distinction between representations of parts produced by the self and others must be maintained in order to allow each performer to retain autonomous control of their own movements. This balance between self–other merging and self–other distinction entails the simultaneous integration and segregation of information from separate sources (Keller et al. 2014). Generally speaking, the human capacity for segregation relies on the ability to isolate parts that constitute a whole object or event, while the capacity for integration relies on the ability to construct a whole by grouping together a collection of parts. Integration and segregation have been studied extensively in the domains of visual and auditory perception. In traditional auditory streaming experiments (e.g. Bregman and Campbell 1971), a sequence of alternating high- and low-pitch tones is perceived by listeners to segregate into a sequence of high tones and a sequence of low tones when the pitch difference between the tones is large and tempo is fast. As a classical example, the cocktail party effect refers to the ability to attend selectively to one stream of information while ignoring others. In line with the effect, when listening to ensemble music, Bigand et al. (2000) found that it is possible to focus attention locally on a particular instrumental part, or to spread attention across parts and focus more globally on harmonic and rhythmic relationships between them. In addition, in the case of listening (Keller 1999), an individual may focus attention on a particular part while simultaneously attending to the interrelationship between parts. Similarly, ensemble performance involves concurrently paying attention to one’s own actions (high priority) and those of others (lower priority) while monitoring the overall ensemble sound. This form of divided attention is called ‘prioritized integrative attending.’ According to Keller’s review (2014), prioritized integrative attending assists ensemble musicians to integrate their own actions with others’ actions while maintaining autonomous control of their own moments. This mode of attention facilitates ensemble cohesion by allowing co- performers to adjust their actions based on the online comparison of mental representations of the ideal ensemble sound and incoming perceptual information about the actual sound. Performers are thus able to deal with changes in the momentary demands of their own parts and the relationship between their own and others’ parts in terms of timing, intensity and timbre. Keller and Burnham (2005) examined the dynamics of prioritized integrative attending using dual-task paradigms designed to capture a subset of the cognitive and motor demands of ensemble performance. In a listening task, musicians were required simultaneously to memorize a target (high priority) part and the overall aggregate structure (resulting from the combination of two complementary parts) of short percussion duets. Recognition memory for both aspects of each duet was influenced by how the target and the aggregate structure could be accommodated within the same metric framework. Next, Keller and Burnham (2005) asked professional

2.10 The Effect of Shared Representation on Musical Ensemble Performance

81

percussionists first to listen to a rhythm pattern, and then to reproduce it on a drum while listening to a concurrently presented pattern that also had to be subsequently reproduced. Reproducing the first pattern in time with the second one means sensorimotor synchronization. Taken together, Keller and Burnham (2005) indicated that musicians are able, first, to prioritize one part while monitoring the relationship between parts when listening to or producing multipart patterns and, next, to form memory representations for different levels (part and whole) of the multipart structure. Using an fMRI, Uhlig et al. (2013) examined the simultaneous segregation and integration of melody and accompaniment parts during listening to piano duos. The melody and accompaniment parts were differentiated in terms of both pitch and rhythm, and participants were asked to attend to one part while judging its temporal relationship to the other part (leading or following). The planum temporale (posterior to the primary auditory cortex) plays a role in segregation, the intraparietal sulcus supports integration, and dorsolateral prefrontal cortex and frontal gyrus regulate attention modulations of the balance between these processes. By recruiting this front-parieto-temporal network, prioritized integrative attending calls upon brain regions that subserve basic forms of auditory attention. In ensemble performance, because simultaneous self-other integration and segregation is important for the task of monitoring an ongoing performance to determine whether shared performance goals are being met, Loehr et al. (2013) examined whether pianists are able to monitor their own and their partner’s part of a duet in parallel. While pairs of pianists performed duets together, their brain activity (EEG) was recorded, and the pitches elicited by each performer’s keystrokes were occasionally altered so that an unexpected pitch was produced. Half of the altered pitches occurred in the upper part of the duet and half occurred in the lower part of the duet. Each unexpected pitch changed either the auditory outcome of one pianist’s action or the joint outcome of the pianists’ combined actions (see Fig. 2.15). The altered pitches elicited two brain responses that are commonly associated with monitoring action outcomes. First, altered pitches elicited a feedback-related negativity (FRN), a negative-going potential that signals a perceived mismatch between expected and actual outcomes and peaks approximately 250 ms after the unexpected action outcome (Oliveira et al. 2007). Equivalent FRNs were elicited whether the altered pitch occurred in the pianist’s own part or the partner’s part, indicating that pianists monitored their own actions and their partners’ actions in parallel. Second, altered pitches elicited a P300 response, a positive-going potential that peaks 300-600 ms after the action outcome. The P300 occurs relatively late in the processing stream and its amplitude is thought to reflect the perceived significance of the unexpected outcome (Nieuwenhuis et al. 2005). Figure 2.15 shows that while the amplitude of the P300 was larger in response to altered pitches that occurred in the pianist’s own part compared to the partner’s part, it was larger in response to altered pitches that changed the joint outcome compared to an individual outcome. Thus, whereas the FRN findings indicated integration of own and others’ part at an early stage of action monitoring, the P300 findings indicate that

82

2 The Background of the Study on Interpersonal Coordination

Fig. 2.15 Brain responses to unexpected pitches for integration and segregation of self and other during action monitoring (Loehr et al. (2013), redraw from Keller et al. (2016) with permission from Cambridge University Press). Top panel: the first half of a duet. One pianist performed the upper art of the duet and the other performed the lower part. Symbols labeled ‘learned harmony’ represent the harmony given in the score. Symbols labeled ‘alteration harmony’ represent the harmony introduced by the altered pitch. (A) Individual outcome altered without changing the harmony of the chord. (B) Joint outcome altered by changing the harmony of the chord. Bottom Panel: brain responses to altered pitches. Left side: the feedback-related negativity. Right side: the P300

pianists differentiate between their own and others’ action outcomes, and between individual and joint action outcomes, at later stage of processing. In addition to monitoring joint musical outcomes, employing a virtual piano duo paradigm, Novembre et al. (2012) examined the representation of self- and other- related actions in the human motor system using a single-pulse TMS. Skilled pianists were asked to learn several piano pieces bimanually before coming to the laboratory. The right-hand part contained a melody line and the left-hand part contained a complementary baseline. In the laboratory a few days later, the pianists were required to perform the right-hand part of each piece while the left-hand part was either not performed or believed to be played by another pianist hidden behind a screen. The experiment consisted of two sessions. While one session was that pianist could hear feedback of their actions as well as the recording, a subsequence session was that the pianists received no feedback but were still aware of the presence of the co-performer behind the screen. In both sessions, single-pulse TMS was applied over the right primary motor cortex to elicit motor-evoked potentials

2.10 The Effect of Shared Representation on Musical Ensemble Performance

83

(MEPs), which were measured from a left forearm muscle that would normally be used to perform the complementary part. It was assumed that bimanual learning of the piece would lead to co-representation of the left-hand part, and would then be associated either with the self or with the other player. Consistent with this assumption, differences in MEP suggested that distinct patterns of cortico-spinal excitability (inhibition and excitation) were associated with the representation of self and other, respectively. These results did not change as a function of whether or not the pianists received auditory feedback about their own performance or the ‘hidden partner’. Inhibition and excitation of the motor system are regarded as markers of the functional segregation of parts performed by self versus other during musical ensemble performance. The finding that this segregation occurred in the mute session suggests that these agent-specific representations arise in response to the potential for interaction with another, and may thus be intrinsically social in nature. Furthermore, Fairhurst et al. (2014) examined effects of the leader-follower relationship as a social factor on the balance of self-other integration and segregation during musical sensorimotor synchronization using an fMRI. Participants were asked to synchronize finger taps with sounds produced by virtual partners who varied in terms of competence at keeping a steady tempo. The human participant must take responsibility for keeping the tempo when the virtual partner cannot. Leaders engaged in less adaptive timing of their taps to virtual partner’s timing than followers. While followers prioritized the task of synchronizing with their partner (self- other integration), leaders focused on stabilizing the tempo of their own performance (self-other segregation). Generally, while the pre-supplementary motor area is activated by self-initiated action, the precuneus is activated by the evolution of agency. The two brain regions were activated more strongly in leaders than followers.

2.10.2 I nternal Models for Self, Other and Joint Action Outcome To support the claim that musicians form internal models of their partner’s part of a duet, Loehr and Palmer (2011) and Palmer and Loehr (2013) compared pianists’ performances of a right-hand melody when paired with a left-hand accompaniment produced by themselves or by a partner. The left-hand accompaniments were either simple or complex. When the pianists performed both parts themselves, the right- hand melody was produced more slowly and with a more pronounced temporal grouping structure than when paired with a simple compared to complex left-hand accompaniment. The same pattern of differences occurred when the left-hand accompaniment was produced by a partner. Thus, pianists’ internal models of the partner’s part modified the production of their own part of a joint performance. Ragert et al. (2013) also examined whether musicians form internal models of their partner’s part of a duet. Pairs of pianists conducted experiments in the laboratory

84

2 The Background of the Study on Interpersonal Coordination

after practicing either one part or both parts of several piano duets at home. The complementary parts of the duets were thus familiar in one condition and unfamiliar in the other. While pianists played repeat performance across six takes in each condition, their keystroke timing was recorded on digital pianos, and their body movements were tracked with a motion-capture system. The analysis showed a partial dissociation between interpersonal coordination at the level of keystrokes and body sway. Variability in keystroke asynchronies decreased across the takes, and was generally lower in the unfamiliar condition than the familiar condition. This indicated that coordination started out better, and remained so, when pianists had not rehearsed their co-performer’s part. By contrast, body sway coordination was high through-out the takes in the familiar condition, while it stared out low and improved across takes in the unfamiliar condition. These findings suggest that knowledge affects interpersonal coordination by influencing anticipations at different timescales. Familiarity with a co-performer’s part results in anticipations about expressive micro-timing variations that are based instead upon one’s own personal playing style, leading to a mismatch between anticipations and actual events at short timescales. As knowledge about a co-performer’s style is acquired, however, the individual learns to simulate the other’s action style through the calibration of internal models. On the other hand, familiarity with the structure of co-performer’s part facilitates anticipations at longer timescales related to high-level metric units and musical phrases, and is reflected in ancillary body sway movements. Furthermore, Novembre et al. (2014) examined how disrupting the cortical brain network underlying action simulation affects the ability of pianists to adapt to tempo changes in familiar and unfamiliar parts during virtual duet performance. Participants were asked to play the right-hand part of piano pieces in synchrony with a recording of the complementary left-hand part, which had or had not been practiced beforehand (Fig. 2.16a). The recordings of the left-hand part contained occasional tempo changes, to which the pianists had to adapt in order to maintain synchrony. In critical condition, these tempo changes were preceded by double-pulse TMS delivered over the right primary motor cortex to interfere with simulation of the left-hand part. TMS impaired tempo adaptation when the left-hand part had been previously trained, but not when the part was untrained, suggesting that tempo adaptation may be underpinned by a different brain network in this case. The pianists who were susceptible to TMS-induced interference with tempo adaptation scored highly on the perspective-taking subscale of an empathy questionnaire (Fig. 2.16d).

2.10.3 Motor Simulation of Self and Other Internal models recruit an ensemble musician’s motor system to simulate co- performers’ actions. Representations of co-performers’ actions may be influenced by one’s own action repertoire and action style. ‘Action repertoire’ refers to the set of musical structures that an individual performer is potentially able to produce given his technical mastery of his instrument. ‘Action style’ refers to the ways of

2.10 The Effect of Shared Representation on Musical Ensemble Performance

85

Fig. 2.16 Schematic representation of the summary studied by November (2014, redraw from Keller et al. (2016) with permission from Cambridge University Press). (a) experimental task. (b) experimental design. (c) tempo adaptation index across conditions. The dashed line shows perfect adaptation, values above or below show deceleration or acceleration, respectively, of the produced melody with respect to the baseline part presented by the computer. (d) correlation between TMS- interference to motor simulation and the perspective-taking (empathy) score of each participant

producing these structures in terms of fluctuations in event micro-timing and intensity. Action style may vary due to a mixture of factors, including learning history, aesthetic preference (Repp 1997), level of expertise (Repp 1995) and biomechanical properties related to individuals’ physical characteristics (Keller 2014). Here, we examine how the effects of action simulation on interpersonal coordination in ensembles are modulated by the degree of overlap in co-performers’ action repertoires and the similarity of their preferred action styles. Synchronizing with a recording of one’s own performance presents a special case of perfect overlap in action repertoire and action style because the sensorimotor system engaged in action simulation is the same system that produced the action

86

2 The Background of the Study on Interpersonal Coordination

in the first place. On such self-synchronization, Keller et al. (2007) asked pianists to record one part from several duets. Seven months later, the pianists were asked to play the complementary part in synchrony with either their own or others’ recordings. As expected, synchronization was best when the pianists played with their own recordings. This finding indicated that pianists anticipated the timing of sounds in the recordings by engaging in online simulation of the performances. Such simulation led to a self-synchronization advantage because the match between simulated timing and actual timing in a complementary part was best when both were products of the same sensorimotor system. In addition, correlations between timing profiles of duet parts recorded separately by the same pianist showed that the self- synchronization advantage was not merely due to a priori self-similarity in performance technique. This corroborates that overlap in action repertoire and action style facilitates interpersonal coordination by enabling the accurate simulation of other co-performers’ actions. Moreover, the self-synchronization advantage suggests that ensemble musicians with similar action styles should be better able to simulate each other’s actions when they perform alone and thus better able to coordinate with each other when they perform together. To confirm this claim, Loehr and Palmer (2011) examined whether performers who are more similar to each other in terms of their preferred solo performance tempi are better able to coordinate with each other during duet performance. Each pianist of a pair was first asked to perform a simple melody alone. The pairs then performed duets together, and the degree to which the pianists were able to synchronize with each other and mutually adapt to fluctuations in each other’s timing was measured. Pairs who were better matched in solo performance tempi were better able to synchronize with each other and displayed mutual adaptation. By contrast, pairs who were less well-matched produced larger asynchronies and exhibited a tendency for one person to track the other’s timing but not vice versa. Thus, coordination between partners was anticipated by the degree of similarity between their preferred solo tempi, but not by either partner’s solo tempo considered alone. This suggests that performers who are better matched are better able to anticipate about each other’s timing. Thus, compatibility in action style may occur at multiple timescales from local micro-timing to global tempo. The way in which similarity in action style influences motor simulation and the process of anticipation at these different timescales depends to some degree on the amount of overlap in action repertoire. Actions such as a pianist’s keystrokes can be simulated accurately only when the actions are in the observer’s behavioral repertoire. Haueisen and Knösche (2001) reported that regions of primary motor cortex that represent specific fingers (thumb and little fingers) become active when pianists listen to sounds that would be played by those particular fingers in the context of a specific musical piece. Thus, brain activations associated with motor simulation is modulated by the degree to which an individual is experienced at producing the movements required to produce the heard sounds.

2.10 The Effect of Shared Representation on Musical Ensemble Performance

87

2.10.4 Musical Synchronization and Social Interaction Music perceptual-motor system and human social understanding are linked in many ways. Humans often spontaneously synchronize their movements to music. Such music synchrony is a typical form of social interactions and shared representation (For a reviw, Waclawik et al. 2016). The understanding of another person’s mental state is fundamental to intentional synchrony, and it may be mediated by mirror neurons. Tognoli et al. (2007) have found that synchronous and asynchronous tapping are associated with different patterns of EEG oscillations. Tognoli and colleagues suggest that, whereas the EEG oscillations may represent enhancement of the mirror neuron system during synchrony, they represent inhibition of the same system during asynchrony. Lindenberger et al. (2009) have also found similar EEG oscillations in pairs of guitarists performing a duet. Such EEG oscillations are a plausible candidate for a neural mechanism of interpersonal synchronization because they are fast enough to allow for quick adjustments to another person’s actions and are related to muscle activity. Although EEG does not allow for precise spatial resolution, synchronization perhaps occurs both in sensorimotor areas and in regions related to social cognition. Thus, the mirror neuron system may be activated during joint music-making, leading to an understanding of others’ actions and promoting a more general understanding of others’ mental states that supports empathy. Synchronized activity increases ‘team sprit’ even when there is no collective musical goal, such as when two people independently move or sing to separate stimuli which happen to have the same beat. Synchronized musical behavior may simply produce thoughts and effects that are similar enough to elicit shared representations. Coincidental synchronization may also lead to prosocial bonding by blurring of self-other distinctions. Gallese and Goldman (1998) have found that, when a person does not move, and observes another’s movement, his motor areas may activate through the mirror neuron system. However, during interpersonal synchrony, sensory feedback that is tightly coupled to one’s own movement is received from both self and other, weakening the self-other boundary (Hove 2008). Although mimicking another’s actions can also increase social connectivity (Chartrand and Bargh 1999), its effects are predicted to be weaker than those of synchronization, as mimicking introduces a temporal delay between the shared actions. Synchronized music-making activates reward areas in the brain, such as the caudate nucleus. Kokal et al. (2011) have found an increase in right caudate nucleus activity when drumming with a synchronous partner compared to an asynchronous one. The level of caudate activity anticipates prosocial behavior toward the synchronous partner. The caudate nucleus has been known to be important for synchronization, response to rewards, decision making that involves taking reward experiences into account, and prosocial behavior. Thus, synchronized music-making may be rewarding, which leads to a positive affiliation with the synchronous partner and an

88

2 The Background of the Study on Interpersonal Coordination

increased tendency to help that partner. The activation in a reward area of the caudate could explain why synchronized music-making is enjoyable. Furthermore, the relation of the caudate activity to prosocial behavior could explain why synchronized behavior results in group cohesion. In addition to caudate activity, the release of endorphins might mediate the rewarding effect of synchronous behavior. Dunbar et al. (2012) have reported that endorphins are related to reward and important for social relationships in primates. Cohen et al. (2010) have found that endorphins are also released during synchronized activity, such as rowing, at levels above those released during physical activity. Passive music listening does not produce the same effect on pain thresholds. As endorphins are important for social interactions, synchronized activity facilitates social bonding. Thus, activation of neural reward areas and release of endorphins may occur during synchronized musical performance, facilitating social bonding and contributing to the behavioral effects described above.

2.10.5 Neural Mechanism of Synchronizing to Music Because most young infants spontaneously move to rhythm (Hannon and Trehub 2005), the connection between music and movement can be seen as early as infancy. Even in people who do not have musical training, they automatically respond to the beat of a drum. The relationship between movement and beat perception occurs at the neural level. Listening to rhythms with a regular beat activates brain regions associated with movement (Grahn and Brett 2007). Motor areas include the supplementary motor area, dorsal premotor cortex, the basal ganglia, and the cerebellum. Superior temporal gyrus as an auditory area is also activated. The basal ganglia seem to be particularly important for ‘feeling for the beat.’ The basal ganglia are activated by beat-based rhythms higher than by non-beat-based (irregular) rhythms, suggesting that the basal ganglia may be involved in sensing and tracking the beat (Grahn and Brett 2007). This finding is supported by studies on patients with Parkinson’s disease. Parkinson’s patients have deficiencies in basal ganglia function, resulting in movement impairments such as slow movement initiation, ‘freezing’ during walking and shuffling of steps (Knutsson 1972). Parkinson’s patients also have impairments in discriminating changes in beat-based rhythms, but not non-beat-based rhythms (Grahn and Brett 2009). These data suggest that the basal ganglia not only activate during beat perception, but are necessary for normal beat perception to occur. Thus, the finding that brain regions that control movements are automatically activated during beat perception may account for spontaneous desire to move when listening to rhythms. Although beat perception enables synchrony in the auditory domain, people are also capable of synchronizing movements using the visual modality. For example, a musician synchronizes his playing to the cues of a conductor. When musicians on stage clap their hands, the synchronous movements of the audience involve both the auditory modality (hearing the clap) and the visual modality (watching the

2.10 The Effect of Shared Representation on Musical Ensemble Performance

89

musicians clap). Although beat perception occurs for auditory rhythms as well as visual rhythms, it is perceived and discriminated in auditory rhythms more accurately than in visual rhythms. For example, Repp and Penel (2004) found that people show poor synchronization performance when tapping with visual than auditory sequences. Grahn (2012) also reported that beat perception appears weaker, or harder to indicate, in visual modality than the auditory modality. Although the ability to rapidly identify and synchronize to the beat may be unique to human among primates, there is an individual difference in the beat perception and production abilities. Most people appear to experience the automatic desire to move or clap along to a song. However, all individuals are unable to clap along with accuracy. The causes for these differences are unclear. Music training can account for some of these differences, but not entirely. For example, Grahn and Schuit (2012) reported that, whereas many individuals with no musical training perform well on the rhythmic tasks, musicians sometimes perform poor, suggesting that differences in short-term memory capacity and sensitivity to beat also account for some variation in rhythm abilities. Furthermore, while mirror neurons respond to both the performance of an action by oneself and observation of that action performed by another, Kohler et al. (2002) found that premotor neurons also respond to an action and the sound of that action in the monkey. Using an fMRI in humans, Aziz-Zadeh et al. (2004) found that premotor neurons in the mirror neuron system respond to both observing and hearing an action. An early function proposed for mirror neurons, understanding the intention of others’ actions (Iacoboni et al. 2005), may be extended to human social interaction such as empathy, theory of mind, and discriminating between self and other. Presumably, the motor component of the mirror neuron system associated with action and observation and the emotional empathic component associated with attributing emotions in others interact with musical activity (Molnar-Szakacs and Overy 2006). Music production is associated with many motor activities. Hitting a drum or modulating movements of vocal tract is raised as an example. This sort of motor activity may involve the same sort of mirror neuron activity as the simple reaching and grasping movements. Activity in the mirror neuron system also appears to be modulated by musical experience. Haueisen and Knösche (2001) found that activity of the motor cortex occurs in pianists greater than in non-pianists when listening to piano music. This system is so specific that even the appropriate finger region is activated when hearing notes that have been played by that specific finger. Haslinger et al. (2005) found that activity of auditory areas also shows in pianists stronger than in non-pianists when observing piano movements without auditory feedback. These findings suggest that there is an auditory-visual-motor mirror neuron system that develops with musical training. Across training, repeated coupling of motor outputs and auditory feedback presumably forms a motor representation. As for effects of training on formation of motor representation, the linkage between auditory and motor representations comes online after even a short training period. Bangert et al. (2006) found increased activation in auditory cortex in response to silent key pressing and activation of motor cortex in response to passive

90

2 The Background of the Study on Interpersonal Coordination

listening when non-musicians were trained over 5 weeks to imitate simple piano key melodies. The activation patterns further differed between the passive listening and silent motion tasks before training, but became more similar over the course of training. This indicates that a transmodal network is formed in response to music in the auditory, visual or motor domain. McIntosh et al. (1998) early found that the occipital cortex shows greater activation by presentation of a tone that previously signaled a visual cue, indicating the formation of audiovisual association. The existence of common activation associated with listening to, observing and performing music suggests how a music producer and a music listener can develop a shared representation of the same musical experience (Molnar-Szakacs and Overy 2006). Shared representations may also develop between student and teacher during musical training. In imitation learning of guitar chords, Buccino et al. (2004) have reported that activation of mirror neurons regions occurs during both the observation and execution stages, suggesting that, even during visual observation, people begin to mentally prepare the motor output necessary for imitation. Thus, a shared representation may allow those who listen to music to enter the mind of the producer and to gain an understanding of the intentions behind the actions of the producer. In addition to the linkage between sensory and motor representations, shared representation provide a hypothetical mechanism for musically-induced emotion. Human actions are expressive. For example, footsteps can have an emotion attributed to actions. Actions thus lend themselves to interpretation regarding the emotional state of executor, and listening to music may elicit simple theory of mind interpretations of the mental state of musician, composer or producer (Molnar- Szakacs and Overy 2006). The recognition of emotion in music engages the same brain areas that are active during theory of mind processing or recognizing the emotional states of others, such as the anterior medial frontal cortex, superior temporal sulcus and temporal poles (Frith and Frith 2003). This effect appears to be related to attributions about the mental state of the composer or producer because these neural regions are only activated when participants believe the music is composed by another person, and not when they believe it is computer-generated (Steibeis and Koelsch 2009). Damage in these areas is associated with deficits in theory of mind abilities, and have been shown to also experience difficulty in attributing emotions to music (Downey et al. 2013). When listening to music, people may use theory of mind to identify the emotion in the music (Downey et al. 2013). Practically applying, we begin to use music as a therapy to improve theory of mind. Similar to dance movement therapy, music therapy holds that bodily similarity produces psychological understanding and fosters empathy (Behrends et al. 2012). Although children with autism are able to attribute emotion to music (Heaton et al. 1999), they show significant impairments on theory of mind tasks (Baron- Cohen et al. 1985). This shows a dichotomy between the ability to understand emotional mood and theory of mind abilities which are though to be related to affective identification in music (Downey et al. 2013). However, the dichotomy suggests that music may have potential for improving social cognition.

References

91

Carr et al. (2003) point out that coupling of affective and sensorimotor systems may further lead to musically-induced empathy. The posterior inferior frontal gyrus is commonly activated during musically-evoked emotional states and appears to play a role in sensorimotor affective coupling that occurs in understanding the emotions of others. The anterior insula is also activated in music-modulated emotion and it has been proposed to be the relay station between the mirror neuron system and the limbic system, which mediates many basic emotion. Thus, the mirror neuron system and limbic system may communicate via the insula, adding an emotional interpretation to the perceptual and motor processing of the stimulus (Molnar- Szakacs and Overy 2006).

References Abreu AM, Mascaluso E, Azevedo RT, Cesari P, Urgesi C, Aglioti SM (2012) Action anticipation beyond the action observation network: a functional magnetic resonance imaging study in expert basketball players. Eur J Neurosci 35:1646–1654 Aflalo TN, Graziano MSA (2006) Partial tuning of motor cortex neurons to final posture in a free- moving paradigm. Proc Natl Acad Sci 103:2909–2914 Aflalo TN, Graziano MSA (2007) Relationship between unconstrained arm movement and single neuron firing in the macaque motor cortex. J Neurosci 27:2760–2780 Aglioti SM, Cesari P, Romani M, Urgesi C (2008) Action anticipation and motor resonance in elite basketball players. Nat Neurosci 11:1109–1116 Aichhorn M, Perner J, Kronbichler M, Staffen W, Ladurner G (2005) Do visual perspective tasks need theory of mind? Neuroimage 30:1059–1068 Alaerts K, Senot P, Swinnen SP, Craighero L, Wenderoth N, Fadiga L (2010) Force requirements of observed object lifting are encoded by the observer’s motor system: a TMS study. Eur J Neurosci 31:1144–1153 Albert NB, Robertson EM, Miall RC (2009) The resting human brain and motor learniong. Curr Biol 19:1023–1027 Amodio DM, Frith CD (2006) Meeting of minds: the medial frontal cortex and social cognition. Nat Rev Neurosci 7:268–277 Andrew RJ (1962) The origin and evolution of the calls and facial expressions of the primates. Behaviour 20:1–107 Apperly IA, Samson D, Chiavarino C, Humphreys GW (2004) Frontal and temporo-parietal lobe contributions to theory of mind: neuropsychological evidence from a false-belief task with reduced language and executive demands. J Cogn Neurosci 16:1773–1784 Araujo D, Davids K, Sainhas J, Fernandes O (2002) Emergent decision-making in sport: a constraints- led approach. Communication to the International Congress on Movement, Attention and Perception, Poitiers: France Araujo D, Davids K, Bennett SJ, Button C, Chapman G (2004) Emergence of sport skills under constraints. In: Williams AM, Hodges NJ (eds) Skill acquisition in sport: research, theory and practice. Routledge, London, pp 409–433 Araujo D, Davids K, Hristovski R (2006) The ecological dynamics of decision making in sport. Psychol Sport Exerc 7:653–676 Arbib MA, Rizzolatti G (1999) Neural expectations: a possible evolutionary path from manual skills to language. In: Loocke PV (ed) The nature of concepts, evolution, structure and representation. Routledge, New York, pp 128–154 Ashford D, Davids K, Bennett SJ (2007) Developmental effects influencing observational modeling: a meta-analysis. J Sports Sci 25:547–558

92

2 The Background of the Study on Interpersonal Coordination

Ashton-James C, Chartrand TL (2009) Social cues for creativity: the impact of behavioral mimicry on convergent and divergent thinking. J Exp Soc Psychol 45:1036–1040 Atmaca S, Sebanz N, Knoblich G (2011) The joint Flanker effect. Exp Brain Res 211:371–385 Avenanti A, Annella L, Candidi M, Urgesi C, Aglioti SM (2013a) Compensatory plasticity in the action observation network: virtual lesions of STS enhance anticipatory simulation of seen actions. Cereb Cortex 23:570–580 Avenanti A, Candidi M, Urgesi C (2013b) Vicarious motor activation during action perception: beyond correlational evidence. Front Hum Neurosci 7:185 Aziz-Zadeh L, Iacoboni M, Zaidel E, Wilson S, Mazziotta J (2004) Left hemisphere motor facilitation in response to manual action sounds. Eur J Neurosci 19:2609–2612 Bangert M, Peschel T, Schlaug G, Rotte M, Drescher D, Hinrichs H, Heinze HJ, Altenmüller E (2006) Shared networks for auditory and motor processing in professional pianists: evidence from fMRI conjunction. NeuroImage 30:917–926 Bardi L, Brass M (2016) The control of shared representations and social cognition. In: Obhi SS, Cross ES (eds) Shared representations. Cambridge University Press, Cambridge, pp 151–170 Baron-Cohen S, Leslie AM, Frith U (1985) Does the autistic child have a ‘theory of mind’? Cognition 21:37–46 Barsade SG (2002) The ripple effect: emotional contagion and its influence on group behavior. Admin Sci Q 47:644–675 Behrends A, Müller S, Dziobek I (2012) Moving in and out of synchrony: a concept for a new intervention fostering empathy through interactional dance and movement. Arts Psychother 39:107–116 Beilock SL, Lyons IM, Mattarella-Micke A, Nusbaum HC, Small SL (2008) Sports experience changes the neural processing of action language. Proc Natl Acad Sci U S A 105:13269–13273 Bernardi NF, Darainy M, Bricolo E, Ostry DJ (2013) Observing motor learning produces somatosensory change. J Neurophysiol 110:1804–1810 Bigand E, McAdams S, Foret S (2000) Divided attention in the listening of polyphonic music. Int J Psychol 35:270–278 Bird CM, Castelli F, Malik O, Frith U, Husain M (2004) The impact of extensive medial frontal lobe damage on ‘Theory of Mind’ and cognition. Brain 127:914–928 Blakemore SJ, Decety J (2001) From the perception of action to the understanding of intention. Nat Rev Neurosci 2:561–567 Blakemore SJ, Bristow D, Bird G, Frith C, Ward J (2005) Somatosensory activations during the observation of touch and a case of vision-touch synaesthesia. Brain 128:1571–1583 Blickensderfer EL, Reynolds D, Salas E, Cannon-Bowers JA (2010) Shared expectations and implicit coordination in tennis doubles teams. J Appl Sport Psychol 22:486–499 Bonazzi L, Viaro R, Lodi E, Canto R, Bonifazzi C, Franchi G (2013) Complex movement topography and extrinsic space representation in the rat forelimb motor cortex as defined by long- duration intracortical microstimulation. J Neurosci 33:2097–2107 Borroni P, Montagna M, Cerri G, Baldissera F (2005) Cyclic time course of motor excitability modulation during the observation of a cyclic hand movement. Brain Res 1065:115–124 Borroni P, Montagna M, Cerri G, Baldissera F (2008) Bilateral motor resonance evoked by observation of a one-hand movement: role of the primary motor cortex. Eur J Neurosci 28:1427–1435 Bourbousson J, Carole S, McGarry T (2010) Space-time coordination dynamics in basketball: part 1. Intra- and inter-coupling among player dyads. J Sports Sci 28:339–347 Bourbousson J, Poizat G, Saury J, Seve C (2012) Temporal aspect of team cognition: a case study on concerns sharing within basketball. J Appl Sport Psychol 24:224–241 Bourgeois P, Hess U (2008) The impact of social context on mimicry. Biol Psychol 77:343–352 Brault S, Bideau B, Craig CM, Kulpa R (2010) Balancing deceit and disguise: how to successfully fool the defender in a 1 vs. 1 situation in rugby. Hum Mov Sci 29:412–425 Bregman AS, Campbell J (1971) Primary auditory stream segregation and perception of order in rapid sequences of tones. J Exp Psychol 89:244–249 Brothers L (1990) The social brain: a project for integrating primate behavior and neurophysiology in a new domain. Concepts Neurosci 1:27–51

References

93

Brown JW, Braver TS (2005) Learned predictions of error likelihood in the anterior cingulate cortex. Science 307:1118–1121 Brown LE, Wilson ET, Gribble PL (2009) Repetitive transcranial magnetic stimulation to the primary motor cortex interferes with motor learning by observation. J Cogn Neurosci 21:1013–1922 Brozzoli C, Gentile G, Bergouignan L, Ehrsson HH (2013) A shared representation of the space near oneself and other in the human premotor cortex. Curr Biol 23:1764–1768 Buccino G, Vogt S, Ritzl A, Fink GR, Zilles K, Freund HJ, Rizzolatti G (2004) Neural circuits underlying imitation learning of hand actions: an event-related fMRI study. Neuron 42:323–334 Buccino G, Baumgaertner A, Colle L, Buechel C, Rizzolatti G, Binkofski F (2007) The neural basis for understanding non-intended actions. NeuroImage 36(Suppl 2):T119–T127 Buchel C, Morris J, Dolan RJ, Friston KJ (1998) Brain systems mediating aversive conditioning: an event- related fMRI study. Neuron 20:947–957 Buckingham G, Wong JD, Tag M, Gribble PL, Goodale MA (2014) Observing object lifting errors modulates cortico-spinal excitability and improves object lifting performance. Cortex 50:115–124 Butterfill SA (2018) Coordinating joint action. In: Ludwig K (ed) The Routledge handbook on collective intentionality. Routledge, London, pp 68–82 Byrne R, Whiten A (eds) (1988) Machiavellian intelligence. Oxford University Press, Oxford Cacioppo JT, Cacioppo S (2013) Social neuroscience. Perspect Psychol Sci 8:667–669 Calvo-Merino B, Glaser DE, Grezes J, Passingham RE, Haggard P (2005) Action observation and acquired motor skills: an fMRI study with expert dancers. Cereb Cortex 15:1243–1249 Calvo-Merino B, Grezes J, Glaser DE, Passingham RE, Haggard P (2006) Seeing or doing? Influence of visual and motor familiarity in action observation. Curr Biol 16:1905–1910 Calvo-Merino B, Erhrenberg S, Leung D, Haggard P (2010) Experts see it all: configural effects in action observation. Psychol Res 74:400–406 Canal-Bruland R, van der Kamp J, van Kesteren J (2010) An examination of motor and perceptual contributions to the recognition of deception from others’ actions. Hum Mov Sci 29:94–102 Carr L, Iacoboni M, Dubeau M, Mazziotta JC, Lenzi GL (2003) Neural mechanisms of empathy in human: a relay from neural systems for imitation to limbic areas. Proc Natl Acad Sci U S A 100:5497–5502 Caruana F, Jezzini A, Sbriscia-Fioretti B, Rizzolatti G, Gallese V (2011) Emotional and social behaviors elicited by electrical stimulation of the insula in the macaque monkey. Curr Biol 21:195–199 Casile A, Giese MA (2006) Nonvisual motor training influences biological motion perception. Curr Biol 16:69–74 Caspers S, Zilles K, Laird AR, Eickhoff SB (2010) ALE met-analysis of action observation and imitation in the human brain. NeuroImage 50:1148–1167 Catmur C, Heyes C (2013) Is it what you do, or when you do it? The roles of contingency and similarity in pro-social effects of imitation. Cogn Sci 37:1541–1552 Catteeuw P, Helsen W, Gilis B, Van Roie E, Wagemans J (2009) Visual scan patterns and decision- making skills of expert assistant referees in offside situations. J Sport Exerc Psychol 31:786–797 Catteeuw P, Gilis B, Wagemans J, Wagemans W (2010) Perceptual-cognitive skills in offside decision making: expertise and training effects. J Sport Exerc Psychol 32:828–844 Chartrand TL, Bargh JA (1999) The chameleon effect: the perception-behavior link and social interaction. J Pers Soc Psychol 76:893–910 Chartrand TL, Lakin JL (2013) The antecedents and consequences of human behavioral mimicry. Annu Rev Psychol 64:285–308 Cheng CM, Chartrand TL (2003) Self-monitoring without awareness: using mimicry as a nonconcious affiliation strategy. J Pers Soc Psychol 85:1170–1179 Clutton-Brock TH, Harvey PH (1980) Primates, brains and ecology. J Zool 190:309–323 Cohen EEA, Ejsmond-Frey R, Knight N, Dunbar RIM (2010) Rowers’ high: behavioral synchrony is correlated with elevated pain thresholds. Biol Lett 6:106–108

94

2 The Background of the Study on Interpersonal Coordination

Coricelli G, Critchley HD, Joffily M, O’Doherty JP, Sirigu A, Dolan RJ (2005) Regret and its avoidance: a neuroimaging study of choice behavior. Nat Neurosci 8:1255–1262 Correia V, Araujo D, Craig CM, Passos P (2011) Prospective information for pass decisional behavior in rugby union. Hum Mov Sci 30:984–997 Correia V, Araujo D, Cummins A, Craig CM (2012) Perceiving and acting upon space in a VR rugby task: expertise effects in affordance detection task achievement. J Sport Exerc Psychol 34:305–321 Craig CM (2013) Understanding perception and action in sport: how can virtual reality technology help? Sports Technol 6:161–169 Craig CM, Bastin J, Montagne G (2011) How information guides movement: intercepting curved free kicks in soccer. Hum Mov Sci 30:931–941 Craig CM, Goulon C, Berton E, Rao G, Fernandez L, Bootsma RJ (2009) Optic variables used to judge future ball arrival position in expert and novice soccer players. Atten Percept Psychophys 71:515–522 Cross ES, Calvo-Merino B (2016) The impact of action expertise on shared representations. In: Obhi SS, Cross ES (eds) Shared representations. Cambridge University Press, Cambridge, pp 541–562 Cross ES, Hamilton AF, Grafton ST (2006) Building a motor simulation de novo: observation of dance by dancers. Neuroimage 31:1257–1267 Cross ES, Kraemer DJ, Hamilton AF, Kelley WM, Grafton ST (2009) Sensitivity of the action observation network to physical and observational learning. Cereb Cortex 19:315–326 Cunningham WA, Johnson MK, Raye CL, Gatenby JC, Gore JC, Banaji MR (2004) Separable neural components in the processing of black and white faces. Psychol Sci 15:806–813 Dalton AN, Chartrand TL, Finkel EJ (2010) The sachema-drive chameleon: how mimicry affects executive and self-regulatory resources. J Pers Soc Psychol 98:605–617 Damasio H, Tranel D, Grabowski T, Adolphs R, Damasio A (2004) Neural systems behind word and concept retrieval. Cognition 92:179–229 Darwin C (1872) The expression of emotions in man and animals. John Murray, London Davids K, Williams AM, Button C, Court M (2001) An integrative modeling approach to the study of intentional movement behavior. In: Singer R, Housenblas H, Janelle C (eds) Handbook of sport psychology, 2nd edn. John Wiley, New York, pp 144–173 Dechurch LA, Mesmer-Magnus JR (2010) Measuring shared team mental models: a met-analysis. Group Dynamics: Theory, Research, and Practice 14:1–14 Desmurget M, Song Z, Mottolese C, Sirigu A (2013) Re-establishing the merits of electrical brain stimulation. Trends Cogn Sci 17:442–449 Dessing JC, Craig CM (2010) Bending it like Beckham: how to visually fool the goalkeeper. PLoS One 5:e13161 Doerrfeld A, Sebanz N, Shiffrar M (2012) Expecting to lift a box together makes the box look lighter. Psychol Res 76:467–475 Dolan RJ (2002) Emotion, cognition, and behavior. Science 298:1191–1119 Dosseville F, Laborde S, Raasb M (2011) Contextual and personal motor experience effects in judo referees’ decision. Sport Psychol 25:67–81 Dovidio JF, Ellyson SL (1982) Decoding visual dominance: attentions of power based on relative percentages of looking while speaking and looking while listening. Soc Psychol Q 45:106–113 Downey LE, Blezat A, Nicholas J, Omar R, Golden HL, Mahoney CJ, Crutch SJ, Warren JD (2013) Mentalising music in frontotemporal dementia. Cortex 49:1844–1855 Draganski B, Gaser C, Busch V, Schuierer G, Bogdahn U, May A (2004) Changes in grey matter induced by training: newly honed juggling skills show up as transient feature on a brain- imaging scan. Nature 427:311–312 Dunbar R (1996) Grooming, gossip, and the evolution of language. Harvard University Press, Cambridge, pp 55–79 Dunbar R, Schultz S (2007) Evolution in the social brain. Science 317:1344–1347

References

95

Dunbar RIM, Kaskatis K, MacDonald I, Barra V (2012) Performance of music elevates pain threshold and positive affect: implications for the evolutionary function of music. Evol Psychol 10:688–702 Eccles DW, Tenenbaum G (2004) Why an expert team is more than a team of experts: a social- cognitive conceptualization of team coordination and communication in sport. J Sport Exerc Psychol 26:542–560 Eccles DW, Tran KB (2012) Getting them on the same page: strategies for enhancing coordination and communication in sports teams. J Sport Psychol Act 3:30–40 Ericsson KA, Krampe RT, Tesch-Römer C (1993) The role of deliberate practice in the acquisition of expert performance. Psychol Rev 100:363–406 Fadiga L, Fogassi L, Pavesi G, Rizzolatti G (1995) Motor facilitation during action observation: a magnetic stimulation study. J Neurophysiol 73:2608–2611 Fairhurst MT, Janata P, Krller PE (2014) Leading the follower: an fMRI investigation of dynamic cooperativity and leader-follower strategies in synchronization with an adaptive virtual partner. NeuroImage 84:688–697 Farrow D, Abernethy B (2003) Do expertise and the degree of perception-action coupling affect natural anticipatory performance? Percept 32:1127–1139 Ferrari PF, Rozzi S, Fogassi L (2005) Mirror neurons responding to observation of actions made with tools in monkey ventral premotor cortex. J Cogn Neurosci 17:212–226 Ferrari PF, Visalberghi E, Paukner A, Fogassi L, Ruggiero A, Suomi SJ (2006) Neonatal imitation in rhesus macaques. PLoS One Biol 4(9):e302 Fiore SM, Ross KG, Jentsch F (2012) A team cognitive readiness framework for small-unit training. J Cogn Eng Decis Mak 6:325–349 Flanagan JR (1996) Action-perception coupling in judgments of hand-held loads. In: Wing AM, Haggard P, Flanagan JR (eds) Hand and Brain: the neurophysiology and psychology of hand movements. Academic Press, San Diego, pp 415–430 Flanagan JR, Beltzner MA (2000) Independence of perceptual and sensorimotor predictions in the size-weight illusion. Nat Neurosci 3:737–741 Flanagan JR, Johansson RS (2003) Action plans used in action observation. Nature 424:769–771 Flanagan JR, Vetter P, Johansson RS, Wolpert DM (2003) Prediction precedes control in motor learning. Curr Biol 13:146–150 Fletcher PC, Happe F, Frith U, Baker SC, Dolan RJ, Frackowiak RS, Frith CD (1995) Other minds in the brain: a functional imaging study of ‘theory of mind’ in story comprehension. Cognition 57:109–128 Fogassi L, Gallese V, Fadiga L, Luppino G, Matelli M, Rizzolatti D (1996) Coding of peripersonal space in inferior premotor cortex (area F4). J Neurophysiol 76:141–157 Frischen A, Bayliss AP, Tipper SP (2007) Gaze cueing of attention: visual attention, social cognition, and individual differences. Psychol Bull 133:694–724 Frith CD (2007) The social brain? Philos Trans R Soc B 362:671–678 Frith U, Frith CD (2003) Development and neurophysiology of mentalizing. Philos Trans R Soc B: Biol Sci 358:459–473 Fritsch G, Hitzig E (1870) Ueber die elektrische Erregbarkeit des Grosshirns. Arch Anat Physiol Wiss Med 37:300–332 Funnell E (2001) Evidence for scripts in semantic dementia. Implications for theories of semantic memory. Cogn Neuropsychol 18:323–341 Gallese V, Goldman A (1998) Mirror neurons and the simulation theory of mindreading. Trends Cogn Sci 2:493–501 Gallese V, Fadiga L, Fogassi L, Rizzolatti G (1996) Action recognition in the premotor cortex. Brain 119:593–609 Gallese V, Fogassi L, Fadiga L, Rizzolatti G (2002) Action representation and the inferior parietal lobule. Atten Perform 19:247–266 Gallese V, Gernsbacher MA, Heyes C, Hickock G, Iacoboni M (2011) Mirror neuron forum. Perspect Psychol Sci 6:369–407

96

2 The Background of the Study on Interpersonal Coordination

Gallup AC, Chong A, Couzin ID (2012a) The directional flow of visual information transfer between pedestrians. Biol Lett 8:520–522 Gallup AC, Hale JJ, Sumpter DJ, Garnier S, Kacelnik A, Krebs JR, Couzin ID (2012b) Visual attention and acquisition of information in human crowds. Proc Natl Acad Sci 109:7245–7250 Gardner H (1983) Frames of mind: the theory of multiple intelligences. BasicBooks, New York Garrison KA, Winstein CJ, Aziz-Zadeh L (2010) The mirror neuron system: a neural substrate for methods in stroke rehabilitation. Neurorehabil Neural Repair 24:404–4121 Gaser C, Schlaug G (2003) Brain structures differ between musicians and non-muscians. J Neurosci 23:9240–9245 Gehring WJ, Willoughby AR (2002) The medial frontal cortex and the rapid processing of monetary gains and losses. Science 295:2279–2282 Georgopoulos AP, Schwartz AB, Kettner RE (1986) Neuronal population coding of movement direction. Science 233:1416–1419 Gershgoren L, Fiho E-M, Tenenbaum G, Schinke RJ (2013) Coaching shared mental models in soccer: a longitudinal case study. J Clin Sport Psychol 7:293–312 Ghasemi A, Momeni M, Jarfarzadehpur E, Rezaee M, Taheri H (2011) Visual skills involved in decision making by expert referees. Percept Mot Skills 112:161–171 Gibson JJ (1979) The ecological approach to visual perception. Houghton-Mifflin, Boston Glenberg AM, Gutierrez T, Levin JR, Japuntich S, Kaschak MP (2004) Activity and imagined activity can enhance young children’s reading comprehension. J Educ Psychol 96:424–436 Glenberg AM, Brown M, Levin JR (2007) Enhancing comprehension in small reading groups using a manipulation strategy. Contemp Educ Psychol 32:389–399 Goel V, Grafman J, Sadato N, Hallett M (1995) Modeling other minds. Neuroreport 6:1741–1746 Gordon AM, Forssberg H, Johansson RS, Westling G (1991) Visual size cues in the programing of manipulative forces during precision grip. Exp Brain Res 83:477–482 Grahn JA (2012) See what I hear? Beat perception in auditory and visual rhythms. Exp Brain Res 220:51–61 Grahn JA, Brett N (2007) Rhythm perception in motor areas of the brain. J Cogn Neurosci 19:893–906 Grahn JA, Brett N (2009) Impairment of beat-based rhythm discrimination in Parkinson’s disease. Cortex 45:54–61 Grahn JA, Schuit D (2012) Individual differences in rhythmic ability: behavioral and neuroimaging investigations. Psychomusicol: Music Mind Brain 22:105–121 Graziano MSA (2016) A new view of the motor cortex and its relation to social behaveor. In: Obhi SS, Cross ES (eds) Shared representations: sensorimotor foundations of social life. Cambridge University Press, Cambridge, pp 38–58 Graziano MSA, Cooke DF (2006) Parieto-frontal interactions, personal space, and defensive behavior. Neuropsychologia 44:845–859 Graziano MSA, Yap GS, Gross CG (1994) Coding of visual space by premotor neurons. Science 266:1054–1057 Graziano MSA, Hu XT, Gross CG (1997) Visuo-spatial properties of ventral premotor cortex. J Neurophysiol 77:2268–2292 Graziano MSA, Taylor CSR, Moore T (2002) Complex movements evoked by microstimulation of precentral cortex. Neuron 34:841–851 Graziano MSA, Aflalo TNS, Cooke DF (2005) Arm movements evoked by electrical stimulation in the motor cortex of monkeys. J Neurophysiol 94:4209–4223 Gregory C, Lough S, Stone V, Erzinclioglu S, Martin L, Baron-Cohen S, Hodges JR (2002) Theory of mind in patients with frontal variant frontotemporal dementia and Alzheimer’s disease: theoretical and practical implications. Brain 125:752–764 Grezes J, Costers N, Decetry J (1998) Top down effect of strategy on the perception of human biological motion: a PET investing. Cogn Neuropsychol 15:553–582 Grezes J, Frith C, Passingham RE (2004a) Brain mechanisms for inferring deceit in the actions of others. J Neurosci 24:5500–5505

References

97

Grezes J, Frith CD, Passingham RE (2004b) Inferring false beliefs from the actions of oneself and others: an fMRI study. Neuroimage 21:744–750 Gruetzmacher N, Panzer S, Blandin Y, Shea CH, Charles H (2011) Observation and physical practice: coding of simple motor sequences. Q J Exp Psychol 64:1111–1123 Gueguen N, Martin A (2009) Incidental similarity facilities behavioral mimicry. Soc Psychol 40:88–92 Gueguen N, Martin A, Meineri S, Simon J (2013) Using mimicry to elicit answers to intimate questions in survey research. Field Methods 22:47–57 Haiss F, Schwarz C (2005) Spatial segregation of different modes of movement control in the whisker representation of rat primary motor cortex. J Neurosci 25:1579–1587 Hallett M (2007) Transcranial magnetic stimulation: a primer. Neuron 55:187–1999 Handford C, Davids K, Bennett S, Button C (1997) Skill acquisition in sport: some applications of an evolving practice ecology. J Sports Sci 15:621–640 Hannon EE, Trehub S (2005) Metrical categories in infancy and adulthood. Psychol Sci 16:48–55 Harrison TC, Ayling OG, Murphy TH (2012) Distinct cortical circuit mechanisms for complex forelimb movement and motor map topography. Neuron 74:397–409 Harrmann E, Call J, Hernandez-Lloreda MV, Hare B, Tomasello M (2007) Human have evolved specialized skills of social cognition: the cultural intelligence hypothesis. Science 317:1360–1366 Haslinger B, Erhard P, Altenmuller E, Schroeder U, Boecker H, Ceballos-Baumann AO (2005) Transmodal sensorimotor networks during action observation in professional pianists. J Cogn Neurosci 17:282–293 Hatfield E, Cacioppo JT, Rapson RL (1993) Emotional contagion. Curr Dir Psychol Sci 2:96–99 Haueisen J, Knösche TR (2001) Involuntary motor activation in pianists evoked by music perception. J Cogn Neurosci 13:786–792 Heaton P, Hermelin B, Pring L (1999) Can children with autistic spectrum disorders perceive affect in music? An experimental investigation. Psychol Med 29:1405–1410 Heiser M, Iacoboni M, Maeda F, Marcus J, Mazziotta JC (2003) The essential role of Broca’s area in imitation. Eur J Neurosci 17:1123–1128 Helsen W, Gilis B, Weston M (2006) Errors in judging ‘offside’ in association football: test of the optical error versus the perceptual flash-lag hypothesis. J Sport Sci 24:521–528 Heyes C (2011) Automatic imitation. Psychol Bull 137:463–483 Hogeveen J, Obhi SS (2012) Social interaction enhances motor resonance for observed human actions. J Neurosci 32:5984–5989 Hogeveen J, Inzlicht M, Obhi SS (2014) Power changes how the brain responds to others. J Exp Psychol Gen 143:755–762 Hove MJ (2008) Shared circuits shared time, and interpersonal synchrony. Behav Brain Sci 31:29–30 Hubbard T (2005) Representation al momentum and related displacements in spatial memory: a review of findings. Psychon Bull Rev 12:822–851 Hynes CA, Baird AA, Grafton ST (2006) Differential role of the orbital frontal lobe in emotional versus cognitive perspective-taking. Neuropsychologia 44:374–383 Iacoboni M (2008) Mirroring people. Farrar, Straus & Giroux, New York Iacoboni M, Mazziotta JC (2007) Mirror neuron system: basic findings and clinical applications. Ann Neurol 62:213–218 Iacoboni M, Woods RP, Brass M, Bekkering H, Mazziotta JC, Rizzolatti G (1999) Cortical mechanisms of human imitation. Science 286:2526–2528 Iacoboni M, Molnar-Szakacs I, Gallese V, Buccino G, Mazziotta JC, Rizzolatti G (2005) Grasping the intentions of others with one’s own mirror neuron system. PloS Biology 3(3):e79 Imamura K, Onoe H, Watanabe Y, Andersson J, Hetta J, Schneider H, Reineck R, Lidstrom B, Okura M, Langstrom B (1996) Regional activation of human cerebral cortex upon an adaptation in mirror drawing. Neurosci Lett 209:185–188 Inui N, Yamanishi M, Tada S (1995) Simple reaction times and timing of serial reactions of adolescents with mental retardation, autism, and Down syndrome. Percept Mot Skills 81:739–745

98

2 The Background of the Study on Interpersonal Coordination

Inzlicht M, Gutsell JN, Legault L (2012) Mimicry reduces racial prejudice. J Exp Soc Psychol 48:361–365 Jackson RC, Warren S, Abernethy B (2006) Anticipation skill and susceptibility to deceptive movement. Act Psychol 123:355–371 James W (1890) Principle of psychology. Holt, New York Jerison HJ (1973) Evolution of the brain and intelligence. Academic, New York Johansson G (1973) Visual perception of biological motion and a model for its analysis. Percept Psychophys 14:201–211 Johansson RS (1996) Sensory control of dexterous manipulation in humans. In: Wing AM, Haggard P, Flanagan JR (eds) Hand and Brain: the neurophysiology and psychology of hand movements. Academic Press, San Diego, pp 381–414 Johansson RS, Westling G (1984) Roles of glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objects. Exp Brain Res 56:550–564 Johansson RS, Westling G (1988) Coordinated isometric muscle commands adequately and erroneously programmed for the weight during lifting task with prediction grip. Exp Brain Res 71:59–71 Kahneman D (2011) Thinking, fast and slow. Macmillan, London Kawawaki D, Shibata T, Goda N, Doya K, Kawato M (2006) Anterior and superior lateral occipito- temporal cortex responsible for target motion prediction during overt and covert visual pursuit. Neurosci Res 54:112–123 Keller PE (1999) Attending in complex musical interactions: the adaptive dual role of meter. Aust J Psychol 51:166–175 Keller PE (2014) Ensemble performance: interpersonal alignment of musical expression. In: Fabian D, Timmers R, Schubert E (eds) Expressiveness in music performance: empirical approaches across styles and cultures. Oxford University Press, Oxford, pp 260–282 Keller PE, Burnham DK (2005) Musical meter in attention to multipart rhythm. Music Percept 22:629–661 Keller PE, Knoblich G, Repp BH (2007) Pianists duet better when they play with themselves: on the possible role of action simulation in synchronization. Conscious Cogn 16:102–111 Keller PE, Novembre G, Hove MJ (2014) Rhythm in joint action: psychological and neurophysiological mechanisms for real-time interpersonal coordination. Philos Trans R Soc Lond B: Biol Sci 369:20130394 Keller PE, Novembre G, Loehr J (2016) Musical ensemble performance: representing self, other and joint action outcomes. In: Obhi SS, Cross ES (eds) Shared representations. Cambridge University Press, Cambridge, pp 280–310 Kelso JAS (1995) dynamic patterns: the self-organization of brain and behavior. MIT Press, Cambridge, MA Keysers C, Wicker B, Gazzola V, Anton JL, Fogassi L, Gallese V (2004) A touching sight: SII/PV activation during the observation and experience of touch. Neuron 42:335–346 Kioumourtzoglou E, Michalopoulou M, Tzetzis G, Kourtessis T (2000) Ability profile of the elite volleyball player. Percept Mot Skills 90:757–770 Kirsch LP, Drommelschmidt KA, Cross ES (2013) The impact of sensorimotor experience on affective evaluation of dance. Front Hum Neurosci 7:1–10 Kirsch LP, Dawson K, Cross ES (2015) Dance experience sculpts anesthetic perception and related brain circuits. Ann N Y Acad Sci 1337:130–139 Kirsch LP, Snagg A, Heerey E, Cross ES (2016) The impact experience on affective responses during action observation. PLoS One 11(5):e0154681 Knoblich G, Sebanz N (2006) The social nature of perception and action. Curr Dir Psychol Sci 15:99–104 Knuf L, Aschersleben G, Prinz W (2001) An analysis of ideomotor action. J Exp Psychol 130:779–798 Knutsson E (1972) An analysis of Parkinsonian gait. Brain 95:475–486

References

99

Knutson B, Taylor J, Kaufman M, Peterson R, Glover G (2005) Distributed neural representation of expected value. J. Neurosci 25:4806–4812 Kobayashi H, Kohshima S (1997) Unique morphology of the human eye. Nature 387:767–768 Kohler E, Keysers C, Umilta MA, Fogassi L, Gallese V, Rizzolatti G (2002) Hearing sounds, understanding actions: action representation in mirror neurons. Science 297:846–848 Kokal I, Engel A, Kirschner S, Keysers C (2011) Synchronized drumming enhances activity in the caudate and facilitates prosocial commitment: if the rhythm comes easily. PLoS One 6:e27272 Kontra C, Albert NB, Beilock SL (2012) Embodied cognition: from the playing field to the classroom. In: Hodges NJ, Williams AM (eds) Skilled acquisition in sport, 2nd edn. Routledge, London, pp 323–336 Kouzakova M, van Baaren R, van Knippenberg A (2010) Lack of behavioral imitation in human interactions enhances salivary cortisol levels. Horm Behav 57:421–426 Krakauer JW, Ghez C, Ghilardi MF (2005) Adaptation to visuomotor transformations: consolidation, interference, and forgetting. J Neurosci 25:473–478 Kugler PN, Shaw RE, Vincente KJ, Kinsella-Shaw J (1990) Inquiry into intentional systems I: issues in ecological physics. Psychol Res 52:98–121 Kuhn G, Tatler BW, Cole GG (2009) You look where I look! Effect of gaze cues on overt and covert attention in misdirection. Vis Cogn 17:925–944 Kuhn S, Muller BC, van Baaren RB, Wietzker A, Dijksterhuis A, Brass M (2010) Why do I like you when you behave like me? Neural mechanisms mediating positive consequences of observing someone being imitated. Soc Neurosci 5:384–392 Lakin JL, Chartrand TL (2003) Using nonconscious behavioral mimicry to create affiliation and rapport. Psychol Sci 14:334–339 Lakin JL, Chartrand TL, Arkin RM (2008) I am too just like you: nonconscious mimicry as an automatic behavioral response to social exclusion. Psychol Sci 19:816–822 Lang PJ, Bradley MM, Cuthbert BN (2005) International affective picture system (IAPS): digitized photographs: instruction manual, and affective ratings (Tech Rep. A-6). University of Florida, Center for Research in Psychophysiology, Gainesville Lebed F, Bar-Eli M (2013) Complexity and control in team sports. Routledge, London, pp 47–70 LeDoux JE (2000) Emotion circuits in the brain. Annu Rev Neurosci 23:155–184 Leighton J, Heyes C (2010) Hand to mouth: automatic imitation across effector systems. J Exp Psychol Hum Percept Perform 36:1174–1118 Levi DM, Klein SA, Aitsebaomo AP (1985) Vernier acuity, crowding and cortical magnification. Vis Res 25:963–977 Lim SB, Larssen BC, Hodges NJ (2013) Manipulating visuo-motor experience to probe for observation-induced after-effects in adaptation learning. Exp Brain Res 232:1–14 Lindenberger U, Li S, Gruber W, Muller V (2009) Brain swinging in concert: cortical phase synchronization while playing guitar. BMC Neurosci 10:22 Loehr JD, Palmer C (2011) Temporal coordination between performing musicians. Q J Exp Psychol 64:2153–2167 Loehr J, Kourtis D, Vesper C, Sebanz N, Knoblich G (2013) Monitoring individual and joint action outcomes in duet music performance. J Cogn Neurosci 25:1049–1061 Loula F, Prasad S, Harber K, Shiffrar M (2005) Recognizing people from their movement. J Exp Psychol Hum Percept Perform 31:210–220 MacLean PD (1973) A triune concept of the brain and behavior. University of Toronto Press, Toronto, pp 6–22 MacMahon C, Mildenhall W (2012) A practical perspective on decision making influences in sports officiating. Int J Sports Sci Coach 7:153–165 Maguire EA, Gadian DG, Johnsrude IS, Good CD, Ashburner J, Frackowiak RSJ, Frith CD (2000) Navigation-related structural change in the hippocampi of taxi drivers. Proc Natl Acad Sci 97:4398–4403 Makris S, Urgesi C (2015) Neural underpinnings of superior action prediction abilities in soccer players. Soc Cogn Affect Neurosci 10:342–351

100

2 The Background of the Study on Interpersonal Coordination

Malfait N, Valyear KF, Culham JC, Anton J-L, Brown LE, Gribble PL (2010) fMRI activation during observation of others’ reach errors. J Cogn Neurosci 22:1493–1503 Martin TA, Keating JG, Goodkin HP, Bastian AJ, Thach WT (1996) Throwing while looking through prisms I. Focal olivocerebellar lesions impair adaptation. Brain 199:1183–1198 Mascarenhas DRD, Collins D, Mortimer P, Morris B (2005a) Training accurate and coherent decision making in rugby. Sport Psychol 19:131–147 Mascarenhas DRD, Collins D, Mortimer P (2005b) Elite refereeing performance: developing a model for sport science support. Sport Psychol 19:364–379 Mascarenhas DRD, Collins D, Mortimer P (2005c) Assessing the accuracy and coherence of decision making in rugby-union referees. J Sport Behav 28:253–271 Mason MF, Tatkow EP, Macrae CN (2005) The look of love: gaze shifts and person perception. Psychol Sci 16:236–239 Mattar AAG, Gribble PL (2005) Motor learning by observing. Neuron 46:153–160 McGarry T, Anderson DI, Wallace SA, Hughes M, Franks IM (2002) Sport competition as a dynamical self-organizing system. J Sports Sci 20:771–781 McGregor H, Gribble PL (2015) Changes in visual and sensory-motor resting-state functional connectivity support motor learning by observing. J Neurophysiol 114:677–688 McGregor H, Gribble PL (2016) Observational motor learning. In: Obhi SS, Cross ES (eds) Shared representations: sensorimotor foundations of social life. Cambridge University Press, Cambridge, pp 525–540 McIntosh DN (2006) Spontaneous facial mimicry, liking and emotional contagion. Pol Psychol Bull 37:31–42 McIntosh AR, Cabeza RE, Lobaugh NJ (1998) Analysis of neural interactions explains the activation of occipital cortex by an auditory stimulus. J Neurophysiol 80:2790–2796 Mellers R, Schwartz A, Cooke A (1998) Judgement and decision making. Annu Rev Psychol 49:447–477 Miles JR, Kivlighan DM Jr (2008) Team cognition in group interventions: the relation between coleaders’ shared mental models and group climate. Group Dyn-Theor Res 12:191–209 Milgram S, Bickman L, Berkowitz L (1969) Note on the drawing power of crowds of different size. J Pers Soc Psychol 13:79–82 Mitchell JP, Heatherton TF, Macrae CN (2002) Distinct neural systems subserve person and object knowledge. Proc Natl Acad Sci USA 99:15238–15243 Mitchell JP, Banaji MR, Macrae CN (2005) General and specific contributions of the medial prefrontal cortex to knowledge about mental states. Neuroimage 28:757–762 Molnar-Szakacs I, Overy K (2006) Music and mirror neurons: from motion to ‘e’ motion. SCAN 1:235–241 Morasso P (1981) Spatial control of arm movements. Exp Brain Res 42:223–227 Morris JS, Friston KJ, Buchel C, Frith CD, Young AW, Calder AJ, Dolan RJ (1998) A neuromodulatory role for the human amygdala in processing emotional facial expressions. Brain 121:47–57 Morrison I, Lloyd D, di Pellegrino G, Roberts N (2004) Vicarious responses to pain in anterior cingulate cortex: is empathy a multisensory issue? Cogn Affect Behav Neurosci 4:270–278 Muellbacher W, Ziemann U, Wissel J, Dang N, Kofler M, Facchini S, Boroojerdi B, Poewe W, Hallett M (2002) Early consolidation in human primary motor cortex. Nature 415:640–644 Myowa-Yamakoshi M, Matsuzawa T (1999) Factors influencing imitation of manipulatory actions in chimpanzees (Pan troglodytes). J Comp Psychol 2:128–136 Naish KR, Houston-Price C, Bremner AJ, Holmes NP (2014) Effects of action observation on corticospinal excitability: muscle specificity, direction, and timing of the mirror response. Neuropsychologia 64C:331–348 Newell KM, Liu Y-T, Mayer-Kress G (2001) Time scales in motor learning and development. Psychol Rev 108:57–82 Nieuwenhuis S, Aston-Jones G, Cohen JD (2005) Decision-making, the P3, and the locus coeruleus-norepinephrine system. Psychol Bull 131:510–532

References

101

Nishitani N, Hari R (2000) Temporal dynamics of cortical representation for action. Proc Natl Acad Sci USA 97:913–918 Nishitani N, Hari R (2002) Viewing lip forms: cortical dynamics. Neuron 36:1211–1220 Novembre G, Ticini LCF, Schutz-Bosbach S, Keller PE (2012) Distinguishing self and other in joint action: evidence from a musical paradigm. Cereb Cortex 22:2894–2903 Novembre G, Ticini LCF, Schutz-Bosbach S, Keller PE (2014) Motor simulation and the coordination of self and other in real-time joint action. Soc Cogn Affect Neurosci 9:1062–1068 O’Shea J, Sebastian C, Boorman ED, Johansen-Berg H, Rushworth MFS (2007) Functional specificity of premotor-motor cortical interaction during action selection. Eur J Neurosci 26:2085–2095 Obhi SS (2016) Neurocognitive explorations of social mimicry. In: Obhi SS, Cross ES (eds) Shared representations: sensorimotor foundations of social life. Cambridge University Press, Cambridge, pp 171–192 Obhi SS, Haggard P, Taylor J, Pascual-Leone A (2002) rTMS to the supplementary motor area disrupts bimanual coordination. Mot Control 6:319–332 Obhi SS, Hogeveen J, Pascual-Leone A (2011) Resonating with others: the effects of self-construal type on motor cortical output. J Neurosci 31:14531–14535 Oliveira FTP, MacDonald JJ, Goodman D (2007) Performance monitoring in the anterior cingulate cortex is not all error related: expectancy deviation and the representation of action-outcome associations. J Cogn Neurosci 19:1994–2004 Ong NT, Hodges NJ (2010) Absence of after-effects for observers after watching a visuomotor adaptation. Exp Brain Res 205:325–334 Ong NT, Larssen BC, Hodges NJ (2012) In the absence of physical practice, observation and imagery do not result in updating of internal models for aiming. Exp Brain Res 218:9–19 Orgs G, Dombrowski J-H, Heil M, Jansen-Osmann P (2008) Expertise in dance modulates alpha/ beta event-related desynchronization during action observation. Eur J Neurosci 27:3380–3384 Ostry DJ, Darainy M, Matter AA, Wong J, Gribble PL (2010) Somatosensory plasticity and motor learning. J Neurosci 30:5384–5393 Overduin SA, d’Avella A, Carmena JM, Bizzi E (2012) Microstimulation activates a handful of muscle synergies. Neuron 76:1017–1077 Padoa-Schioppa C, Assad JA (2006) Neurons in the orbitofrontal cortex encode economic value. Nature 441:223–226 Palmer C, Loehr JD (2013) Meeting of two minds in duet piano performance. In: Bernstein LF, Rozin A (eds) Musical implications: essays in honor of Eugene Narmour. Pendragon Press, Hillsdale, pp 323–338 Pantev C, Oostenveld R, Engelien A, Ross B, Roberts LE, Hoke M (1998) Increased auditory cortical representation in musicians. Nature 392(6678):811 Pelphrey KA, Morris JP, McCarthy G (2004a) Grasping the intentions of others: the perceived intentionality of an action influences activity in the superior temporal sulcus during social perception. J Cogn Neurosci 16:1706–1716 Pelphrey KA, Viola RJ, McCarthy G (2004b) When strangers pass: processing of mutual and averted social gaze in the superior temporal sulcus. Psychol Sci 15:598–603 Pelphrey KA, Morris JP, Michelich CR, Allison T, McCarthy G (2005) Functional anatomy of biological motion perception in posterior temporal cortex: an FMRI study of eye, mouth and hand movements. Cereb Cortex 15:1866–1876 Penfield W, Boldrey E (1937) Somatic motor and sensory representation in the cerebral cortex of man as studied by electrical stimulation. Brain 60:389–443 Phelps EA, O’Connor KJ, Cunningham WA, Funayama ES, Gatenby JC, Gore JC, Banaji MR (2000) Performance on indirect measures of race evaluation predicts amygdala activation. J Cogn Neurosci 12:729–738 Phelps EA, Cannistraci CJ, Cunningham WA (2003) Intact performance on an indirect measure of race bias following amygdala damage. Neuropsychologia 41:203–208

102

2 The Background of the Study on Interpersonal Coordination

Phillips ML, Young AW, Senior C, Brammer M, Andrew C, Calder AJ, Bullmore ET, Perrett DI, Rowland D, Williams SC, Gray JA, David AS (1997) A specific neural substrate for perceiving facial expressions of disgust. Nature 389:495–498 Pineda JA (2005) The functional significance of mu rhythms: translating ‘seeing’ and ‘hearing’ into ‘doing’. Brain Res Rev 50:57–68 Premack D, Woodruff G (1978) Does the chimpanzee have a theory of mind? Behav Brain Sci 1:515–526 Press C, Gillmeister H, Heyes C (2007) Sensorimotor experience enhances automatic imitation of robotic action. Proc R Soc B: Biol Sci 274:2509–2514 Prinz W (1990) A common coding approach to perception and action. In: Neumann O, Prinz W (eds) Relationships between perception and action: current approaches. Springer, Berlin, pp 167–201 Prinz W (1997) Perception and action planning. Eur J Cogn Psychol 9:129–154 Prinz W (2005) An ideomotor approach to imitation. In: Hurley S, Chater N (eds) Perspectives on imitation: from neuroscience to social science, vol 1. MIT Press, Cambridge, MA, pp 141–156 Put K, Baldo MV, Cravo AM, Wagemans J, Helsen WF (2013) Experts in offside decision making learn to compensate for their illusory perceptions. J Sport Exerc Psychol 35:576–584 Qin P, Northoff G (2011) How is our self related to midline regions and the default-mode network? NeuroImage 57:1221–1233 Ragert M, Schroeder T, Keller PE (2013) Knowing too little or two much: the effects of familiarity with a co-performer’s part on interpersonal coordination in musical ensembles. Front Psychol 4:368 Ramanathan D, Conner JM, Tuszynski MH (2006) A form of motor cortical plasticity that correlates with recovery of function after brain injury. Proc Natl Acad Sci USA 103:11370–11375 Reichelt AF, Ash AM, Baugh LA, Johansson RS, Flanagan JR (2013) Adaptation of lift forces in object manipulation through action observation. Exp Brain Res 228:221–234 Repp BH (1995) Expressive timing in Schumann’s ‘Traumerei:’ an analysis of performances by graduate student pianists. J Acoust Soc Am 98:2313–2427 Repp BH (1997) The aesthetic quality of a quantitatively average music performance: two preliminary experiments. Music Percept 14:419–444 Repp BH, Penel A (2004) Rhythmic movement is attracted more strongly to auditory than to visual rhythms. Psychol Res 68:252–270 Restle F, Burnside BL (1972) Tracking of serial patterns. J Exp Psychol 95:299–307 Richards P, Mascarenhas DRD, Collins D (2009) Implementing reflective practice approaches with elite team athletes: parameters of success. Reflect Pract 10:353–363 Richards P, Collins D, Mascarenhas DRD (2012) Developing rapid high pressure team decision making skills. The integration of slow deliberate reflective learning within the competitive performance environment: a case study of elite netball. J Reflect Pract 13:455–469 Richardson MJ, Dale R (2005) Looking to understand: the coupling between speakers’ and listeners’ eye movements and its relationship to discourse comprehension. Cogn Sci 29:1045–1060 Richardson MJ, Gobel MS (2015) Social attention. In: Fawcett JM, Risko EF, Kingstone (eds) The handbook of attention. MIT Press, Cambridge, MA, pp 349–367 Richardson MJ, Dale R, Kirkham NZ (2007) The art of conversation is coordination, and beliefs about visual context. Cogn Sci 33:1468–1482 Richardson MJ, Street CN, Tan JYM, Kirkham NZ, Hoover MA, Cavanaugh AG (2012) Joint perception: gaze and social context. Front Hum Neurosci 6:1–8 Ripoll H, Kerlirzin Y, Stein JF, Reine B (1995) Analysis of information processing decision making, and visual strategies in complex problem solving sport situations. Hum Mov Sci 14:325–349 Rizzolatti G, Craighero L (2004) The mirror-neuron system. Annu Rev Neurosci 27:169–192 Rizzolatti G, Scandolara C, Matelli M, Gentilucci M (1981) Afferent properties of periarcuate neurons in macaque monkeys. II. Visual response. Behav Brain Res 2:147–163 Rizzolatti G, Fadiga L, Gallese V, Fogassi L (1996) Premotor cortex and the recognition of motor actions. Cogn Brain Res 3:131–141

References

103

Rizzolatti G, Fogassi L, Gallese V (2001) Neurophysiological mechanisms underlying the understanding and imitation of action. Nat Rev Neurosci 2:661–670 Rizzolatti G, Fogassi L, Gallese V (2002) Motor cognitive functions of the ventral premotor cortex. Curr Opin Neurobiol 12:149–154 Rowe AD, Bullock PR, Polkey CE, Morris RG (2001) ‘Theory of mind’ impairments and their relationship to executive functioning following frontal lobe excisions. Brain 124:600–616 Rozin P, Royzman EB (2001) Negativity bias, negativity dominance, and contagion. Pers Soc Psychol Rev 5:296–320 Sartre JP (1958) Being and nothingess: an essay on phenomenological ontology. Routledge, London Savelsbergh GJP, Williams AM, van der Kamp J, Ward P (2002) Visual search, anticipation and expertise in soccer goalkeepers. J Sports Sci 20:279–287 Saxe R, Kanwisher N (2003) People thinking about thinking people. The role of the temporo- parietal junction in ‘theory of mind. Neuroimage 19:1835–1842 Saxe R, Xiao DK, Kovacs G, Perrett DI, Kanwisher N (2004) A region of right posterior superior temporal sulcus responds to observed intentional actions. Neuropsychologia 42:1435–1446 Schmorrow DD, Bolstad CA, May KA, Cuevas HM (2012) Editor’s introduction to the special issue on exploring cognitive readiness in complex operational environments: advances in theory and practice, part II. J Cogn Eng Decis Mak 6:355–357 Schnall S, Harber KD, Stefanucci JK, Proffitt DR (2008) Social support and the perception of geographical slant. J Exp Soc Psychol 44:1246–1255 Schultz J, Friston KJ, O’Doherty J, Wolpert DM, Frith CD (2005) Activation in posterior superior temporal sulcus parallels parameter inducing the percept of animacy. Neuron 45:625–635 Senju A, Johnson MH (2009) Atypical eye contact in autism: models, mechanisms and development. Neurosci Biobehav Rev 33:1204–1214 Shadmehr R, Mussa-Ivaldi FA (1994) Adaptive representation of dynamics during learning of a motor task. J Neurosci 14:3208–3224 Shadmehr R, Smith MA, Krakauer JW (2010) Error correction, sensory prediction, and adaptation in motor control. Annu Rev Neurosci 33:89–108 Shea CH, Wright DL, Wulf G, Whitacre C (2000) Physical and observational practice afford unique learning opportunities. J Mot Behav 32:27–36 Sherif M (1937) An experimental approach to the study of attitudes. Sociometry 1:90–98 Shteynberg G (2010) A silent emergence of culture: the social tuning effect. J Pers Soc Psychol 99:683–689 Silva P, Garganta J, Araujo D, Davids K, Aguiar P (2013) Shared knowledge of shared or shared affordances? Insights from an ecological dynamics approach to team coordination in sports. Sport Med 43:765–772 Singer T, Seymour B, O’Doherty J, Kaube H, Dolan RJ, Frith CD (2004) Empathy for pain involves the affective but not sensory components of pain. Science 303:1157–1162 Sparenberg P, Topolinski S, Springer A, Prinz W (2012) Minimal mimicry: mere effector matching induces preference. Brain Cogn 80:291–300 Steele JD, Lawrie SM (2004) Segregation of cognitive and emotional function in the prefrontal cortex: a stereotactic meta-analysis. Neuroimage 21:868–875 Steibeis N, Koelsch S (2009) Understanding the intentions behind man-made products elicits neural activity in area dedicated to mental state attribution. Cereb Cortex 19:619–623 Stefan K, Cohen LG, Duque J, Mazzocchio R, Celink P, Sawaki L, Ungerleider L, Classen J (2005) Formation of a motor memory by action observation. J Neurosci 25:9339–9346 Stepniewska I, Fang PC, Kaas JH (2005) Microstimulation reveals specialized subregions for different complex movements in posterior parietal cortex of prosimian galagos. Proc Natl Acad Sci USA 102:4878–4883 Strafella AP, Paus T (2000) Modulation of cortical excitability during action observation: a transcranial magnetic stimulation study. NeuroReport 11:2289–2292 Striano T, Reid VM (2006) Social cognition in the first year. Trends Cogn Sci 10:471–476

104

2 The Background of the Study on Interpersonal Coordination

Subiaul F, Cantlon JF, Holloway RL, Terrace HS (2004) Cognitive imitation in rhesus macaques. Science 305:407–410 Subiaul F, Renner E, Krajkowski E (2016) The comparative study of imitation mechanisms in non- human primates. In: Obhi SS, Cross ES (eds) Shared representations: sensorimotor foundations of social life. Cambridge University Press, Cambridge, pp 109–135 Summers JJ (1975) The role of timing in motor program representation. J Mot Behav 7:229–241 Teneggi C, Canzoneri E, di Pellegrino G, Serino A (2013) Social modulation of peripersonal space boundaries. Curr Biol 23:406–411 Thompson L, Fine GA (1999) Socially shared cognition, affect, and behavior: a review and integration. Pers Soc Psychol Rev 3:278–302 Thorpe WH (1956) Learning and instinct in animals. Methuen, London Tognoli E, Lagarde J, DeBuzman GC, Kelso JAS (2007) The phi complex as a neuromarker of human social coordination. Proc Natl Acad Sci 104:8190–8195 Tomasello M, Carpenter M, Call J, Behne T, Moll H (2005) Understanding and sharing intentions: the origin of cultural cognition. Behav Brain Sci 28:675–735 Tomeo E, Cesari P, Aglioti SM, Urgesi C (2013) Fooling the kickers but not the goalkeepers: behavioral and neurophysiological correlates of fake action detection in soccer. Cereb Cortex 23:2765–2778 Uhlig M, Fairhurst MT, Keller PE (2013) The importance of integration and top-down salience when listening to complex multi-part musical stimuli. NeuroImage 77:52–61 Ulzen NR, van Fiorio M, Cesari P (2013) Motor resonance evoked by observation of subtle nonverbal behavior. Soc Neurosci 8:347–355 Urgesi C, Makris S (2016) Sport performance: motor expertise and observational learning in sport. In: Obhi SS, Cross ES (eds) Shared representations: sensorimotor foundations of social life. Cambridge University Press, Cambridge, pp 565–587 Urgesi C, Savonitto M, Fabbro F, Aglioti SM (2012) Long- and short-term plastic modeling of action prediction abilities in volleyball. Psychol Res 76:540–562 Van Acker GM III, Amundsen SM, Messamore WG, Zhang HY, Luchies CW, Kovac A, Cheney PD (2013) Effective intracortical microstimulation parameters applied to primary motorcortex for evoking forelimb movements to stable spatial end points. J Neurophysiol 110:1180–1189 van Baaren RB, Maddux WW Chartrand TL, de Bouter C, van Knippenberg A (2003) It takes two to mimic: behavioral consequences of self-construals. J Pers Soc Psychol 84:1093–1102 van de Waal E, Whiten A (2012) Spontaneous emergence, imitation and spread of alternative foraging techniques among groups of vervet monkeys. PloS One 7(10):e47008 van Honk J, Schutter DJ, D’alfonso AA, Kessels RP, de Haan EH (2002) 1hz rTMS over the right prefrontal cortex reduces vigilant attention to unmasked but not to masked fearful faces. Biol Psychiatry 52:312–317 van Schie HT, Mars RB, Coles MG, Bekkering H (2004) Modulation of activity in medial frontal and motor cortices during error observation. Nat Neurosci 7:549–554 Voelkl B, Huber L (2007) Imitation as faithful copying of a novel technique in marmoset monkeys. PloS One 2(7):e611 Vogeley K, May M, Ritzl A, Falkai P, Zilles K, Fink GR (2004) Neural correlates of first-person perspective as one constituent of human self-consciousness. J Cogn Neurosci 16:817–827 von Zimmermann J, Richardson DC (2016) Joint perception. In: Obhi SS, Cross ES (eds) Shared representations: sensorimotor foundations of social life. Cambridge University Press, Cambridge, pp 236–253 Waclawik K, Watson S, Grahn JA (2016) Musical synchronization, social interaction and the brain. In: Obhi SS, Cross ES (eds) Shared representations: sensorimotor foundations of social life. Cambridge University Press, Cambridge, pp 603–626 Walter H, Adenzato M, Ciaramidaro A, Enrici I, Pia L, Bara BG (2004) Understanding intentions in social interaction: the role of the anterior paracingulate cortex. J Cogn Neurosci 16:1854–1863 Walton ME, Devlin JT, Rushworth MF (2004) Interactions between decision making and performance monitoring within prefrontal cortex. Nat Neurosci 7:1259–1265

References

105

Whiten A (1998) Imitation of the seuencial structure of actions by chimpanzees (Pan troglodytes). J Comp Psychol 112:270–281 Wicker B, Keysers C, Plailly J, Royet JP, Gallese V, Rizzolatti G (2003) Both of us disgusted in My insula: the common neural basis of seeing and feeling disgust. Neuron 40:655–664 Williams AM (2000) Perceptual skill in soccer: implications for talent identification and development. J Sports Sci 18:737–750 Williams AM, Davids K (1998) Visual search strategy, elective attention, and expertise in soccer. Res Q Exerc Sport 69:111–128 Williams AM, Davids K, Williams JG (1999) Visual perception and action in sport. E & FN SPON, London Williams AM, Hodges NJ (eds) (2004) Skill acquisition in sport: research, theory and practice. Routledge, London, pp 409–433 Winston JS, Strange BA, O’Doherty J, Dolan RJ (2002) Automatic and intentional brain responses during evaluation of trustworthiness of faces. Nat Neurosci 5:277–283 Wolpert DM, Flanagan JR (2001) Motor prediction. Curr Biol 11:R726–R732 Yabar Y, Johnston L, Miles L, Peace V (2006) Implicit behavioral mimicry: investigating the impact of group membership. J Nonverbal Behav 30:97–113 Zacks JM, Vettel JM, Michelon P (2003) Imagined viewer and object rotations dissociated with event-related FMRI. J Cogn Neurosci 15:1002–1018 Zeki S, Watson JD, Lueck CJ, Friston KJ, Kennard C, Frackowiak RS (1991) A direct demonstration of functional specialization in human visual cortex. J Neurosci 11:641–649

Chapter 3

An Overview of the Study on Interpersonal Coordination

Abstract This chapter gives an overview of previous studies on interpersonal coordination. The first section reviews studies on unintentional interpersonal coordination in cases of rhythmic behaviors, such as synchronized movements between people. For example, when two people perform a rhythmic behavior such as swinging a pendulum, rocking in a rocking chair, or walking side-by-side, their behavior tend to synchronize automatically. Such automatic synchrony is known as entrainment. The second section reviews studies on intentional interpersonal coordination in cases of non-rhythmic behavior, often discussed in context of imitative versus complementary movement. In coordination that relies on synchronization, people perform rhythmic behaviors, which combine to achieve the shared action goal. These motor behaviors are often identical, and hence symmetrical. By contrast, in complementary coordination, while two people perform different behaviors, their behaviors link together so that the one compensates for the other to achieve the shared goal. From a developmental viewpoint, the third section reviews studies on the growth of joint action over the first 2 years of life. As for adult-infant interaction, it takes approximately 2 years for infants to become autonomous contributions to sustained, goal-directed joint activity active, collaborative partners. As for infant- infant interaction, 2-year olds were thus able to act jointly with each action in a novel situation, without the support of familiar partners, goals, or routines. Keywords Entrainment · Complementary movement · Shared goal · Anticipation Daily life, as well as sports and art, often requires coordination of movements performed by more than two people. For example, two or four people can lift and carry a table in real life, while two pianists can synchronize their performance to each other’s timing with asynchonies of 30 ms (Keller et al. 2007). Such group action coordination has recently been studied using the term ‘joint action’, which is defined as a social interaction whereby two or more individuals coordinate their actions in space and time to bring about a change in the environment (Sebanz et al. 2006). Whereas initial interest in joint action centered to philosophical questions about the nature of shared intentions or intersubjectivity (Clark 1996), over the last decade psychologists and neuroscientists, who is interested in human motor control and © Springer Nature Singapore Pte Ltd. 2018 N. Inui, Interpersonal Coordination, https://doi.org/10.1007/978-981-13-1765-1_3

107

108

3 An Overview of the Study on Interpersonal Coordination

learning, have conducted many experimental studies to understand the mechanisms underlying interpersonal coordination. In this chapter, I provide an overview of some of the basic mechanisms that make joint action possible. One type of the studies on joint action has mainly examined the mechanism of unintentional interpersonal coordination in cases of rhythmic behaviors, such as synchronized movements between people. Another type of the studies has investigated the mechanism of intentional interpersonal coordination in cases of non-rhythmic behavior, often discussed in context of imitative versus complementary movement. Here, I address the former in the Sect. 3.1 while the latter in the Sect. 3.2. In addition, the Sect. 3.3 gives an overview of studies on the growth of joint action over the first 2 years of life.

3.1 Unintentional Interpersonal Entrainment 3.1.1 Applying HKB Model to Human Coordination In the 1980s, Kelso, Kugler, and Turvey were proposing a new theory of the coordination and control of human movement from a dynamic systems approach (For a review, Kugler et al. 1982). The theory is thought to examine how the many degree of freedom of the perceptual-motor system are control. The traditional machine theory was replaced with a dynamical conception of order based on theories of physical biology. Neurophysiological conceptions, such as a distinction between afference and efference, were replaced by synergic linkages between muscles that were dynamically constrained. Cognitive programs were replaced by equations of constraint that channel and guide a dynamic unfolding of behavior in a non-prescriptive manner. Kelso et al. (1981) examined how the behavioral patterns associated with coordinated rhythmic limb movements can be modeled as representing the dynamics of periodic attractors or limit cycles. They further examined how the behavioral transitions between oscillatory phase modes can be examined in bimanual tasks and understood as phase transitions or bifurcations. In particular, the bimanual phenomenon was that the anti-phase coordination of wrist or index fingers becomes increasingly less stable as the frequency of oscillation is increased, eventually breaking down and leading to a transition to in-phase coordination. Such behavioral switching was captured by a mathematical formalism that modeled both the steady state and phase transition behavior of coordinated rhythmic limb movements by capturing the dynamics of the relative phase angle (φ) between limbs.

j = a sin j　 2b sin 2j +

Qz

(3.1)

In this equation (HKB model), φ is the rate of change of the relative phase angle formed between the two oscillations, a and b are coefficients whose magnitudes govern the strength of the between-oscillator coupling, and ζ is a Gaussian

3.1 Unintentional Interpersonal Entrainment

109 3.5

50 Inphase Antiphase

3 Mean Cycles per Trail

SD of Relative Phase [˚]

45 40 35 30 25 20

2.5 2 1.5 1 0.5

15

0

10 0

0.5

1 1.5 2 Frequency (Hz)

2.5

0

0.5

2 1 1.5 Frequency (Hz)

2.5

Fig. 3.1 The destabilization of relative phase with frequency increase in the interpersonal coordination of lower legs. Both relative phase variability (left panel) and occurrence of the other modes (right panel) are much greater for antiphase at higher frequencies. (Redraw from Schmidt and Richardson 2008)

white noise process dictating a stochastic force of strength Q (Schöner et al. 1986). Using the terminology of synergetics, φ is an order parameter and the variables that influence its stability are control parameters. For example, while an order parameter summarizes the spatial temporal order of the rhythmic unit, a control parameter is frequency of oscillation. This model of behavioral dynamics is extended from intrapersonal to interpersonal coordination, and it has subsequently generated a mount of coordination research including interpersonal coordination. As a seminal research, Schmidt et al. (1990) asked two seated participants to visually coordinate their lower legs in either in-phase or anti-phase while oscillating them at a tempo of an auditory metronome pulse that increased in frequency from 0.6 to 2.0 Hz by 0.2 Hz increments every 5 s. In the first experiment, six pairs of participants were instructed to keep and return to their original coordination phase mode if they fell out of it. The trials were videotaped and the relative phase angle between the legs of the participants was measured frame by frame. Schmidt and colleagues examined the relative stability of the two modes of phasing and whether the likelihood of a breakdown in phase locking would grow as the frequency of oscillation was increased. Participants had a hard time maintaining the anti-phase coordination of their lower legs at the higher frequencies. The left panel of Fig. 3.1 showed that the variability at the higher frequencies for the anti-phase mode is indicative of a breakdown in coordination, whereas this variability for in-phase is larger but still indicative of a stable state. In addition, The right panel of Fig. 3.1 showed that the mean number of cycles that were in the other phase mode was 1.5 times great for anti-phase than in-phase for lower frequencies but nearly three times as great for higher frequencies of oscillation. In the second experiment, to analyze the characteristics of the anti-phase breakdown and

110

3 An Overview of the Study on Interpersonal Coordination

transition to in-phase, participants were instructed to allow the transition to naturally emerge. The transitions had the formal properties of a dynamical reorganization, namely divergence, critical fluctuations, and hysteresis. These visually coordinated interpersonal movements expressed the same differential dynamic stability of the intrapersonal phase modes modeled by Haken et al. (1985). Namely, the in-phase mode of interpersonal leg swinging is the globally stable behavioral attractor while the anti-phase mode is a dynamically unstable local attractor. Scaling the oscillation frequency results in a gradual wakening of the local attractor, followed by a state of criticality (increase in relaxation time and amplification of fluctuations), and then leads to a sudden annihilation of a local attractor and a switch to the globally attractive in-phase mode (divergent response). The third experiment showed that the critical nature of this phase transition is conditioned by the rate that the control parameter is increased (Kelso et al. 1986). The length of time the relative phase was observed from 5 to 10 s resulted in an elimination of the state of criticality immediately before the transition because the observation time scale (plateau time = 10 s) was now equivalent to the random walk or first passage time required for such a transition to occur probabilistically due to chance. Taken together, the three experiments of Schmidt et al. (1990) found that the self- organizing coordination dynamics that Kelso et al. (1981) observed in bimanual coordination of an individual also occur in interpersonal coordination of movements. The interpersonal nature of these behavioral attractors demonstrated that the dynamical organizing principles of the HKB equation can operate in neurally based behavioral oscillatory systems that are coupled by perceptual information.

3.1.2 Frequency Detuning The mass coordination of firefly flashing in New Guinea and Malaysia is studied as an example of inter-organism dynamical synchronization. At dusk, male fireflies congregate in trees. As the night progresses, a synchronization of their flashing emerges a large light-pulsing mass of fireflies. Hanson (1978) experimentally examined and dynamically modeled the firefly synchronization process. He manipulated the rate at which he periodically flashed a light at a firefly. The dynamic process in Hanson’s model of firefly entrainment is captured by an extension of the HKB equation that contains a Δω term which specifies the eigenfrequency difference or frequency detuning between the two rhythmic units:

j = Dw　 a sin j　 2b sin 2j +

Q z.

(3.2)

Entrainment of the fireflies in this model will occur when the eigenfrequency difference between the firefly flashing and the experimental flashing of the light (Δω) is small enough to be balanced by the coordination forces of the coupling function (−a sinφ −2b sin 2φ). If the eigenfrequency difference is too large, φ will

3.1 Unintentional Interpersonal Entrainment

111 40

200 Intrinsic frequency

150

Phase Lag (˚)

Lag (ms)

100 50 0 -50

20

0

-20

-100 -150 -200 .85

.9

.95

1

1.05

1.1

Flashing Frequency (Hz)

1.15

1.2

-40 -.6

-.4

-.2

0

.2

.4

.6

∆ω (Hz)

Fig. 3.2 The effect of frequency detuning (Δω) on the relative phase lag in fireflies (left panel, Hanson 1978, redraw from Schmidt and Richardson 2008) and interpersonal coordination of wrist pendulums (right panel, Schmidt and Turvey 1994, redraw from Schmidt and Richardson 2008). Phase lag increases as the detuning greater as expected by the extended HKB model

be non-zero and absolute coordination or phase locking will not occur and the firefly will not fully entrain to the light. The dynamical model also predicts that when the balancing of Δω and the coupling forces does occur and phase locking ensues, a phase lag between the two light flashes will emerge that is proportional in size to the Δω. The model further predicts that when the flashing light is inherently faster, it should lead in the cycle, and when it is inherently slower, it should lag in the cycle. Because the phase lag demonstrates that the oscillating rhythmic unit (in this case, the firefly) is retaining some vestige of its original dynamic when in the coordinative state, von Holst (1939) referred to it as evidence of a ‘maintenance tendency’ of the oscillator. The left panel of Fig. 3.2 shows the increase in phase lag with an increase in frequency detuning. Such phase lag patterning of oscillators with different eigenfrequencies has been observed in a number of other examples of neurally based biological coordination such as cockroach locomotion (Foth and Graham 1983) and the coordination of breathing and sucking in infants (Goldfield et al. 1999). In addition, using an experimental paradigm of swing wrist-pendulums, Schmidt et al. (1993), Sternad et al. (1992), and Turvey et al. (1986) have demonstrated that similar detuning patterning is seen in human bimanual coordination. This paradigm allows the manipulation of the detuning by having individuals bimanually swing handheld pendulums whose lengths (and hence, whose frequencies) can be manipulated. Two pendulums of identical lengths will have a Δω of 0, whereas two pendulums of different lengths will have a non-zero Δω whose magnitude depends on the length difference. Using this wrist-pendulum paradigm, Schmidt and Turvey (1995) showed that the intrapersonal and bimanual relative phase patterning support the predictions of the extended HBK model. Moreover, Schmidt and Turvey (1994) asked two participants sitting side by side to visually coordinate wrist-pendulum pairs that differed in their Δω’s. Participants could easily coordinate the pendulums in synchrony. However, the right panel of

112

3 An Overview of the Study on Interpersonal Coordination

Fig. 3.2 showed that, as the length difference between the wrist-pendulums becomes greater, the phase lag between the pendulums increases such that the person with the shorter pendulum leads in the cycle. Using wrist-pendulum, subsequent interpersonal studies examined effects of phase mode and frequency of oscillation predicted by Amazeen et al. (1995). Visual interpersonal coordination of wrist-pendulum was found to be weaker as found by exaggerated relative phase lags and fluctuations for anti-phase compared to in-phase and for higher frequencies of oscillation compared to lower frequencies (Amazeen et al. 1995; Schmidt et al. 1998). These results indicate that interpersonal rhythmic coordination is subject to the same dynamical power of the explicit modeling of the dynamical self-organizing processes captured by HKB equation.

3.1.3 Unintentional Interpersonal Entrainment Social psychologists have been examining entrainment in social interactions since 1960s, and their study has been directed at understanding what role this coordination of social behavior plays in human communication. For example, Condon and Ogston (1966) investigated movement coordination interactions using film analysis. They found harmonious or synchronous organizations of change between body motion and speech in both intrapersonal and interpersonal behavior. The dynamical processes of self-organization captured by the HKB equation are the basis for the behavioral phenomenon of interactional synchrony. Using the modified visual wrist-pendulum task, Schmidt and O’Brien (1997) examined whether such unintentional entrainment would occur. Participants were asked to swing a pendulum at its comfort mode tempo. During the first half of a trial, pairs of participants were instructed to look straight ahead so that they could not see each other. During the second half of a trial, participants were asked to look at the other participants’ moving pendulum but maintain their preferred tempo from the first half of the trial. In the first half of the trials, no phase entrainment was observed. The relative phase angle was not constant and all phase angles were equally observed in the coordination of the participants’ pendulums. In the second half of the trials, the relative phase angle was not constant and all phase angles were also observed but this time not equally: relative phase angles near the attractor regions predicted by the HKB model (near 0 and 180°) tended to dominate. The participants’ movements were weakly phase entrained. The coordination was not the absolute coordination (phase locking) exhibited in intentional visual coordination but rather relative coordination produced by dynamical systems with weak attractor basins and intrinsic noise. Such dynamical systems demonstrate the property of intermittency: a constant change in state with an attraction to certain regions of their underlying phase space. Schmidt and O’Brien (1997) further examined the property of intermittency measuring the rate of change of relative phase angle (φ). In the first half of the trials, the average Δφ calculated for nine 20° relative phase regions between 0 and 180°is

3.1 Unintentional Interpersonal Entrainment

7 Trial Half 1 Trial Half 2

6

5 DF

Fig. 3.3 The rate of change of relative phase (Δφ) at different relative phase regions without visual information (trial half 1) and with visual information (trial half 2) (Schmidt and O’Brien 1997, redraw from Schmidt and Richardson 2008). With vision available, Δφ tends to decrease near 0 and 180° suggesting un intentional intermittent entrainment

113

4

3

2 0

40

160 80 120 Relative Phase Region

200

larger and shows no pattern. But, in the second half of the trials when vision was available, the average Δφ across the relative phase regions has an inverted u-shape (Fig. 3.3). The minimum rates of change near 0 and 180° suggests that the system is attracted to these regions. These results demonstrate that the dynamical principles of self-organization in the HKB model can constrain interpersonal coordination unintentionally. Subsequent studies confirmed this basic conclusion found by Schmidt and O’Brien (1997) using more naturalistic interaction circumstances. Neither looking at another’s movements, nor swinging a wrist pendulum is a typical everyday task. To further reveal the relationship between visual information and entrainment, Richardson et al. (2007) examined the unintentional coordination of movements between two people rocking in rocking chairs. Richardson and colleagues examined whether participants would unintentionally entrain their rocking movements, and they manipulated the amount of visual information each participant had of each other’s rocking movements. To manipulated the information available, the experiment was examining how different postural configurations affected the stability of rocking, and participants had to turn their head to focus on the red target that was located either on the arm rest of their partner’s chair, directly in front of them, or on the side away from their partner. These locations corresponded to the participants having focal, peripheral, or no information about their partner’s movements. Figure 3.4 showed that entrainment is observed for the focal information condition significantly more than for the peripheral information condition. Unintentional

3 An Overview of the Study on Interpersonal Coordination

% Occurrence

114 35

No information

30

Peripheral condition Focal condition

25 20 15 10 5 0

30

50 70 90 110 130 Relative Phase Region

150

180

Fig. 3.4 Distribution of relative phase angles between the rocking chair movements of co- participants as a function of the availability of visual information (Richardson and Schmidt 2006, redraw from Schmidt and Richardson 2008). More unintentional entrainment near inphase was found for the focal as compared to the peripheral information condition

entrainment to the in-phase mode occurs and the degree of attraction to this mode is influence by the degree of information available. This and previous studies thus provide clear evidence that the dynamical processes of self-organization modeled by the HKB equation can constrain the unintentional synchrony in these laboratory tasks and are possibly response for the synchrony observed in ordinary, everyday social intentions.

3.1.4 I nterpersonal Synergy Involving Intrapersonal Movements Although Schmidt and O’Brien (1997) and Richardson et al. (2005) examined the interpersonal synchronization of isolated limb movements using wrist-pendulum task, natural interpersonal interactions are whole body interactions that involve intrapersonal subtasks. For example, two people have a conversation while lifting and carrying a table in real life. The interpersonal coordination in their conversation and the interpersonal rhythmic synchrony of their leg movements occur along with other intrapersonally coordinated subtasks, in this case, carrying a table. The organization of these subtasks needs to be coordinated within each person. Interpersonal coordination thus occurs in a context of ongoing intrapersonal coordination (see Chap. 6 as a motor control hierarchy in interpersonal coordination that involves bimanual movement). In dynamical motor control terminology, interpersonal coordinative structure or synergy that self-organizes across two people in a naturalistic interaction involves

A

Oscillating Stimulus

Relative Phase Occurrence (%)

3.1 Unintentional Interpersonal Entrainment

115

30

Control Non-Tracking Tracking

25 20 15 10 5 0

0

30

120 150 60 90 Phase Region (˚)

180

Fig. 3.5 Experimental setup of environmental entrainment (left panel) and distribution of relative phase angles between the visual stimulus and participants’ wrist movements as a function of visual tracking conditions (right panel) (Schmidt et al. 2007, redraw from Schmidt and Richardson 2008). More unintentional entrainment near inphase was found for the tracking compared to the non- tracking condition

speech, bimanual, bipedal and postural intrapersonal synergies that organize the whole body movements of an individual. How do the various intrapersonal synergies interact with or mediate the interpersonal entrainment that emerges in an interaction? In addition, our environmental interactions involve the active pickup of visual information that requires eye movements. Franz et al. (2001) find the constraints that movements of one effector have on the movements of another. In spite of the work of Gibson (1979), however, because visual systems generally conceived as static, many studies ignore the constraints that eye movements have on the coordination of effector movements that are occurring simultaneously. However, because previous studies find a high degree of intrapersonal coordination between limb and saccade eye movements (e.g., Henriques and Crawford 2002), the eye movements constraint the action you make. Consequently, Schmidt et al. (2007) examined whether participants would unintentionally entrain to a rhythmically moving stimulus. The left panel of Fig. 3.5 showed environmental entrainment setup to examine the role of eye movements on unintentional entrainment. Participants were instructed to read aloud letters that randomly appeared on a projection screen while simultaneously swing a wrist- pendulum. In addition to the letter stimuli, a sinusoidally oscillating stimulus moved horizontally across the screen. The speed and accuracy of their reading were measured while their wrist movements and the oscillating stimulus were motor and perceptual distracters. The aim of the experiment was to examine whether participants would entrain their wrist movements to the oscillating stimulus and whether visual tracking the stimulus would facilitate this unintentional entrainment. Visual

116

3 An Overview of the Study on Interpersonal Coordination

tracking was manipulated by controlling where the letter to be read appeared. When the letters appeared in the center of the screen, the participants were required to fix their gaze directly at the center of the screen (non-tracking condition). When the letters appeared on the visual stimulus, the participants needed to track the stimulus with their eyes as it oscillated from side to side in order to read the letters (tracking condition). To measure chance level coordination, trials were performed in which the letters appeared in the center of the screen along with an invisible oscillating stimulus (control condition). The right panel of Fig. 3.5 showed that the tracking condition produced greater unintentional entrainment than the non-tracking condition. However, the non-tracking condition still exhibited some unintentional phase entrainment near 0 and 180°. In addition to an oscillating, does eye tracing facilitate the unintentional entrainment to the movements of another person? As a follow-up study, Schmidt and Richardson (2008) examined whether this result generalized to interpersonal rhythmic coordination using the dyadic problem-solving paradigm of Richardson et al. (2005) with tracking and non-tracking conditions. In the visual tracking condition, the cartoon pictures were attached to the end of the pendulums such that each participant had to visually track the motion of their co-actor’s movements in order to complete the dyadic task. In the non-tracking condition, the pictures were displayed on a floor stand positioned directly behind the motion of their co-actor’s pendulum. Thus, no tracking movement of the eyes was asked to perform the problem-solving task. The wrist movements of the participants became more strongly entrained when visual tracking of their partner’s movements was asked to complete the dyadic puzzle task. The results of both the environmental and interpersonal entrainment studies suggest that dynamical and interpersonal coordinative structure includes the movements of two people coupled via an active and visual information pickup dynamic which is intrapersonally coupled to the individual’s limbs. Does the interaction of separate intrapersonal synergy such as rhythmic wrist and eye tracking constrains the creation of an interpersonal synergy? In future study, we pay attention to the composite nature of the whole body system in studying environmental and interpersonal interaction. A full understanding of the rhythmic synergy underlying interpersonal coordination involves understanding the intrapersonal coordination of motor and perceptual rhythms. Another behavior found in natural interpersonal interaction that needs to be intrapersonally coordinated with other body movements is speech. Shockley et al. (2003) examined whether conversation is sufficient to entrain the postural sway of two interacting participants. Using a dyadic puzzle task, participants interacted visually and verbally, interacted just verbally, interacted just visually, or did not interact. Postural entrainment did not occur when participants had visual information about the other person available by itself, but did when participants were interacting verbally. Using the interpersonal wrist-pendulum task, Richardson et al. (2005) ran similar verbal condition to examine whether such verbal information would affect interpersonal entrainment of rhythmic limb movements. In contrast to the results of Shockley et al. (2003), Richardson et al. (2005) found that entrainment occurred when the participants could see each other but only a chance level

3.1 Unintentional Interpersonal Entrainment

117

0.08 % Occurrence

Period Difference (s)

0.10

0.06 0.04 0.02 0.00

control no read out loud Condition

silent

18 16 14 12 10 8 6 4 2 0

control no read out loud silent

0

40 80 120 160 200 Relative Phase Region (˚)

Fig. 3.6 Period difference (left panel) and distributions of relative phase angles between the letter- to-be-read stimulus and participants’ wrist movements (right panel) for the different rhythmic speed conditions (Schmidt 2005, redraw from Schmidt and Richardson 2008)

e ntrainment occurred when information was transmitted verbally. An explanation for why interpersonal postural entrainment is affected by speech but limb entrainment is not has been suggested by a follow-up study. Using a similar method, Shockley et al. (2007) found that verbal entrainment of posture does not occur on the basis of perceived speed signals but rather is dependent upon the rhythmic nature of the speech productions. Because speech rhythms of two people in conversation are coordinated and these speech rhythms produce postural changes, the postural sway between the two interactions becomes coordinated. Furthermore, why was not the interpersonal entrainment of limb movements influenced by speech rhythms in the study of Richardson et al. (2005)? Chui (2005) showed intrapersonal synchronization of speech rhythms and hand movements. Furuyama (2002) also reported that interpersonal gestural synchronization occurs in natural conversation although it depends heavily on the communicative and functional context of a gesture. Rhythms latent in the interpersonal conversation may be too subtle to have any observable influence on the participants’ rhythmic limb movements in the study of Richardson et al. (2005). There seems to be a lack of intrapersonal intimacy between the rhythms of a natural conversation and the sinusoidal rhythms of wrist-pendulum swing. Schmidt and Richardson (2008) further examined a coupling between rhythmic limb movements and speech. Participants were asked to read letters that appeared rhythmically in the middle of a computer screen while swing a wrist-pendulum at a comport mode tempo. Although the participants were asked to read letters fast and accurately, experimenters really measured the degree of unintentional entrainment between the wrist-pendulum swing and the rhythm of the appearing letters. This entrainment was evaluated under four speech conditions: out loud reading, silent reading, no reading with a rhythmically appearing visual stimulus, and no reading with no visual stimulus (control condition). The tempo of the appearing letters was also manipulated and presented at a period equal to the participant’s self-selected comfort mode tempo or at a period that was slightly faster or slower than that tempo. The left panel of Fig. 3.6 showed that the difference between the period of the wrist

118

3 An Overview of the Study on Interpersonal Coordination

movements and the rhythmic stimulus decreased when either out loud or silent speech was performed. The right panel of Fig. 3.6 further showed that the relative phase distributions indicated a tendency to be at the front or back of the cycle of the pendulum (0° was defined as the pendulum being away from the screen and 180° was defined as the pendulum being toward the screen) when the letter appeared in either the out loud or the silent speech conditions. These results indicate that speech rhythms entrain limb rhythms in the two reading conditions. In addition, if the speech rhythm is synchronized with an environment rhythm, speech can mediated the entrainment of a limb rhythm and an environmental rhythm. Results under the silent reading indicate that overt motor speech movements are not necessary for the entrainment between speech rhythms and limb rhythms. The attention rhythms rather than the movements play an important role for the entrainment. In the studies of Richardson et al. (2005) and Shockley et al. (2003), however, participants’ communication was not directly associated with the task of the periodic limb movement. In contrast to these previous studies, when two people row a boat, they often call to each other to synchronize their strokes. Such a call perhaps plays an important role in periodic joint action. Such a call in rowing a boat is thought to have a strong influence on periodic joint action. It is predicted that periodic speech promotes periodic joint action. As described in Sect. 4.4, Masumoto and Inui (2014b) thus examined the effects of speech on both complementary and synchronous production of periodic isometric forces.

3.1.5 Informational and Dynamic Constraints on Entrainment As described already, interpersonal entrainment emerges from dynamical laws operating via informational interactions. Individual differences observed in previous studies (for a review, Schmidt and Richardson 2008) show that interpersonal entrainment does not always occur. Such entrainment occurs only if certain conditions are fulfilled. This subsection thus addresses the dynamical and informational constraints on unintentional interpersonal entrainment. Equation (3.2) suggests that the weak phase entrainment seen in the unintentional interpersonal coordination will occur as long as the difference in inherent tempos of the oscillations (Δω) is not much larger than the coupling strength. If two participants establish comfort mode tempos that are very different, the coupling dynamic (2) may not be strong enough to parry their respective ‘maintenance’ tendencies. In the experiment of Schmidt and O’Brien (1997), Δω was manipulated by having participants swing pendulums of same or different lengths. Relative phase angles around the attractors of 0 and 180° was found when the participants swung pendulums of different lengths less than when they swung pendulums of similar length. Figure 3.7 further shows a reanalysis of data of Richardson et al. (2005). This reanalysis suggests that there is a range of period differences over which interpersonal coordination will occur and that beyond this range the occurrence of unintentional

3.1 Unintentional Interpersonal Entrainment

119

1

Coherence

0.8 0.6 0.4

R2 = 0.59

0.2 0 0

0.05

0.1

0.15

0.2

Absolute Period Difference

Fig. 3.7 Cross-spectral coherence as a function of the difference in the natural (self-selected) period of participants’ wrist-pendulum movements (Richardson et al. 2005, redraw from Schmidt and Richardson 2008). The natural period for each participant in a pair was calculated during no visual information control trials. The coherence decreases as the period difference (naturally occurring Δω) increases, suggesting the existence of a period basin of entrainment

coordination is highly unlikely. In other words, there appears to be a period basin of entrainment for unintentional interpersonal coordination. Using the environmental coordination paradigm, Schmidt and Richardson (2008) measured this period basin for unintentional coordination to an environmental rhythm. Participants were asked to read letters presented on a sinusoidally oscillating visual stimulus while swinging a pendulum at a comfort mode tempo. This task was the same as the visual tracking condition of Schmidt et al. (2007). The participant’s comfort mode tempo was determined in the start of the experiment, and then the period of the oscillating stimulus was manipulated in about 25 ms increments over a range 200 ms above and below the comfort mode tempo. Schmidt and Richardson (2008) found that rhythmic limb movements become unintentionally entrained to the environmental rhythm when the period of the environmental rhythm was within ±150 ms of the individual’s natural period. This rage is similar to that suggested by the interpersonal data in Fig. 3.7. A variable that may be related to size of this basin is the amount of period variability exhibited by an individual. Because the cross-spectral coherence between participants’ wrist movements and an oscillating visual stimulus mirrors the mean distribution of participants’ cycle-to-cycle periods (Fig. 3.8), Lopresti-Goodman et al. (2006) examined whether pairs of participant with more overlap of their distributions of period will tend to have greater basins of entrainment and thus are more likely to become unintentionally entrained. A question further concerns the constructive effect of movement variability or noise (Collins 1999) on unintentional interpersonal coordination. Such noise-based enhancement reflects a process known as stochastic resonance, which occurs when the flow of information through a nonlinear system is maximized by the presence of sub-threshold noise (Collins 1999). Thus, not only might an individual’s movement variability be a parameter constraint on visual entrainment, but it is also possible that such noise may operate to increase the stability of entrainment. Using the environmental coordination paradigm and

120

3 An Overview of the Study on Interpersonal Coordination Distribution of Subjects’ Cycle-to-Cycle Period -3 SD -2 SD -1 SD Mean 1 SD 2 SD 3 SD

0.6

0.2

0.5

0.1

0.3 Chance 0.2

R2 = 0.97

0.05

% Occurrence

Mean Coherence

0.15 0.4

0.1 0 -0.20

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

0 0.20

Difference between Subject and Stimulus Period

Fig. 3.8 Cross-spectral coherence between a participant’s wrist-pendulum movements and an oscillating stimulus as a function of the difference between the participant’s natural period of movements and the stimulus period (solid line, black dots, bottom x-axis, left y-axis). Average period distributeon of participants swinging a wrist pendulum for a 30-s trial (dashed line, white dots, top x-axis, right y-axis). (Redraw from Schmidt and Richardson 2008)

adding different magnitudes of sub-threshold period variability to the trajectory of the oscillating stimulus, Richardson et al. (2007) examined this possibility. Participants are expected to become entrainment to the visual stimulus when the stimulus is equal to the participant’s comfort tempo and contains no variability. However, more unintentional coordination is expected to be observed for noisy stimulus when the stimulus’ period is less than or greater than the participant’s comfort tempo. Thus, the basin of entrainment for unintentional coordination is expected to be extended for a noisy’ stimulus compared to a non-noise stimulus. Previous studies on locomotion (Warren and Yaffe 1989) and rhythmic juggling (Santvoord and Beek 1994) have reported that the pickup of information during rhythmic tasks is not uniform and that the information available at certain discrete points may not be more important for stable coordination. In line with these reports, Roerdink et al. (2005) have shown how individuals more often fix their gaze on the endpoints of an oscillating visual stimulus during a manually tracking task. Byblow et al. (1994) have also found that the peak extension and flexion points of rhythmically moving limbs are the perceptual pickup points for bimanual coordination. Thus, the pickup of information at the endpoints of a movement is important for visual coordination. Top left and bottom panels of Fig. 3.9 show that participants are asked to intentionally coordinate with an oscillating visual stimulus that had different phase regions and phase amounts of stimulus’ trajectory occluded from view (Schmidt and Richardson 2008). From an analysis of the entrainment variability, top right panel of Fig. 3.9 shows that wrist-stimulus coordination is significantly less stable when occluding the endpoints than when occluding the middle phase regions.

3.1 Unintentional Interpersonal Entrainment

121

Occluded Phase Region

80° 120° 160°

SDφ

20

15

10 0°/180°

90°/270° 45°/135° Location

Location 0°/180°

90°/270°

45°/135°

Amount

80°

120°

160°

Fig. 3.9 Experimental setup (left panel) and stimuli (bottom panel) used to study effects of visual occlusion (both location and amount) on intentional visual entrainment (Hajnal et al. 2006, redraw from Schmidt and Richardson 2008). The variability of relative phase is only increased by occluding the ends (0°/180°) of the trajectory

When the middle phase regions were occulted, the stability of the coordination was the same as when none of the stimulus trajectory was occluded. In addition, the amount of phase occlusion was found to have no affect on the stability of visual coordination. The above studies have examined effects of the informational and dynamical constrains on the stability and emergence of unintentional interpersonal coordination. The results may provide understanding when and why interpersonal interactions succeed or break down.

122

3 An Overview of the Study on Interpersonal Coordination

3.1.6 Social-Psychological Variables In the interpersonal coordination studies, Schmidt and Richardson (2008) have reviewed how the stability of interpersonal coordination is influenced by variables of action and perception. In addition to dynamical principles of motor synergies and their variables, if these action and perception processes occur within a social context, we can examine how the stability of coordination dynamics is influenced by variables at the social-psychological scale. This question is investigated by social psychologists in their studies of interactional synchrony in adults (Julien et al. 2000) as well as infants (Feldman and Eidelman 2004; Isabella and Belsky 1991). These studies find that social variables such as attachment and rapport as well as psychological variables such as learning disabilities, mental illness, and expressivity constraint the degree of interactional synchrony observed in interactions. While these studies involve naturalistic tasks where the interpersonal coordination is implicit within the interaction, the problem with such studies is the difficulty of measuring the interactors’ movements and evaluating their coordination. Thus, Schmidt et al. (1994) examined the influence of social variables on interpersonal entrainment using more stereotypic perception and action tasks to circumvent these methodical problems. Schmidt and colleagues used intentional anti-phase wrist-pendulum coordination to examine the effects of social competence on interpersonal coordination stability. Participants were asked to swing three pairs of pendulums whose length differences comprised three levels of detuning (Δω = −0.32, 0, and 0.32) at two tempos (slow: 0.65 Hz and fast: 1.5 Hz). Using a concise version of the Riggio social skills inventory (1986), participants were selected to create homogeneous social competence dyads (both participants having high or both participants having low social competence) and heterogeneous dyads (one participant having high and the other having low competence). Socially competent people presumably have greater skill in social interactions, and are more adept at perceiving the various possibilities for social action, by being more attuned to the facts of a situation and what types of actions they required. Thus, it is predicted that pairs of participants with higher social competence would have more coordinated interactions than those with lower social competence. The heterogeneous (high-low) pairs exhibited significantly fewer breakdowns in coordination than both the homogeneous (high-high and low-low) social competence pairs (high-high: 50%, low-low: 52%, high-low: 86% phase-locked trials), indicating a stronger coordination dynamic for the heterogeneous pairs. This result was primarily observed for the faster tempos when coordination is generally less stable. In addition, for the trials that were phase-locked, only the high-high pairs did not show the typical phase lag relationship with the detuning variable Δω (i.e., the shorter pendulum leading in its cycle) that is predicted by (2). This result suggests that these high-high pairs did not use a dynamical strategy for solving their coordination problem, noting that the measure of social competence correlated with a social control subscale. This correlation suggests that the type of competence characterizing the dyads was that of leadership or dominance. The results indicate that

3.1 Unintentional Interpersonal Entrainment

123

reciprocity (leader-follower) rather than symmetry (leader-leader or follower- follower) of this social competence facilitated the social coordination. The high- competence individuals would be more likely to sustain a stronger tendency to maintain their own preferred dynamic than the low-competence individuals. Two results suggest that the high social competence individuals had a stronger maintenance tendency when paired with a low-competence partner. First, the high- competence participants in these pairs led the low-competence participants in their cycles (by 2°, although a non-significant difference). Second, the high-competence participants had lower fluctuations in their periods of oscillation than the low- competence participants. Taken together, these results suggest that a person’s personality characteristics become embodied in their movement and these personality traits thus constrain the synchrony of their social interactions. Moreover, Richardson collaborated with social psychologists, and examined whether the dynamics of entrainment would reflect the participants performed a cooperative or a competitive dyadic task using the rocking chair unintentional coordination paradigm (2007) and the dyadic problem-solving task (2005). A pair of participants sat in rocking chairs side by side. Cartoon faces were attached to their armrests such that each could only see their partner’s cartoon face. The participants were asked to determine the differences between the two cartoon faces. However, this task was performed in one of two ways to make it either a competitive or a cooperative task. After the task was performed, all participants were asked to rate how much they linked their partner and how pleasant they found the interaction. Cooperative pairs exhibited more in-phase coordination and marginally greater cross-spectral coherence than the competitive pairs, indicating an influence of the social goal on the coordination dynamics. Regression analyses further found that the degrees of liking and perceived pleasantness of the interaction were each correlated with the cross-spectral coherence. These results suggest that the social circumstance in which individuals find themselves will constrain the degree of unintentional entrainment and that the stability of this interactional synchrony reflects the experience of psychological connectedness between the participants. Summing up a series of studies of Schmidt, Richardson and colleagues (for a review 2008), first, the processes of interpersonal coordination need to be understood in terms of the dynamical processes of self-organization. The work of Kelso and colleagues in intrapersonal coordination dynamics is extended for understanding the dynamics of interpersonal entrainment. Second, because these dynamic processes of interpersonal coordination operate automatically, outside of their awareness, the processes seem to be unintentional synchrony observed in natural interactions. Third, in order to understand the perception and action processes that sustain dynamical interpersonal synergies, Schmidt and colleagues indicate not only the dynamical and informational constraints on these processes but also how intrapersonal perception and productions rhythms interact with limb movements to create whole-body interpersonal synchronization. Fourth, the psychology properties of a person and the social properties of the entrainment constrain the dynamics of interpersonal coordination.

124

3 An Overview of the Study on Interpersonal Coordination

3.2 Intentional Interpersonal Coordination 3.2.1 Perception–Action Matching On the relationship between perception and action, whereas some researchers claim direct links between perception and action that do not involve mental representation (Gibson 1979), others point out that perception and action are tightly linked through common underlying representation (Prinz 1997). When we observe somebody acting, this activates a corresponding representation of the action in our own motor system. Such relationship between perception and action through common representation is referred to the term ‘perception-action matching.’ The common representation may help to imitate, anticipate, and understand the action of others. According to the common representation (Prinz 1997), actions are coded in term of their distal perceptual consequences. For example, Ford et al. (2006, 2009) have found that, when kicking a ball, the kicking action is specified in term of the visual consequence of the action (the ball trajectory) rather than in terms of the particular muscles to be moved. From the assumption that actions are coded in terms of their effects, it follows that perceived actions and the actions we produce ourselves can be represented in same format. In other words, a ball trajectory can be represented in the same way regardless of whether I am planning to throw the ball or you have kicked it. Because similar motor representations are activated for actions we observe and for actions we are able to produce (Brass et al. 2001), this ‘common coding’ creates an interface between action execution and action observation. From behavioral studies on perception-action matching, Brass et al. (2001) have found that people are faster at lifting their finger when they concurrently watch someone lifting a finger (a corresponding movement) than they watch someone moving a finger down (a opposite movement). Many neurophysiological studies have reported the neural basis of perception-action matching. Single neuron studies discovered mirror neurons in motor cortices of the macaque monkey (Gallese et al. 1996). The mirror neurons fire not only when the monkey performs an action, but also when the monkey observes the same action performed by someone else (for a review, Rizzolatti and Sinigaglia 2010). Fadiga et al. (2005) find an analogous mirroring circuitry in the human brain. The human mirror system containing the premotor cortex and inferior parietal lobule similarly responds to observation of corresponding actions (Carvo-Merino et al. 2005; Kilner et al. 2009). If action perception and action execution activate common motor representations, the level of motor skill should influence the extent to which action-related brain regions respond to observed actions. As described in Sects. 2.4 and 2.8, some neuroimaging studies find an increase in the activity of the mirror neurons system when people observe movements they are able to perform compared with movements that are not in their motor repertoire. Using an fMRI, for example, Calvo- Merino et al. (2005) found brain activation when expert ballet and capoeira dancers watched ballet and capoeira moves. Higher activation in the observers’ mirroring areas was recorded when they watched movements in which they had expertise.

3.2 Intentional Interpersonal Coordination

125

Cross et al. (2006) also used an fMRI and tracked changes in mirroring activity as expert dancers learned novel dance sequences over the course of 5 weeks. A higher activation of the mirror neurons systems was elicited when the dancers watched sequences that they had learned than when they watched unfamiliar sequences. These results of neuroimaging studies suggest that the match between perceived and performed actions allows the observer to simulate observed actions as they unfold in real time (Wolpert et al. 2003). The motor system of the observer runs these stimulations using internal models, which comprise one’s bodily mechanics and previously learned links between actions and their consequences. The same internal models that guide execution of actions are put into use when observing actions, and these internal models serve anticipations about the likely outcomes of the observed actions (Knoblich 2008). Such claim is demonstrated by some studies on motor laws in perception. If the motor system runs simulations of the observed actions, then the motor laws that govern execution of actions should govern their perception in similar ways. For example, Fitts’s law (Fitts 1954), which formulates the speed-accuracy constraints in biological movements, applies to perception of movements as well (Eskenazi et al. 2009; Grosjean et al. 2007). Similarly, if internal models are applies to the observed actions, one should be better at anticipating the outcome of recordings of one’s own actions than at anticipating the outcomes of other’s actions. For example, studies on clapping (Flach et al. 2004), dart throwing (Knoblich and Flach 2001), and piano performance (Repp and Knoblich 2004) have supported the hypothesis that the match between the observed action effects and the corresponding motor program of observer is highest when one perceives one’s own action effects. The notion of internal models also predicts that, because sports experts presumably have a higher perceptual skill than novices, the experts should be better than the novices at anticipating the outcomes of actions that are part of their action repertoire. As described in Sect. 2.9, Aglioti et al. (2008) compared the ability of professional basket players and basket reporters to anticipate the success of basket shots. The professional group was better at anticipating the outcomes of shots than the reporters who were only visually familiar with basketball moves. Internal models may not only serve to anticipate what others are going to do next, but also help to anticipate the timing of others’ actions, which is essential for intentional interpersonal coordination (Sebanz and Knoblich 2009). The mechanisms of anticipation people use when they perform an action may be used to anticipate the actions of others, and aid temporal coordination of one’s own actions with those of others. As described in Sect. 2.11, as an indirect evidence, Keller et al. (2007) have examined whether people are better able to synchronize with their own earlier actions than with the actions of others. Expert pianists were asked to perform parts of duet pieces. The pianists were then asked to coordinate with previously recorded pieces, performed either by themselves or by another pianists. As expected, the synchronization accuracy was best when the pianist performed a duet with their own recordings, suggesting that it is easier to coordinate with people who are similar to oneself in terms of motor performance. Thus, expert athletes and musicians achieve

126

3 An Overview of the Study on Interpersonal Coordination

better coordinate when playing with team members who are at a comparable skill level, and who have leaned to perform particular movements in a similar way. On the other hand, what do actors anticipate about the actions of others in interpersonal coordination? For example, a team of rowers may learn to coordinate strokes by learning to anticipate the combined consequences of their strokes on the movement of the boat, rather than making specific anticipations about each other’s strokes. Knoblich and Jordan (2003) asked participants and pairs to perform a motor task keeping a tracker on a target. The target was moved horizontally across a computer monitor by pressing right and left keys that incremented the tracker’s velocity to the right or left, respectively. van der Wel et al. (2011) also asked participants to move a pole back and forth between two targets. The participants did so by pulling on cords attached to the base of the pole, one on each side. In the individual condition, one participant performed this task bimanually, and in the joint condition two participants each controlled one cord. In these studies, the participants may anticipate the consequences of the combined effects of their own actions and the actions of others on the environment. Thus, through training, it is possible to integrate anticipations about one’s own actions and the actions of others. Two people coordinating their actions to control a moving object became as coordinated as single individuals performing the same task bimanually.

3.2.2 Shared Intentionality If two people prepare a meal together, they do not start moving around in the kitchen without a plan for intentional interpersonal coordination. Two people first talk about a recipe, and decide on a joint action plan that specifies who take over which tasks. Then, each of them conducts what they planed. This procedure indicates the nature of shared intention and the role of commitment. Are shared intentions fundamentally different from individual intentions? Should we conceive of shared intentions as individual intentions that interlock in particular ways? Is commitment a constitutive part of intentional interpersonal coordination? In addition, is it possible to perform a joint task without having a shared intention? Could there be minimal commitment that arises from acting together? In particular, answers to these questions are important for understanding coordinated interactions that happen within split seconds, where there is no time for prior planning. It is sufficient for individuals to have a representation of the goal of interpersonal coordination (i.e., the desired outcome of joint action) in addition to having a representation of their own task. While philosophers theoretically discuss these questions (for a review, Butterfill 2018), psychologists and neuroscientists experimentally examined the notions of shared intention and commitment (for a review, Knoblich et al. 2011).

3.2 Intentional Interpersonal Coordination

127

3.2.3 Shared Representation Perception-action matching is a crucial component of intentional interpersonal coordination. In soccer or basketball, for example, when a passer passes a ball to players of his own team, the passer will consider the specific role of the player (defense or attack) towards whom he is directing the ball. It is thus necessary for perception-action matching to know what a partner’s task is and to make anticipations about his actions. Using Simon task (Simon et al. 1970), Sebanz et al. (2003) examined whether individuals form representations detailing not only their own tasks, but also the task of their interaction partners. In the classical Simon task, participants are asked to respond to one of two stimulus features. If the ring is red, a left key press is required. By contrast, if the ring is green, a right key press is required. The general result is that the participants are slower in responding with the left key when the finger is pointing to the right, and vice versa, indicating a compatibility effect. The spatial stimulus feature (pointing direction) automatically activates the spatially corresponding response. In a social version of the Simon task, Sebanz et al. (2003) asked two participants to perform the task together. Each participant was required to respond to only one of the two colors of the ring placed on the finger. The participant sitting on the left was to respond with the left key press to red, and the one sitting on the right with the right key press to green. In this go/no-go task, the irrelevant stimulus feature (direction of the pointing finger) interfered with the participants’ response. The compatibility effect was observed only when participants performed the task with another person. However, when participants performed the same half of the task on their own, the compatibility effect not occur. This finding indicates that, when performing the task together, two participants share a common representation to perform their task. A shared representation does not require the actual presence of an interaction partner. The mere belief that another person is performing a particular task is sufficient for representing their task (Tsai et al. 2006). However, when participants were told that a computer was taking care of the other task, this task was not represented. Atmaca et al. (2011) showed that a partner’s task is represented only when the partner acts intentionally and not when a machine is controlling his or her actions. In addition to the perceived intentionality of the co-actor, there are some other factors to modulate a shared representation of a task. First, in intentional interpersonal coordination, the correspondence between an individual’s goal and the partner’s goal defines the social context, i.e., competition or cooperation (van Avermaet 1995). If the two goals are complementary or the same, the partner cooperates. By contrast, if the attainment of one person’s goal would result in the failure of the other person to achieve their goal, the partner competes. Some studies have

128

3 An Overview of the Study on Interpersonal Coordination

found that cooperation and competition are associated with different cortical activity, as measured by fMRI (Decety et al. 2004; de Bruijn et al. 2009), and differences in behavior (Braun et al. 2009; Becchio et al. 2008; Ruys and Aarts 2010). In an auditory version of the social Simon task, Ruys and Aarts (2010) found social contexts where the individual goal to receive a monetary reward was linked to one’s task partner, regardless of whether people were cooperating with or competing against the task partner, compared with a context in which individuals’ goals were independent of their partner’s goals. This finding suggests that a shared representation of the task is not restricted to joint action, but serves a general role in generating anticipations about others’ actions. To the contrary, Iani et al. (2011) showed that another’s task is represented only in a cooperative setting, but not in a competitive setting. Second, affect also seems to play a role in a shared representation of a task. Hommel et al. (2009) reported that participants showed shared representation effects when they jointly performed a task with a confederate who was friendly and cooperative, but not when they performed with one who was competitive and intimidating. Kuhbandner et al. (2010) also found that effects of shared representation were shown to occur when participants were in a positive or neutral mood, but not when they were in a negative mood. Furthermore, a shared representation of a task may allow us to monitor other’s performance and notice mistakes committed by our interaction partners. Van Schie et al. (2004) examined the neural mechanisms underlying observation of erroneous actions by means of human event-related potentials (ERP). They reported that observing another’s error elicites an error-related negativity of the ERP component that is elicited by self-generated errors. Schuch and Tipper (2007) also showed that participants become slower and more accurate in their subsequent actions not only following their own errors, but also following an interaction partner’s errors. de Bruijn et al. (2009) asked participants to play a computerized game with another person, either together or against one another, finding that observing others’ error during competition activated reward-related brain region in the observer. Although there are some pieces of evidence to suggest that similar representations and processes underlie individual and joint task performance, there may be important differences between individual and joint performance. In line with this view, Milanese et al. (2010) used a Simon task to investigate whether it is possible to obtain transfer-of-learning effects from a joint to a joint context, from a joint to an individual context, and from an individual to a joint context. Participants performed a Simon task after practicing a spatial compatibility task with an incompatible stimulus-response mapping. The incompatible practice performed in a joint context affected performance in the subsequent Simon transfer task, as long as the latter was performed jointly. The individual practice task transferred to the joint Simon task, but not vice versa (also see Milanese et al. 2011). These findings suggested that individually and jointly acquired stimulus-response associations remained functional in joint actions, whereas jointly acquired stimulus-response associations did not transfer to individual task performance. The fact that joint practice does not

3.2 Intentional Interpersonal Coordination

129

affect subsequent individual performance indicates that the partner’s actions may not always be represented in a functionally equivalent way as one’s own. On the contrary, as described in Chap. 5, Masumoto and Inui (2017) showed bidirectional transfer between individual and joint conditions for the accuracy and variability of force production. Whereas participants processed spatial compatibility or incompatibility in the Simon task (Milanese et al. 2010, 2011), Masumoto and Inui (2017) controlled the timing and magnitude of force production in a task of discrete force production. Such a difference between tasks appears to result in the difference between the results of the two studies on learning transfer.

3.2.4 Coordination Strategy We sometimes perform interpersonal coordination without having detailed representations about a co-actor or a co-actor’s task when there is not enough time to form such representations or when insufficient information is available. Problems of interpersonal coordination under minimal information have been examined not only in decision-making tasks where people have to match their choice but also in situations where people perform interpersonal coordination together in real time. For example, think of a surprise birthday party where all friends want to start singing ‘happy birthday’ at exact moment the to-be-surprised person has entered the room. In fact, this is a difficult task because if everyone took their time starting or if everyone waited for the others to begin, the outcome would most likely be a highly uncoordinated performance. However, people were often able to start singing together in synchrony, even in the dark. How is synchronized action performance with minimal information about other’s action achieved? One coordination strategy is making oneself predictable (Vesper et al. 2011), which is achieved by performing actions as constantly as possible. In other words, the strategy is achieved by reducing the variability of one’s own actions. If faster actions generally tend to be less variable (Repp 2005), making oneself predictable is sometimes achieved by speeding up, that is, by responding as fast as possible after a common event in the environment (Vesper et al. 2011, 2016). Thus, in order to start singing ‘happy birthday’ at the same time, everyone should start singing as fast as possible after the birthday celebrant has walked in, in doing so, reducing overall action variability.

3.2.5 Perceiving Others’ Abilities More information about others’ actions is available in interpersonal coordination than in the situations described in the previous subsection. We often consider details about our interaction partners, including their feature and abilities, and we may be

130

3 An Overview of the Study on Interpersonal Coordination

sensitive to the relation between our own and their actions. Some studies have demonstrated that we perceive relations between our own and others’ possible actions in a particular context. These joint action affordances constitute or reflect our perception of what we can achieve together with others. For example, Richardson et al. (2007) found that participants take into account the abilities of all the participants when deciding whether to carry an object alone or together with another. In addition, people take into account the combined abilities and anatomical characteristics of all agents involved (e.g., combined arm span, or joint distance and speed) whether to hold the door open for others (Santamaria and Rosenbaum 2011) or whether one would fit through a door frame walking next to someone (Davis et al. 2010). Joint action affordances also change flexibly depending on the characteristics of another person. Doerrfeld et al. (2012) found that participants systematically underestimated the weight when judging the weight of boxes that they would later be lifting together. Heavy boxes were judged to be lighter than they actually were. However, if the co-actor wore a bandage implying an arm injury, participants judged the weight of boxes as heavier, suggesting that people consider potential co-actors in terms of how much they can contribute to the joint action.

3.2.6 Representing Others’ Task In addition to another person’s abilities and characteristics, we often have information about another’s task. If information about a co-actor’s task is available, people have a strong tendency to represent aspects of this task. Sebanz et al. (2003) showed that a conflict between two action alternatives arose not only when a single person has to choose between the two alternatives, but also when two co-actors each controlled one of the two action alternatives, suggesting that each of them was representing the action alternative at the other’s disposal (also see Tsai et al. 2011). Atmaca et al. (2008) and Böckler et al. (2012) show that co-actors form representations of others’ tasks that allow them to anticipate when their partner will act. In a picture-naming task, Baus et al. (2014) found that participants showed sensitivity to the lexical frequency of an object to be named by their co-actor, suggesting that task partners mentally performed each other’s tasks. How do social relations affect task co-representation? Some studies have showed that the degree to which people show sensitivity to their partner’s task varies depending on whether the other is perceived as likable or hostile (Hommel et al. 2009) and on whether the task partner is perceived as an in-group or out-group member (He et al. 2011; McClung et al. 2013). These findings suggest that task co-representation depends on the perceived interdependency between the actors.

3.2 Intentional Interpersonal Coordination

131

3.2.7 Motor Anticipation In interpersonal coordination, representing information about others, their task, or the relation of one group to anther is used to anticipate how another person’s actions will unfold in the near future. For example, when trying to catch a basketball, one does not follow the ball’s trajectory but anticipate the likely target location based on the thrower’s action goal, body orientation and arm speed (e.g. Urgesi et al. 2012). Most of what we know about action anticipation comes from tasks in which individuals observe what others are doing. These studies identify common representations and processes involved in performing actions and perceiving other’s actions. In particular, motor anticipation or motor simulation involves forward models in the observer’s motor system that generate anticipations based on the internal motor commands that the observer would use for performing the action himself (Wolpert et al. 2003). Perception and action are thus represented in a common representational format (Prinz 1997) that allows using the same processes underlying individual action planning for anticipating other’s actions. Motor simulation helps an observer in understanding other’s actions (Gallese et al. 2004) and in inferring and interpreting action goals and intentions (Iacoboni et al. 2005). Furthermore, expertise and familiarity with an action modulate motor anticipation such that one’s own motor system is more actively engaged in making anticipations when one has frequently performed the action oneself (see Sect. 2.8 for a case of expert ballet and capoeira dancers, Calvo-Merino et al. 2005; see Sect. 2.9 for a case of basketball, Aglioti et al. 2008). However, we are always unable to see or hear our interaction partners during joint action. Is anticipation of action also possible without online sensory information? Can task representations alone trigger motor anticipations about others’ actions? To examine these questions, Vesper et al. (2013) studied performance in a joint task where co-actors did not receive online visual information about each other’s actions because they were separate by an opaque occlude. But they knew what the other’s exact task was. Dyads coordinated forward jumps of different distances, with the joint goal of synchronizing their landing times. Although no visual or auditory feedback was given before and during the jumping, each of them had to time their own action with respect to the anticipated timing of their own and the partner’s actions. If one person’s task was to jump a short distance and the other’s task was to jump a long distance, they would manage to land at the same time if at least one of them modulated their natural jump performance (e.g., waiting longer before jumping). Participants with the relatively shorter jumps systematically adapted the time of initiating their movement and the height of their jumps depending on the specific relation between their own and the partner’s jump distance. This indicates that even without perceptual information about the interaction partner, the actions can be accurately anticipated and integrated into one’s action planning and performance.

132

3 An Overview of the Study on Interpersonal Coordination

Such anticipation happens not only during an ongoing action performance but also at the planning stage. Motor anticipation often occurs in the complete absence of any movement execution, as when someone only imagines performing a joint action. Using EEG, Kourtis et al. (2013) have found that participants receiving an object from another person exhibit motor activation preceding the giver’s action onset that reflects anticipation of the giver’s action, indicating that motor anticipation about the task partner’s actions is generated during planning a joint action. In addition, task representations can be useful for monitoring the accuracy of the joint performance and the progress towards the joint goal. For example, if a player throwing a basketball knows where and how the partner should catch it, the actual outcome of the interaction can be compared to the anticipation. Using EEG, Loehr et al. (2013) have found that co-actors monitor each other’s actions in playing a piano duet. During the pianists’ playing some tones were experimentally modulated so that the outcome was not as expected based on their prior experience. At an early stage of processing, the pianists’ brain responses to such modulations were detected independently of whether the modulation occurred in their own or in the partner’s playing. However, modulations that affected the overall musical outcome were later processed as being more relevant than modulations that affected only their individual part.

3.2.8 Communication Through Action We often use actions to communicate with others. For example, two dancers interact with each other through a cue such as movement speed, force or direction made by partners’ action. Even a conventional action like handing over a cup, on the other hand, can be used to communicate information. While handling a cup to anther person, and an impression on a person when moving it slowly differs from the impression when moving it roughly (Constanble et al. 2011). From studies on action observation, people are known to be sensitive to information from others’ action kinematics. For example, one can detect whether an observed action is cooperative or competitive (Georgiou et al. 2007; Sartori et al. 2011) and whether an action outcome will be successful or not (Aglioti et al. 2008). In addition, when performing an action together, this sensitivity can be supported by exaggerating relevant information about an action. This is known as ‘signaling’ and might be especially useful when others need additional information for task performance. Vesper and Richardson (2014) asked pairs of participants to move to different targets while only one person knew where the next targets would be. To facilitate coordination, the person having task knowledge moved with overall higher amplitude such that it helped disambiguate the targets. Because these modulations increased the amount of information contained in the action kinematics, a new group of participants more accurately anticipated the correct target from

3.2 Intentional Interpersonal Coordination

133

visual displays of the trajectories. Sacheli et al. (2013) have also reported that a co-actor who knows where to jointly grasp a bottle-shaped object exaggerates his movements comparing with individual performance. The co-actor’s grasp is higher when moving to the top and lower when moving to the bottom. Goebl and Palmer (2009) further asked two pianists to coordinate their playing without receiving auditory information. By the signaling that pianists may lift their fingers higher, they make it easier for the partner to anticipate the timing of their actions and to play in synchrony.

3.2.9 I nteraction of Coordination Mechanisms Using Asymmetrical Joint Action Task The previous subsections introduced different coordination mechanisms and described what representations are involved. What are the links between these different mechanisms? How do they interact? If the partner’s task is not known and no perceptional information about his actions is available, it will be difficult to accurately anticipate his behavior, and the best strategy is thus to make oneself predictable. A useful way to study the interaction of different coordination mechanisms is to look at asymmetrical joint action. The asymmetrical joint action is set by presenting pairs of participants different types or amounts of perceptual information. Asymmetries may afford people to employ different coordination mechanisms that perform each other. In daily life, if someone carrying a heavy box with both hands might just walk straight towards a door, another person adapts his actions cooperatively by opening it at the most suitable time. In some cases, co-actors’ task might explicitly involve performing different actions that perform each other so that only their combined effort will result in successful joint action. In the jumping study described earlier (Vesper et al. 2013), the co-actor with the shorter jump engages in motor simulation to adapt his movement to the partner’s anticipated timing, whereas the one with the longer jump will generally speed up, consistent with a coordination strategy to make oneself. Receiving different amounts of information has an influence on a leader-follower relationship in joint action. To examine the directionality of the interaction on a millisecond timescale in real time, for example, Konvalinka et al. (2010) set up an experiment in which pairs were coupled through their headphones. Konvalinka and colleagues asked pairs of participants to maintain a given beat while synchronizing their tapping movements to an audible signal. They gave different instructions and different amounts of shared auditory information to two participants. They found a leader- follower strategy when the interaction between participants was unidirectional. However, as shown in Sect. 4.3, Masumoto and Inui (2014a) gave an evidence for the emergence of the strategy with bidirectional interaction.

134

3 An Overview of the Study on Interpersonal Coordination

3.2.10 Representation of Self and Other Actions As already stated in the subsection of ‘perception-action matching’, theories that postulate overlap between motor and perceptual processes have increased during the last few decades. ‘Common coding theory’ shows that actions are represented in terms of their contingent sensory consequences, or action-effects (Hommel et al. 2001; Prinz 1997). When individuals perform actions, they acquire bidirectional associations between motor patterns and action-effects. Motor processes thus influence perception and vice versa. The common coding theory implies that perceiving others’ actions should activate motor patterns in the mind of the observer. Rizzolatti and colleagues have confirmed that observing others’ actions leads to activation in many of the same areas of the brain involved in preparing and executing one’s own actions (for a review, Rizzolatti and Sinigaglia 2010). While motor activity during action observation is called ‘motor resonance’, its activity might serve imitation and understanding others’ intention. The close overlap between motor and perceptual processes enables observers to perform ‘mental simulations’ of other agents’ actions, and the simulation thus anticipates upcoming action-effects. According the idea of mental simulation, familiar actions are easier to anticipate than unfamiliar actions. Knoblich and Flash (2001) have reported that observers watching movies of people throwing darts are more accurate in anticipating where the darts will land when viewing a movie of themselves compared with a stranger. This suggests that first-hand action experience plays an important role in the representation and understanding of others’ actions. The ability to map observed actions onto one’s own action repertoire is observed early in life. Infants develop the ability to imitate others during the first 2 years (Jones 2009). As a famous finding, Meltzoff and Moore (1983) have showed that even newborn infants show a rudimentary ability to imitate adults’ facial expressions. Infants further begin to recognize other people’s actions as goal-directed during the first year (Sommerville and Woodward 2005). Infants’ sensitivity to goals in the actions of others is modulated by their own experiences performing similar actions. For example, Sommerville et al. (2005) examined 3-month-old infants’ sensitivity to other people’s action goals measuring their eye gaze while an actor reached for an object. Infants with previous experience reaching for the same object showed greater surprise (longer looking time) when the actor reached for a different- than-expected object compared to infants with no prior experience reaching for the object. Skerry et al. (2013) also reported that infants show increased surprise when observing an actor reaching for an object in an inefficient manner if they have previous experience reaching for the same object. These findings suggest that the experience of performing goal-directed actions anticipates similar goal-directed actions in others. Similarly, action experiences influence how adults perceive other’s action. As a most famous example described in Sect. 2.9, skilled basketball players show greater motor resonance and better at anticipating if another person’s shot will go through the basket compared with novices (Aglioti et al. 2008). We have one question

3.2 Intentional Interpersonal Coordination

135

whether athletes are better than average at anticipating action-effects in general. Diersch et al. (2012) compared older and younger figure skating experts with age- matching novices. Although experts at all ages showed superior anticipation performance for figure skating moves, they were not better than novices at anticipating simple movement exercises. In addition, the ability to anticipate others’ actions was lower in older experts than in younger experts. These findings suggest that the effects of expertise are domain specific. Anticipating others’ actions does not only depend on mental simulation but also on rule-based inference, reasoning about other’s mental states by deduction or rules- of-thumb. As an earlier example, an observer might anticipate that a basketball player’s shot will go into the basket, not as a result of a mental simulation, but simply because the observer believes that the player is highly skilled. The difference between rule-based inference and simulation in social cognition is supported by neurophysiological experiments (Van Overwalle and Baetens 2009). Neurons involved in action simulation are located in the premotor cortex, a region involved in planning movements, and in parts of parietal cortex, in particular the anterior intraparietal sulcus. By contrast, rule-based inferences involve the temporoparietal junction, precuneous, and medial prefrontal cortex. Both systems play important roles in reasoning about others’ actions, and likely work together in a complementary fashion. In an fMRI study on human participants, Buccino et al. (2004) found that observing actions belonging to the motor repertoire of the observer activated the motor system even when they were performed by a different species (dogs or monkeys). By contrast, a movie of a barking dog did not activate any motor system.

3.2.11 Representation of Space in Relation to Self and Other Many actions are directed toward particular objects or people. From an action- oriented perspective, one particular important property of objects or people is their spatial location. To effectively guide the body through space, it is obviously necessary to track the position of nearby objects. However, the spatial coordinates of object with respect to a sense organ (e.g., eyes or ears) differ from the spatial coordinates of objects with respect to a motor effector (e.g., the hands). This presents a problem how the brain represents the spatial location of objects. Graziano et al. (1994) discovered neurons in the ventral premotor cortex of the macaque monkey which specially responded to visual stimuli adjacent to the hand or arm. The neurons’ activity changed when the monkey moved its arm, but not when it moved its eyes. These neurons thus seem to provide a representation of space near the body related to the visual control of reaching movements. The portion of the extra- personal space adjacent to the limits of the body is called ‘peripersonal space’. For convenience, because properties of an environment which support the possibility of action are called affordances (Gibson 1979), Dewey and Knoblich (2016) refer to populations of neurons that show sensitivity to the position of objects in

136

3 An Overview of the Study on Interpersonal Coordination

peripersonal space as ‘affordance maps’. Affordance maps are flexible and can be altered by action experience. In particular, affordance maps are re-shaped by tool use. Witt et al. (2005) asked participants to estimate distances to targets while they either did or did not hold a reaching tool. Estimates of perceived distance were shorter when participants held the tool and intended to reach with it, suggesting that participants’ representation of space was influenced by the action affordances provided by the tool. In addition, Iriki et al. (1996) asked Japanese macaque monkeys to train to use a simple rake to collect a food pellet placed outside hand’s reach. Iriki and his colleagues observed that some of the parietal cortical neurons exhibited both a somatosensory receptive field, located in the monkey’s hand, and an equivalent visual receptive field, centered on the external space immediately surrounding the hand. Iriki and his colleagues also observed that as the monkey moved its hand to a new location in space, whereas the somatosensory receptive field of the cortical neuron remained focused on the same skin region, the visual receptive field migrated to represent the distinct peripersonal space that now surrounded the animal’s hand. Thus, the neuron’s visual receptive field had instantaneously updated the hand’s position. Iriki’s group further found that, after a monkey used the rake to collet food pellets for about 5 min, the visual receptive field of the same bimodal cortical neurons suddenly enlarged to include the peri-personal space surrounding the entire tool, in addition to the space around the monkey’s hand. This dramatic enlargement of the visual field only occurred when the monkey was actively using the rake. This finding suggests that the monkey’s brain was assimilating the rake as an extension of the animal’s arm. On the other hand, human are generally very good at taking the properties of potential co-actors into account when planning actions. For example, Davis et al. (2010) have reported that people account for the combined shoulder width of themselves and another person when estimating whether a door is sufficiently wide for both to walk through simultaneously. Their ability to do so depends on the perspective from which they view the other. To examine the effect of action affordance on participants’ estimation, Isenhower et al. (2010) also asked dyads to work together to move planks from a conveyor belt onto a drop-off ramp. The planks could be moved either individually for short planks or with the partner for long planks. The plank length at which the dyads transitioned from solo to joint lifting was determined by the arm span of the partner with shorter arms, indicating that the capabilities of co-actors are spontaneously taken into account during joint actions. If co-actors influence action affordances, we need to think what effects other people may have on affordance maps and the representation of peripersonal space. To examine this question, Constantini et al. (2011) tested whether observation of tool- use influences perceived distance to active tool-use. They found that both performing and observing tool actions extended the range of peripersonal space. The brain thus represents other agents to tools which offer extended opportunities for interacting with shared environments. Constantini et al. (2011) further showed that both response times and motor-evoked potentials are influenced by the spatial alignment of an object’s handle with respect to a hand (congruent or incongruent), regardless of whether the hand belongs to the participant or a computer avatar. Taken together,

3.2 Intentional Interpersonal Coordination

137

action affordances are represented similarly for self and other. Representing objects in a shared action space may help support coordinated joint actions, in particular when there is some disparity between the perspectives of self and other.

3.2.12 Distinguishing Self and Other The previous subsection emphasized similarities between the representations of executed and observed actions. This overlap may be useful for anticipating other’s actions and representing action affordances in a shared action space. Similarly, it is important that actors know how to distinguish their own actions from other’s actions. Distinguishing the actions of self and others involves both ‘sense of body ownership’ and ‘sense of agency’. While the term ‘sense of body ownership’ refers to a subjective awareness with one’s own body as distinct from other’s bodies, the term ‘sense of agency’ refers to awareness that one is initiating an action or some other event (Gallagher 2000). Belief in what we think belongs to our body can be easily manipulated by experiment. Individuals are induced to feel a sense of ownership over objects external to their body or to have out-of-body experiences. These phenomena are known as ‘body-transfer illusions.’ As a famous example, in the rubber hand illusion, Botvinick and Cohen (1998) asked participants to watch an artificial hand being stroked with a brush while brush strokes are simultaneously applied to their own hand which is hidden from view. The participants reported a spatial fusion between the seen and felt touch, as if they were feeling the touch of the brush in the location where they saw the rubber hand touched. They often described this illusion by saying that it felt as if the rubber hand had become their hand. In line with ‘body- transfer illusion’, Ehrsson (2007) induced ‘out-transfer illusion’ by showing participants images from cameras placed behind their head, while using plastic rods to simultaneously touch the participant’s actual chest and the chest of the illusory body just below the camera’s view. This gave participants the sensation of sitting behind their physical bodies. Body-transfer illusions are strongly dependent on the compatibility of multisensory information. Such sense of body ownership may allow individuals to recognize their own body as distinct from other bodies. However, because many actions have consequences that are not directly related to body, the sense of body ownership alone is not sufficient to distinguish between actions of self and other. On the other hand, a clearest example of the sense of agency is a subjective awareness elicited by a voluntary movement. The sense of agency consists of a combination of motor and perceptual information. The perceptual information contains both sensory feedback produced by motor outcome and efference copy. In recent decades, the sense of agency is thought to originate with forward models (Blakemore et al. 2002). Because the models predict and monitor action-effects produced by voluntary movements, they contained a few comparators. The model is thus referred to as the comparator model or central monitoring hypothesis. As a

138

3 An Overview of the Study on Interpersonal Coordination

Fig. 3.10 The comparator model of agency (Blakemore et al. 2002, redraw from Dewy and Knoblich (2016) with permission from Cambridge University Press). According to the central monitoring hypothesis, motor commands produce movements while a corollary discharge or efferent copy of the motor command is processed in parallel to anticipate the future state of the system. These anticipations include upcoming body position and the sensory consequences of the movement. The sense of agency depends on the degree of match between the anticipatory and actual states (in the bottom right of this figure). Additional comparators may exist to monitor consistency between intended and estimated states (top left), and intended and anticipatory states (top right)

theory of agency, the comparator model (Fig. 3.10) proposes that, while motor commands produce movements, a corollary discharge or efference copy of the motor command is processed in parallel to predict the future state of the system. These predictions include upcoming body position and the sensory consequences of the movement. The sense of agency thus depends on the degree of match between the predicted and actual states in the lower right of the figure. Additional comparators may exist to monitor consistency between intended and estimated states (upper left), and intended and predicted states (upper right). In line with the forward models, many studies have confirmed that judgments of self-agency are influenced by the predictability of action-effects. For example, Farrer et al. (2008) reported that introducing spatial and temporal discrepancies between executed movements and the visual feedback of those movements can cause participants to misidentify their own movements as belonging to another person. By contrast, Sato and Yasuda (2005) reported that coincidental agreement between self-generated actions and sensory feedback can produce a misleading sense of agency for action-effects produced by others.

3.3 Development of Interpersonal Coordination Infants early gain experience with dyadic interaction, and they are enveloped in social exchange with their parents or nursery school teachers (Reddy 2008; Stern 2002; Trevarthen 1979). In the second half of their first year, their interaction with

3.3 Development of Interpersonal Coordination

139

adults start to be organized around objects of common interest, which provides opportunities for experiencing the relation between their own and the partner’s attention and action (Barresi and Moore 1996). Tomasello et al. (2005) and Carpenter (2009) suggest that children have a special motivation to share attention and to engage in cooperative activities. The excellent review of Brownell (2011) describes that children gradually become able to contribute actively and independently to joint actions, relying on adults to structure the interaction during their first 2 years. Whereas infants engage in joint action during social play with adults from early in life, regular occurrences of spontaneously generated, mutually sustained joint activity with peers does not emerge until the end of second year of life, and even then is quite limited and rudimentary (Brownell and Brown 1992). Later in the preschool years, children adapt and coordinate multiple roles and actions with each other in joint play and problem solving, routinely negotiate and share goals, and support one another’s activity with respect to their shared goals (Ashley and Tomasello 1998; Hamann et al. 2012). For example, 1-year-olds recognize when their partner fails to take a turn during joint toy play with adults, and they attempt to re-engage the partner in the activity (Ross and Lollis 1987), consistent with sharing a cooperative or joint goal with the partner. During the interaction, however, their behavior is only minimally coordinated with the partner’s action, and they often appear to be pursuing their own individual goals rather than joint or shared goals. Further, with a same-age peer partner infants never attempt to repair such failed turn-taking sequences (Brownell and Carriger 1991). By contrast, 2-year-olds coordinate behavior with a partner to achieve a goal, whether the partner is a peer or an adult (Brownell and Carriger 1990; Warneken et al. 2006). However, they make little effort to help a partner who has differently fulfilling his or her part in the cooperative activity. They seem unaware of any sort of joint commitment to a shared goal (Hamann et al. 2012). Thus, without the structure and scaffolding provided by the expert adult partner, 1-year-olds are unable to generate and sustain joint action with each other in order to achieve an external goal. By contrast, 2-year-olds can do so readily, even with unfamiliar agents and on novel, unfamiliar tasks.

3.3.1 Adult-Infant Interaction In the first weeks of life, joint action begins during parent-infant interaction (Brownell 2011). Parents draw their young infants into face-to-face interactive exchange, structuring and scaffolding social participation with an especially socially responsive infant in a dyadic dance that includes motor action, gazing, cooing, touching, and smiling along with other facial expressions. Bigelow and Walden (2009) have reported that infants become progressively tuned to the timing and structure of dyadic exchange. Over several months, infants gradually come to anticipate the partner’s affect and behavior in face-to-face dyadic activity. At the same time they lean how to coordinate and alternate their own attention, affect,

140

3 An Overview of the Study on Interpersonal Coordination

vocalization, and motor action with responsive adult partners across an increasing variety of interactive contexts. During the first months of life, Bornstein and Tamis-LeMonda (2001) have pointed out that participating in dyadic interaction with adults provides the young infant with the basic skills and anticipations for joint action in social exchange, as well as the positive arousal that serves to sustain interest, engagement, and emotional commitment to interaction. Infants come to understand and interpret the signaling value of adults’ affect and behavior in routine social interaction, and acquire enormous amounts of practice in putting their growing understanding and skill to work in ways that garner positive emotional experience. However, these very early examples of joint action do not require the infant to represent, infer, or consciously share the adult’s intention, goal, or desires. The young infants must simply be interested in, engaged by, and responsive to the adult’s efforts to establish the joint action. During the second half of the first year of life, Moore and Dunham (1995) have pointed out that adult-infant dyadic interactions expand to include objects, events, and individuals outside of the dyad. For example, parents call infants’ attention to a toy by shaking it or banging it, and the infants follow the adult’s attention to focus on and play with the toy while they maintain engagement with the adult. Reversely, the adults follow into the infant’s own attention to or engagement with an object so that together the two share the same activity, generating joint action. These triadic interactions are strongly tied to the immediate social and physical context, and occur in highly routinized action frames such as social games and play routines in which adults again structure the goals, content, and timing of the interaction and often direct the child how to behave in accordance. Infants thus come to share attention, interest, affect, and action with the partner in reference to something that they experience together, simultaneously. Adults implicitly or explicitly support this new form of joint activity when it emerges in the first year, and they continue to do so into the second year in challenging situations (Deák et al. 2000). Furthermore, infants begin themselves to initiate joint action with adults and to respond in unique ways when adults violate their anticipations for participation in the joint action. For example, Liszkowski et al. (2006) have shown that, if a parent becomes distracted during peek-a-boo and fails to take her turn, 1-year-olds may try to re-start the game by vocalizing to the adult or by re-enacting a well-rehearsed part of the game such as placing the cloth over their own face and waiting. One-year-old infants also begin to point to interesting sighs and events to share their interest and affect and they anticipate adults to respond appropriately by looking. Thus, as joint action with adults develops in complexity over the first year, infants’ contributions to and control over the structure of joint action also become more complex, along with advances in anticipations for a partner’s behavior and new skills for initiating and responding in more varied to others’ behavior. In a series of pioneering studies on adult-infant interaction, Bakeman and Adamson (1984, 1986) followed infants from 6 to 18 months of age, observing them playing with their mothers and toys. Mothers initiated joint engagement by following onto the focus of infants’ attention or by demonstrating an interesting

3.3 Development of Interpersonal Coordination

141

object to capture infants’ attention. Mothers then complemented their infants’ behavior with their own and helped their infants remain jointly engaged. Infants were jointly engaged with mothers in such exchange up to 35% of the time by 15 months of age. Roughly half of that time involved a regular routine or action format so that most joint activity occurred in the context of specific, well-rehearsed scripts that specify the role, behaviors, and timing for a joint play such as peek-a- boo, tickle, rhythmic games, or chase. Thus, regular and familiar routines with mothers supported infants’ growing participation in joint action. It was not until 18 months of age that every infant participated in at least one episode of what Bakeman and Adamson called ‘coordinated joint engagement’ in which the infant was an active participant in the interaction rather than being passively led by the mother’s actions. This more complex and active form of joint engagement constituted only 11% of total play time at 15 months and only 27% at 18 months, but this was more than 2–4% among infants of 12 months or younger. Although 12-month- olds are capable of engaging in joint attention such as following others’ gaze and pointing, active coordination of joint activity around toys and other objects is not a regular occurrence until much later in the second year. Hay (1979) also reported similar findings in a study of mothers’ and infants’ turn-taking games between 12 and 24 months of age, such as give-and-take play or rolling a ball back. The majority of 18-month-old infants actively took part in at least one coordinated game with their mothers. By 18 months many infants participated at least in such exchanges when they involved coordinating well-rehearsed actions within the envelope of familiar routines, and by 24 months this form of joint activity was a regular feature of mother-infant interactions. Rutter and Durkin (1987) further found that vocal sequences between mothers and infants were fully controlled by mothers before 24 months of age. Infants vocalized at will and mothers coordinated the turn-taking by altering their own vocal activity. After 24 months, however, infants increasingly coordinated their own vocalizations with their mothers’ by waiting their turn to vocalize and then doing so at the appropriate pause in the mother’s vocal activity. The transition was somewhat earlier for gaze coordination, with rapid increase between 18 and 24 months in the sequencing of gaze. In particular, infants began looking up at their mothers at the end of their own turns. Such “terminal looks” signal the partner that one’s turn is finished and offer the floor for next turn by the partner. Older infants further looked at their mothers much more often at the end of the mother’s turn, which permitted them to time and coordinate their behavior with the mother’s turn. Thus, by 24–30 months of age, children had clearly begun to take an active role in coordinating and sequencing various types of joint activity with mothers. Dix et al. (2009) have recently found that 27-month-olds are much more able to initiate and sustain positive involvement with their parents than were 14-month olds, by approaching, showing, pointing, vocalizing, or touching the parent, and by timing these behaviors to correspond to a parent’s availability and responsiveness so that they resulted in mutual social exchange. In a longitudinal study on adult-infant interaction, Aureli and Presaghi (2010) have further found that ‘unidirectional’ social engagement predominates early in the second year during mother-infant toy

142

3 An Overview of the Study on Interpersonal Coordination

play, when mothers engage infants’ interest and attention while the infant simply watches without otherwise becoming actively involved with the object or theme. By the end of the second year, ‘bidirectional’ engagement has become predominant, in which child and mother are jointly engaged, actively sharing attention, affect, and actions, and accommodating to and influencing one another. The decrease in unidirectional engagement and the corresponding increase in bidirectional engagement occur gradually over the second year with the latter surpassing the former only in the second half of the period. Thus, early in the second year children’s contributions to joint action with adult around toys and other objects are not autonomous although 1-year-olds need the adult to direct and structure the joint part of the activity. The joint action of infants is relatively inflexible early in the second year, largely restricted to well-learned and practiced routines and games. The development of adult-infant interaction thus is an extended process that occurs under the tutelage of adults, with the first evidence of autonomy and mastery occurring toward the end of the second year. In contrast to familiar routines, Warneken et al. (2006) asked 1-and 2-year olds to play with adults using novel cooperative game. One task required partners to grip a small rubber trampoline on opposite sides and shake it together to make a rubber ball bounce in the middle. Another required one person to push a small cylinder up through an opening to release a ball to the partner who has to retrieve it at the appropriate moment. Thus, the partners had to act together by positioning themselves appropriately in relation to one another and timing their own behavior in relation to the other’s actions. Eighteen-month-olds’ behavior with the adult partner was rated as predominantly uncoordinated (vs. coordinated or very coordinated) and the infants exhibited low cooperative engagement (vs. medium or high). On those tasks requiring infants to anticipate the partner’s actions and to adjust their behavior accordingly, 18-month-olds’ performance did not differ from chance. By age two, infants operated at medium levels of cooperative engagement and were above chance in anticipating and coordinating their behavior with the adult. This age- related pattern roughly parallels those found in the research reviewed above, although the infant’s contribution to action coordination in unfamiliar situations with unfamiliar adults appears to be lower at age two than it is when engaged in familiar games and routines with mothers. Thus, when toys, activities, or partner are novel and unfamiliar, infants’ ability to coordinate their actions with an adult partner develops somewhat later, appearing in more rudimentary form at the end of the second year of life. Taken together, infants learn about cooperation by participating in joint action structured by skilled and knowledgeable interactive partners before they can represent, understand, or generate it themselves. Cooperative joint action develops with adults in which the adult initially takes response for and actively structures the joint activity and the infant progressively comes to master the structure, timing, and communications involved in the joint action with the support and guidance of the adult. Early on, infants are pulled into joint action by adults’ skillful efforts. Over the first several months of life, infants gradually take more control of and contribute more actively to joint dyadic engagement in face-to-face interactions. Later in the first

3.3 Development of Interpersonal Coordination

143

year, joint actions become more complex as adults introduce objects into their interactions with infants and help them to coordinate their attention and actions with those of the adult around such objects. Over the second year, infants gradually take more control of and contribute more activity to this new form of joint activity until they can initiate and manage their contributions themselves with minimal help and support from their adult partner. It takes approximately 2 years for infants to become autonomous contributions to sustained, goal-directed joint activity active, collaborative partners.

3.3.2 Infants’ Joint Action with Other Infant When and how does joint action arise between infants who have similar, primitive social understanding and comparable social skills? In some of the studies of mother- infant interaction reviewed above, infants were also observed with an agemate, and were found to engage in joint action several months later with peers than with mothers. Bakeman and Adamson (1984) reported that, at 18 months when every infant participated in at least one bout of coordinated joint engagement with mothers and a quarter of their play consisted of this more advanced form of joint action, only 7% of their play with a peer included any coordinated joint engagement. The joint action routines learned and practiced with each infant’s own mother were not sufficiently general to be used to generate joint action with peers. Among familiar children in childcare groups, Holmberg (1980) found that coordinated actions between young peers during play were infrequent in the second year of life, but began to increase between 24 and 30 months of age. In laboratory-based longitudinal study of joint action, Eckerman et al. (1989) reported that coordinated action grew threefold between 16 and 28 months of age, with a steep increase between 24 and 28 months. While mutual imitation or chase/follow games constituted the bulk of children’s joint activity in the second year, they were unable to sustain joint action using more varied, non-routine, complementary actions until 28 months of age. Eckerman and Peterman (2001) have thus reviewed that, whereas 1-year-old infants are quite limited in their ability to coordinate actions with a common theme or goal, 2-year-olds are capable of a wide variety of joint action with peers. Furthermore, Brownell and Carriger (1990, 1991) asked 12- to 30-month old infants to act together to retrieve small shareable toys from inside a clear box or cylinder. One child had to push a handle to make the toys available at the opposite end of the box while the partner had to move into place to retrieve them. The task was constructed so that a single child could not perform both roles, indicating that infants had to position themselves in space opposite one another and perform different, separate actions in temporal sequence. Brownell and Carriger (1990, 1991) found that neither 12- nor 18-month-olds could coordinate their actions with a peer to achieve this simple outcome. By contrast, 24- and 30-month-olds were able to coordinate their actions deliberately and to coordinate successfully several times in

144

3 An Overview of the Study on Interpersonal Coordination

succession. From analyses of each child’s individual behavior, 12-and 18-month olds were generally failing to consider the partner’s position or actions with respect to their own. They rarely monitored their partner visually, often did not act when the partner was nearby and available, did not anticipate the partner’s position or behavior or alter their own behavior. By contrast, 24- and 30-month olds were able to anticipate their partner’s actions and locations and to adjust their own behavior in anticipation. Two-year olds were thus able to act jointly with each action in a novel situation, without the support of familiar partners, goals, or routines. To do so depends on the infants attending to and anticipating one another’s actions, considering the partner’s actions in relation to one’s own, and representing the state of task and task outcomes as a function of each individual’s behavior. More recent studies on joint action using similar kinds of tasks (Hunnius et al. 2009; Warneken et al. 2006) are consistent with the results of Brownell and Carriger (1990, 1991).

3.3.3 D evelopmental Mechanisms for Interpersonal Coordination What cognitive processes must be in place for infants to be able to engage in joint action autonomously? Vesper and Sebanz (2016) raise three factors for the cognitive processes to engage in joint action. The first factor is the ability to understand how others’ actions relate to one’s own goals and actions (for a review, Brownell 2011). Infants perhaps start out with a minimal understanding of others’ contributions to joint outcomes. If infants specify others’ actions and the relation between ones’ own and others’ parts, and form task representation, they are allowed for more flexible and more coordinated interaction. The second factor is the ability to inhibit one’s actions, which is important for turn-taking. Meyer et al. (2015) found that toddlers who showed better inhibitory control in an individual task proved better at taking turns during joint action. The third factor is being able to anticipate the timing of others’ actions. The better toddlers were at anticipating the timing of others’ actions in an independent task, the less variable their actions were with respect to a partner’s in a joint coordination task (Meyer et al. 2015). As for the first factor, Barresi and Moore (1996) propose that an “intentional schema” develops in the second year of life in which infants consciously reflect on and integrate subjectively derived information and knowledge about their own object-directed intentions and actions (first person perspective) with objectively derived information about others’ actions (third person perspective). In other words, in the first year of life, self and other are represented separately in terms of first and third person information respectively. Over the second year, however, they are integrated and represented together within a single representational format. Tomasello et al. (2005) further proposes the construct of a “dialogic cognitive representation” to describe the common representational format that underlies and permits cooperative activities. Both partners represent the other’s goals and actions plans as part of

3.3 Development of Interpersonal Coordination

145

their own goals and action plans. Both of these models entail conscious self-other representation at the levels of desires, and intentions as well as actions. Substantial evidence points to a major transformation in self-other representation during the latter half of the second year. Moore (2007) points out that, between 18 and 24 months of age, infants become consciously, reflectively aware of themselves for first time. The prototypical indicator is mirror self-recognition, in which the infant recognizes himself as an indentifiable individual and the mirror image as an objective representation of self separate from the actual physical self (Amsterdam 1972). Prior to this, the infant either ignores the mirror image or treats it as if it were another child. With the advent of mirror self-recognition, the infant demonstrates that he can consider himself from a third person perspective. In addition, to represent oneself as an object of others’ attention, perception, and action also entails a simultaneous representation of the other’s attention and action. Brownell et al. (2006) suggest that infants’ joint action becomes progressively more controlled over the second year by this conscious and self-reflective system. The emergence of this system permits infants to infer and to represent explicitly the causal, temporal, and spatial relations between the respective actions of self and a partner, to do so in advance of action and in relations to an external goal, and thus to act to coordinate behavior around that goal. Infants consequently become more able to take the partner’s goal-related activity into account in concert with their own and to adjust their own behavior by monitoring, timing, and sequencing their behavior together with adults and peers in relation to a common goal. Brownell (2011) reviews that advances in infants’ ability to coordinate their behavior with one another are associated with multiple measures of developing self- other representations. For example, Brownell and Carriger (1990) found that 1- and 2-year-olds’ symbolic representation of self and other in pretend play was related to amount of coordinated behavior they produced with a peer on the structured cooperation tasks described above. In particular, dyads in which at least one of the infants produced advanced self-other representation in pretense were able to cooperate successfully to achieve the common goal. Brownell et al. (2006) similarly reported that infants who were more advanced in understanding and following adults’ referential points and gazes more often monitored their peer’s behavior during joint action, and more frequently and effectively coordinated their own behavior with the peer. In addition, infants who better produced and comprehended language about their own and others’ feeling and actions, and who could refer to themselves and others using personal pronouns also monitored their peer’s behavior more often and produced more joint action with the peer. Thus, as infants become progressively more able to represent and consciously reflect on self in relation to other, and vice versa, they also become more able to coordinate their behavior actively and autonomously with a rage of others toward common goals. However, the “common” goal may or may not be shared in the strict sense defined by philosophers of adult cooperative action and communication (Clark 2006; Gilbert 2009; Seale 1990; Tuomela 2005). This is reflected in the fact that even though 2-year-old infants’ joint action with peers is autonomous and deliberate, in contrast to 1-year-olds, it is far from adult-like. For example, Brownell (2011)

146

3 An Overview of the Study on Interpersonal Coordination

points out that 2-year-old peer dyads produce uncoordinated behavior at rates similar 1-year-olds. Their actions often seemed to thwart the partner’s efforts to achieve the goal. Rather than subordinating their individual actions to a joint intention to act together to achieve a shared goal, 2-year-olds may more simply be responding to the behavioral topology of the task in order to achieve their own goal. They can do so in a more sophisticated fashion than by adjusting their behavior to the immediate task demands and opportunities. In line with the comment of Brownell (2011), Hamann et al. (2012) showed that, although 2-year-olds could act jointly to retrieve toys from a box, once one infant had obtained a toy, that infant cease acting together with the partner, precluding the partner from getting a toy. By contrast, 3-year-olds persisted in the joint action until both partners obtained their toys, clearly sharing the goal. Grafenhain et al. (2009) found that 2-year-olds attempted re-engage a partner who played next to the infant if the partner stopped acting. To the contrary, 3-year-olds recognized the difference between an adult acting in parallel next to the infant and an adult acting jointly with the infant. They only attempted to re-establish joint action with the latter when play was interrupted. As for the second and third factors, serial information processing with an interpersonal representation involves the ability to inhibit one’s actions and anticipation of others’ actions. Two-year-olds presumably understand how others’ behavior in such joint action context is relevant to their own ends, but vice versa. They can consciously represent their own and another’s behavior in relation to one another, but only from the perspective of either their own goal or the other’s, not yet simultaneously from both perspectives. Thus, in these joint actions, they may still be pursuing the goal individually, using the partner as a means to their own ends in a complex spatiotemporal, causal sequence of actions which does not necessarily include a high-order representation of the joint task and shared goal that might govern their behavior.

References Aglioti SM, Cesari P, Romani M, Urgesi C (2008) Action anticipation and motor resonance in elite basketball players. Nat Neurosci 11:1109–1116 Amazeen PG, Schmidt RC, Turvey MT (1995) Frequency detuning of the phase entrainment dynamics of visually coupled rhythmic movements. Biol Cybern 72:511–518 Amsterdam B (1972) Mirror self-image reactions before age two. Dev Psychobiol 5:297–305 Ashley J, Tomasello M (1998) Cooperative problem-solving and teaching in preschoolers. Soc Dev 7:143–163 Atmaca S, Sebanz N, Prinz W, Knoblich G (2008) Action co-representation: the joint SNARC effect. Soc Neurosci 3:410–420 Atmaca S, Sebanz N, Knoblich G (2011) The joint Flanker effect. Exp Brain Res 211:371–385 Aureli T, Presaghi F (2010) Developmental trajectories for mother-infant coregulation in the second year of life. Infancy 15:557–585 Bakeman R, Adamson LB (1984) Coordinating attention to people and objects in mother-infant and peer- infant interaction. Child Dev 55:1278–1289

References

147

Bakeman R, Adamson LB (1986) Infants’ conventionalized acts: gestures and words with mothers and peers. Infant Behav Dev 9:215–230 Barresi J, Moore (1996) Intentional relations and social understanding. Behav Brain Sci 19:107–122 Baus C, Sebanz N, de la Fuente V, Brauzi FM, Martin C, Costa A (2014) On predicting others’ words: electrophysiological evidence of prediction in speech production. Cognition 133:395–407 Becchio C, Satrtori L, Bulgheroni M, Castiello U (2008) Both your intention and mine are reflected in the kinematics of my reach-to-grasp movement. Cognition 106:894–912 Bigelow A, Walden L (2009) Infants’ response to maternal mirroring in the still face and replay tasks. Infancy 14:526–549 Blakemore S, Wolpert D, Frith C (2002) Abnormalities in the awareness of action. Trends Cogn Sci 6:237–242 Böckler A, Knoblich G, Sebanz N (2012) Effects of a cofactor’s focus of attention on task performance. J Exp Psychol Hum Percept Perform 38:1404–1415 Bornstein M, Tamis-LeMonda K (2001) Mother-infant interaction. In: Bremner G, Fogel A (eds) Infant development. Blackwell, Oxford, pp 269–295 Botvinick M, Cohen J (1998) Rubber hands ‘feel’ touch that eyes see. Nature 391:756 Brass M, Bekkering H, Prinz W (2001) Movement observation affects movement execution in a simple response task. Acta Psychol 106:3–22 Braun DA, Ortega PA, Wolpert DM (2009) Nash equilibria in muti-agent motor interactions. PLoS Comput Biol 5:e1000468 Brownell CA (2011) Early developments in joint action. Rev Philos Psychol 2:193–211 Brownell C, Brown E (1992) Peers and play in infants and toddlers. In: Van Hasselt V, Hersen M (eds) Handbook of social development: a lifespan perspective. Plenum Press, New York, pp 183–200 Brownell C, Carriger M (1990) Changes in cooperation and self-other differentiation during the second year. Child Dev 61:1164–1174 Brownell C, Carriger M (1991) Collaborations among toddler peers: individual contributions to social contexts. In: Resnick L, Levine J (eds) Perspectives on socially shared cognition. Am Psychol Assoc, Washington, pp 365–383 Brownell C, Ramani G, Zerwas S (2006) Becoming a social partner with peers: cooperation and social understanding in one- and two-year-olds. Child Dev 77:804–821 Buccino G, Lui F, Canessa N, Patteri I, Lagravinese G, Benuzzi F, Porro CA, Rizzolatti G (2004) Neural circuits involved in the recognition of action performed by nonconspecifics: an fMRI study. J Cogn Neurosci 16:114–126 Butterfill S (2018) Coordinating joint action. In: Jankovic M, Ludwiig K (eds) The Routledge handbook of collective intentionality. Routledge, London, pp 68–82 Byblow WD, Carson RG, Goodman D (1994) Expressions of asymmetries and anchoring in bimanual coordination. Hum Mov Sci 13:3–28 Carpenter M (2009) Just how joint is joint action in infancy? Top Cogn Sci 1:380–392 Carvo-Merino B, Glaser D, Grezes J, Passingham R, Haggard P (2005) Action observation and acquired motor skills: an fMRI study with expert dancers. Cereb Cortex 15:1243–1249 Chui K (2005) Temporal patterning of speech and iconic gestures in conversational discourse. J Pragmat 37:871–887 Clark HH (1996) Using language. Cambridge University Press, Cambridge Clark H (2006) Social actions, social commitments. In: Enfield N, Levinson S (eds) Roots of human sociality: culture, cognition & interaction. Berg. Publishers, Oxford Collins JJ (1999) Fishing for function in noise. Nature 402:241–242 Constanble MD, Kritikos A, Bayliss AP (2011) Grasping the concept of personal property. Cognition 119:430–437 Constantini M, Ambrosini E, Sinigaglia C, Gallese V (2011) Tool-use observation makes far objects ready-to-hand. Neuropsychologia 49:2658–2663

148

3 An Overview of the Study on Interpersonal Coordination

Cordon WS, Ogston WD (1966) Sound film analysis of normal and pathological behavior patterns. J Nerv Ment Dis 143:338–347 Cross ES, Hamilton AF, Grafton ST (2006) Building a motor simulation de novo: observation of dance by dancers. NeuroImage 31:1257–1267 Davis T, Riley M, Shockley K, Cummins-Sebree S (2010) Perceiving affordances for joint actions. Perception 39:1624–1644 de Bruijin ERA, de Lange FP, von Cramon DY, Ullsperger M (2009) When errors are rewarding. J Neurosci 29:12183–12186 Deák GO, Flom RA, Pick AD (2000) Effects of gesture and target on 12- and 18-month-olds’ joint visual attention to objects in front of or behind them. Dev Psychol 36:511–523 Decety J, Jackson PL, Sommerville JA, Chaminade T, Meltzoff AN (2004) The neural bases of cooperation and competition: an fMRI investing. NeuroImage 23:744–741 Dewey JA, Knoblich G (2016) Representation of self versus others’ actions. In: Obhi SS, Cross ES (eds) Shared representations: sensorimotor foundations of social life. Cambridge University Press, Cambridge, pp 351–373 Diersch N, Cross E, Stadler W, Schutz-Bosbach S, Rieger M (2012) Representing others’ actions: the role of expertise in the aging mind. Psychol Res 76:525–541 Dix T, Cheng N, Day W (2009) Connecting with parents: mothers’ depressive symptoms and responsive behaviors in the regulation of social contact by one- and young two-year-olds. Soc Dev 18:24–50 Doerrfeld A, Sebanz N, Shiffrar M (2012) Expecting to lift a box together makes the load look lighter. Psychol Res 76:467–475 Eckerman CO, Peterman K (2001) Peers and infant social/communicative development. In: Fogel A, Bremner G (eds) Blackwell handbook of infant development. Blackwell, Oxford, pp 326–350 Eckerman CO, Davis CC, Didow SM (1989) Toddlers’ emerging ways of achieving social coordination with a peer. Child Dev 60:440–453 Ehrsson HH (2007) The experimental induction of out-of-body experiences. Science 317(5841):1048 Eskenazi T, Grosjean M, Humphreys G, Knoblich G (2009) The role of motor simulation in action perception: a neuropsychological case study. Psychol Res 73:477–485 Fadiga L, Craighero L, Olivier E (2005) Human motor cortex excitability during the perception of others’ action. Curr Opin Neurol 15:213–218 Farrer C, Bouchereau M, Jeannerod M, Franck N (2008) Effect of distorted visual feedback on the sense of agency. Behav Neurol 19:53–57 Feldman R, Eidelman AI (2004) Parent-infant synchrony and the social-emotional development of triplets. Dev Pschol 40:1133–1147 Fitts PM (1954) The information capacity of the human motor system in control the amplitude of movement. J Exp Psychol 47:381–391 Flach R, Knoblich G, Prinz W (2004) The two-thirds power law in motion perception: when do motor anticipations come into play? Vis Cogn 11:461–481 Ford P, Hodges N, Huys WAM (2006) The role of external action-effects in the execution of a soccer kick: a comparison across skilled-level. Mot Control 10:386–404 Ford P, Hodges N, Huys WAM (2009) Evidence for end-point trajectory planning during a kicking action. Mot Control 13:1–24 Foth E, Graham D (1983) Influence of loading parallel to the body axis on the walking coordination of an insect. Biol Cybern 48:1133–1147 Franz EA, Zelaznik HN, Swinnen S, Walter C (2001) Spatial conceptual influences on the coordination of bimanual actions: when a dual task becomes a single task. J Mot Behav 33:103–112 Furuyama N (2002) Prolegomena of a theory of between-person coordination of speech and gesture. Int J hum-Comput St 57:347–374 Gallagher S (2000) Philosophical conceptions of the self: implications for cognitive science. Trends Cogn Sci 4:14–21

References

149

Gallese V, Fadiga L, Fogassi L, Rizzolatti G (1996) Action recognition in the premotor cortex. Brain 119:593–609 Gallese V, Keysers C, Rizzolatti G (2004) A unifying view of the basis of social cognition. Trends Cogn Sci 8:396–403 Georgiou I, Becchio C, Glover S, Castiello U (2007) Different action patterns for cooperative and competitive behaviour. Cognition 102:415–433 Gibson JJ (1979) The ecological approach to visual perception. Houghton-Mifflin, Boston Gilbert M (2009) Shared intention and personal intentions. Philos Stud 144:167–187 Goebl W, Palmer C (2009) synchronization of timing and motion among performing musicians. Music Percept 26:427–438 Goldfield EC, Schmidt RC, Fitzpatrick P (1999) Coordination dynamics of abdomen and chest during infant breathing: a comparison of full-term and preterm infants at 38 weeks postconceptional age. Ecol Psychol 11:209–232 Grafenhain M, Behne T, Carpenter M, Tomasello M (2009) Young children’s understanding of joint commitments. Dev Psychol 45:1430–1443 Graziano M, Yap G, Gross C (1994) Coding of visual space by premotor neurons. Science 266(5187):1054–1057 Grosjean M, Shiffrar M, Knoblich G (2007) Fitts’ law holds for action perception. Psychol Sci 18:95–99 Haken H, Kelso JAS, Bunz H (1985) A theoretical model of phase transitions in human hand movements. Biol Cybern 51:347–356 Hamann K, Warneken F, Tomasello M (2012) Children’s developing commitments to joint goals. Child Dev 83:137–145 Hanson FE (1978) Comparative studies of firefly pacemakers. Fed Proc 37:2158–2164 Hay DF (1979) Cooperative interactions and sharing between very young children and their parents. Dev Psychol 15:647–653 He X, Lever AG, Humphreys GW (2011) Interpersonal memory-based guidance of attention is reduced for ingroup members. Exp Brain Res 211:429–438 Henriques DYP, Crawford JD (2002) Role of eye, head, and shoulder geometry in the planning of accurate arm movements. J Neurophysiol 87:1677–1685 Holmberg M (1980) The development of social interchange patterns from 12 to 42 months. Child Dev 51:448–456 Hommel B, Musseler J, Aschersleben G, Prinz W (2001) The theory of event coding (TEC): a framework for perception and action. Behhav Brain Sci 24:849–937 Hommel B, Colzato LS, van den Wildenberg WPM (2009) How social are task representations? Psychol Sci 20:794–798 Hunnius S, Bekkering H, Cillessen A (2009) The association between intention understanding and peer cooperation in toddlers. Eur J Dev Sci 3:278–298 Iacoboni M, Molnar-Szakacs I, Gallese V, Buccino G, Mazziotta JC, Rizzolatti G (2005) Grasping the intentions of others with one’s own mirror neuron system. PLoS Biol 3(3):e79 Iani C, Anelli F, Nicolletti R, Arcuri L, Rubichi S (2011) The role of group membership on the modulation of joint action. Exp Brain Res 211:439–445 Iriki A, Tanaka M, Iwamura Y (1996) Coding of modified body schema during tool use by macaque postcentral neurons. Neuroreport 7:2325–2330 Isabella RA, Belsky J (1991) Interactinal synchrony and the origins of infant-mother attachment: a replication study. Child Dev 62:373–384 Isenhower RW, Richardson MJ, Carello C, Baron RM, Marsh KL (2010) Affording cooperation: embodied constrains, dynamics, and action-scaled invariance in joint lifting. Psychon Bull Rev 17:342–347 Jones SS (2009) The development of imitation in infancy. Philos Trans R Soc Lond Ser B Biol Sci 364(1528):2325–2335 Julien D, Brault M, Chartrand E, Begin J (2000) Immediacy behaviours and synchrony in satisfied and dissatisfied couples. Can J Behav Sci 32:84–90

150

3 An Overview of the Study on Interpersonal Coordination

Keller PE, Knoblich G, Repp BH (2007) Pianists duet better when they play with themselves: on the possible role of action simulation in synchronization. Conscious Cogn 16:102–111 Kelso JAS, Holt KG, Rubin P, Kugler PN (1981) Patterns of human interlimb coordination emerge from the properties of nonlinear limit cycle processes: theory and data. J Mot Behav 13:226–261 Kelso JAS, Scholz JP, Schöner G (1986) Nonequilibrium phase transitions in coordinated biological motion: critical fluctuations. Phys Lett 118:279–284 Kilner JK, Neal A, Weiskopf N, Friston KJ, Frith C (2009) Evidence of mirror neurons in human inferior frontal gyrus. J Neurosci 29:10153–10159 Knoblich G (2008) Bodily and motor contributeons to action perception. In: Klatzky RL, MacWhinney B, Behrmann M (eds) Embodiment, ego-space, and action. Psychology Press, Pittsburgh, pp 45–78 Knoblich G, Flach R (2001) Predicting the effect of actions: interactions of perception ans action. Psychol Sci 12:467–472 Knoblich G, Jordan JS (2003) Action coordination in groups and individuals: learning anticipatory control. J Exp Psychol Learn Mem Cogn 29:1006–1016 Knoblich G, Butterfill S, Sebanz N (2011) Psychological research on joint action: theory and data. In: Burlington RB (ed) The psychology of learning and motivation, vol 54. Academic, Burlington, pp 59–101 Konvalinka I, Vuust P, Roepstorff A, Frith CD (2010) Follow you, follow me: continuous mutual prediction and adaptation in joint tapping. Q J Exp Psychol 63:2220–2230 Kourtis D, Sebanz N, Knoblich G (2013) Predictive representation of other people’s actions in joint action planning: an EEG study. Soc Neurosci 8:31–42 Kugler PN, Kelso JAS, Turvey MT (1982) On the concept of coordination of naturally developing systems. In: Kelso JAS, Clark JE (eds) The development of movement control and coordination. Wiley, New York, pp 5–78 Kuhbandner C, Pekrun R, Maier M (2010) The role of negative affect in the “mirroring” of other person’s actions. Cogn Emotion 24:1182–1190 Liszkowski U, Carpenter M, Striano T, Tomasello M (2006) 12- and 18-month-olds point to provide information for others. J Cogn Dev 7:173–187 Loehr J, Kourtis D, Vesper C, Sebanz N, Knoblich G (2013) Monitoring individual and joint action outcomes in duet music performance. J Cogn Neurosci 25:1049–1061 Lopresti-Goodman SM, Silva PL, Richardson MJ, Schmidt RC (2006) Frequency basin of entrainment for unintentional coordination. Paper presented at the North American Meeting of the International Society for ecological Psychology. University of Cincinnati, Cincinnati Masumoto J, Inui N (2014a) A leader-follower relationship in joint action on a discrete force production task. Exp Brain Res 232:3525–3533 Masumoto J, Inui N (2014b) Effects of speech on both complementary and synchronous strategies in joint action. Exp Brain Res 232:2421–2429 Masumoto J, Inui N (2017) Bidirectinal transfer between joint and individual actions in a task of discrete force production. Exp Brain Res 235:2259–2265 McClung JS, Jentzsch I, Reich SD (2013) Group membership affects spontaneous mental representation: failure to represent the out-group in a joint action task. PLoS One 8(11):e79178 Meltzoff AN, Moore MK (1983) Newborn infants imitate adult facial gestures. Child Dev 54:702–709 Meyer M, Bekkering H, Haartsen R, Stapel JC, Hunnius S (2015) The role of action prediction and inhibitory control for joint action coordination in toddlers. J Exp Child Psychol 139:203–220 Milanese N, Iani C, Rubichi S (2010) Shared learning shapes human performance: transfer effects in task sharing. Cognition 116:15–22 Milanese N, Iani C, Sebanz N, Rubichi S (2011) Contextual determinants of the social-transfer-of- learning effect. Exp Brain Res 211:415–422 Moore C (2007) Understanding self and others in the second year. In: Brownell CA, Kopp CB (eds) Socioemotional development in the toddler years: transitions and transformations. Guilford Press, New York, pp 43–65

References

151

Moore C, Dunham PJ (1995) Joint attention: its origins and role in development. Lawrence Erlbaum Associates, Hillsdale Prinz W (1997) Perception and action planning. Eur J Cogn Psychol 9:129–154 Reddy V (2008) How infants know minds. Harvard University Press, Cambridge Repp BH (2005) Sensorimotor synchronization: a review of taping literature. Psychon Bull Rev 12:969–992 Repp BH, Knoblich G (2004) Perceiving action identity: how pianists recognize their own performances. Psychol Sci 15:604–609 Richardson MJ, Marsh KL, Schmidt RC (2005) Effects of visual and verbal information on unintentional interpersonal coordination. J Exp Psychol Human Percep Perfom 31:62–79 Richardson MJ, Marsh KL, Isenhower R, Goodman J, Schmidt RC (2007) Rocking together: dynamics of intentional and unintentional interpersonal coordination. Hum Mov Sci 26:867–891 Riggo R (1986) Assessment of basic social skills. J Pers Soc Psychol 51:649–660 Rizzolatti G, Sinigaglia C (2010) The functional role of the parteto-frontal mirror circuit: interpretations an\d misinterpretations. Nat Rev Neurosci 11:264–274 Roerdink J, Peper CM, Beek PJ (2005) Effects of correct and transformed visual feedback on rhythmic visuo-motor tracking performance and visual search behavior. Hum Mov Sci 24:379–402 Ross HS, Lollis SP (1987) Communication within infant social games. Dev Psychol 23:241–248 Rutter DR, Durkin K (1987) Turn-taking in mother-infant interaction: an examination of vocalizations and gaze. Dev Psychol 23:54–61 Ruys KI, Aarts H (2010) When competition merges people’s behavior: interdependency activates shared action representations. J Exp Soc Psychol 46:1130–1133 Sacheli L, Tidoni E, Pavone E, Aglioti S, Candidi M (2013) Kinematics fingerprints of leader and follower role-taking during cooperative joint actions. Exp Brain Res 226:473–486 Santamaria JP, Rosenbaum DA (2011) Etiquette and effort: holding doors for others. Psychol Sci 22:584–588 Santvoord AAM, Beek PJ (1994) Phasing and the pickup of optical information in cascade juggling. Ecol Psychol 6:239–263 Sartori L, Becchio C, Castiello U (2011) Cues to intention: the role of movement information. Cognition 119:242–252 Sato A, Yasuda A (2005) Illusion of sense of self-agency: discrepancy between the predicted and actual sensory consequences of actions modulates the sense of self-agency, but not the sense of sense of self-ownership. Cognition 94:241–255 Schmidt RC, O’Brien B (1997) Evaluating the dynamics of unintended interpersonal coordination. Ecol Psychol 9(3):189–206 Schmidt RC, Richardson MJ (2008) Dynamics of interpersonal coordination. In: Fuchs A, Jirsa VK (eds) Coordination: neural, behavioral and social dynamics. Springer, Berlin, pp 281–308 Schmidt RC, Turvey MT (1994) Phase-entrainment dynamics of visually coupled rhythmic movements. Biol Cybern 70:369–376 Schmidt RC, Turvey MT (1995) Models of interlimb coordination: equilibria, local analyses, and spectral patterning. J Exp Psychol: Hum Percept Perform 21:432–443 Schmidt RC, Carello C, Turvey MT (1990) Phase transitions and critical fluctuations in the visual coordination of rhythmic movements between people. J Exp Psychol Hum Percept Perform 16:227–247 Schmidt RC, Shaw BK, Turvey MT (1993) Coupling dynamics in interlimb coordination. J Exp Psychol Hum Percept Perform 19:397–415 Schmidt RC, Christianson N, Carello C, Baron R (1994) Effects of social and physical variables on between-person visual coordination. Ecol Psychol 6:159–183 Schmidt RC, Bienvenu M, Fitzpatrick PA, Amazeen PG (1998) A comparison of intra- and interpersonal interlimb coordination: coordination breakdowns and coupling strength. J Exp Psychol Human Percep Perfom 24:884–900 Schmidt RC, Richardson MJ, Arsenault CA, Galantucci B (2007) Visual tracking and entrainent to an environmental rhythm. J Exp Psychol Hum Percept Perform 33:860–870

152

3 An Overview of the Study on Interpersonal Coordination

Schöner G, Haken H, Kelso JAS (1986) A stochastic theory of phase transitions in human hand movement. Biol Cybern 53:247–257 Schuch S, Tipper SP (2007) On observing another person’s actions: influences of observed inhibition and errors. Percept Psychophys 69:828–837 Seale J (1990) Collective intentions and actions. In: Cohen P, Morgan J (eds) Intentions in communication. MIT Press, Cambridge, pp 401–415 Sebanz N, Knoblich G (2009) Prediction in joint action: what, when, and where. Top Cogn Sci 1:353–367 Sebanz N, Knoblich G, Prinz W (2003) Representing others’ actions: just like one’s own? Cognition 88:B11–B21 Sebanz N, Bekkering H, Knoblich G (2006) Joint action: bodies and minds moving together. Trends Cogn Sci 10:70–76 Shockley K, Santana MV, Fowler CA (2003) Mutual interpersonal postural constraints are involved in cooperative conversation. J Exp Psychol Hum Percept Perform 29:326–332 Shockley K, Baker AA, Richardson MJ, Fowler CA (2007) Verbal constraints on interpersonal postural coordination. J Exp Psychol Hum Percept Perform 33:201–208 Simon JR, Hinrich JV, Craft JL (1970) Auditory S-R compatibility: reaction time as a function of ear-hand correspondence and ear-response-location correspondence. J Exp Psychol 86:97–102 Skerry AE, Carey SE, Spelke ES (2013) First-person action experience reveals sensitivity to action efficiency in prereaching infants. Proc Natl Acad Sci 110(46):18728–18733 Sommerville JA, Woodward AL, Needham A (2005) Action experience alters 3-month-old infants’ perception of others’ action. Cognition 96:B1–B11 Stern DN (2002) The first relationship: infant and mother. Harvard University Press, Cambridge Sternad D, Turvey MT, Schmidt RC (1992) Average phase difference theory and 1:1 phase entrainment in interlimb coordination. Biol Cybern 67:223–231 Tomasello M, Carpenter M, Call J, Behne T, Moll H (2005) Understanding and sharing intentions: the origins of cultural cognition. Behav Brain Sci 28:675–735 Trevarthen C (1979) Communication and cooperation in early infancy: a description of primary intersubjectivity. In: Bullowa M (ed) Before speech: the beginning of interpersonal communication. Cambridge University Press, Cambridge, pp 321–380 Tsai C-C, Kuo W-J, Jing J-T, Hung DL, Tzeng OJ-L (2006) A common coding framework in self- other interaction: evidence from joint action task. Exp Brain Res 175:353–362 Tsai C-C, Sebanz N, Knoblich G (2011) The GROUP effect: groups mimic group action. Cognition 118:135–140 Tuomela R (2005) We-intentions revisited. Philos Stud 125:327–369 Turvey MT, Rosenblum LD, Schmidt RC, Kugler PN (1986) Fluctuations and phase symmetry in coordinated rhythmic movement. J Exp Psychol Hum Percept Perform 12:564–583 Urgesi C, Savonitto M, Fabbro F, Aglioti SM (2012) Long- and short-term plastic modeling of action prediction abilities in volleyball. Psychol Res 76:540–562 van Avermaet E (1995) Cooperation and competition. In: ASR M, Hewstne M (eds) The Blackwell encyclopedia of social psychology. Blackwell, Oxford, pp 136–141 van der Wel R, Knoblich G, Sebanz N (2011) Let the force be with us: dyads exploit haptic coupling for coordination. J Exp Psychol Hum Percept Perform 37:1420–1431 Van Overwalle F, Baetens K (2009) Understanding others’ actions and goals by mirror and mentalizing systems: a meta-analysis. NeuroImage 48:564–584 van Schie HT, Mars RB, Coles MGH, Bekkering H (2004) Modulation of activity in medial frontal and motor cortices during error observation. Nat Neurosci 5:1089–1092 Vesper C, Richardson J (2014) Strategic communication and behavioral coupling in asymmetric joint action. Exp Brain Res 232:2945–2956 Vesper C, Sebanz N (2016) Acting together: representations and coordination processes. In: Obhi SS, Cross ES (eds) Shared representations: sensorimotor foundations of social life. Cambridge University Press, Cambridge, pp 216–235

References

153

Vesper C, van der Wel R, Knoblich G, Sebanz N (2011) Making oneself predictable: reduced temporal variability facilitates joint action coordination. Exp Brain Res 211:517–530 Vesper C, van der Wel R, Knoblich G, Sebanz N (2013) Are you ready to jump? Predictive mechanisms in interpersonal coordination. J Exp Psychol Hum Percept Perform 39:48–61 Vesper C, Schmitz L, Safra L, Sebanz N, Knoblich G (2016) The role of shared visual information for joint action coordination. Cognition 153:118–123 von Holst E (1973) The collected papers of Eric von Holst. In: Martin R, Trans. The behavioral physiology of animal and man, vol 1. University of Miami Press, Coral Gables (Original work published 1939) Warneken F, Chen F, Tomasello M (2006) Cooperative activities in young children and chimpanzees. Child Dev 77:640–663 Warren WH, Yaffe DM (1989) Dynamics of step length adjustment during running: a comment on Patla. Robinson, Samways, Armstrong. J Exp Psychol Hum Pecept Perfom 15:618–623 Witt JK, Proffitt DR, Epstein W (2005) Tool use affects perceived distance, but only when you intend to use it. J Exp Psychol Hum Percept Perform 31:880–888 Wolpert DM, Doya K, Kawato M (2003) A unifying computational framework for motor control and interaction. Philos Trans R Soc Lond B 358:593–602

Chapter 4

Complementary and Synchronous Force Production in Joint Action

Abstract If two people lift and carry an object, they not only produce complementary forces on the object but also walk in synchrony. The first study thus tested the hypothesis that complementary and synchronous strategies simultaneously facilitated the action coordination performed by two people in a task of periodic isometric force production. When the total force was visible, the correlation between forces produced by two participants was highly negative, and the relative phase angles were then distributed at the 0–20° phase region. These innovative findings indicate that one participant compensated for another’s force errors and produced forces in synchrony exclusively when the total force was visible. The joint action then controlled force more accurately than the individual action. The second study examined the new force-sharing patterns in joint action performed by two, three, or four participants, and it further tested whether an increase in the number of participants results in a decrease in individual contribution (social loafing). In both the three- and four-person tasks, the correlation between forces produced by two of the three or four participants was negative, and the remaining one or two participants produced intermediate forces. The errors of force and movement interval and force variabilities were smaller in four- and three-people groups than individuals. The second study thus eliminated the effect of social loafing. The third study examined development of a leader-follower relationship in joint action performed by participants with different skill levels. Participants with low force variability produced a stronger and earlier force than those with high force variability. If two people row a boat, they often call to each other to synchronize their strokes. It is anticipated that such a call promotes periodic joint action. The fourth study thus examined the effects of speech on both complementary and synchronous strategies in joint action. Although periodically uttering a syllable worsened complementary force production when the target was visible, it promoted synchronization of their performance to each other’s timing when the target was invisible. All motor control theories (i.e., uncontrolled manifold hypothesis) predict that as the error produced by participants increases, error compensation also increases. The fifth study thus tested the hypothesis that a load perturbation facilitates interpersonal compensation for force error. Two cooperative participants a and b produced a target force, and the force pro-

© Springer Nature Singapore Pte Ltd. 2018 N. Inui, Interpersonal Coordination, https://doi.org/10.1007/978-981-13-1765-1_4

155

156

4 Complementary and Synchronous Force Production in Joint Action

duced by another participant c increased or decreased the sum of the forces produced by participants a and b. The load perturbation facilitated interpersonal compensation for force error, but caused performance to deteriorate. Keywords Complementarity · Synchronization · Social loafing · Leader-follower relationship · Speech · Load perturbation

4.1 Two Heads Are Better Than One Daily life, as well as sports and art, sometimes requires coordination of movements performed by two persons. For example, two people can lift and carry a table in real life, while two pianists can synchronize their performance to each other’s timing with asynchronies of 30 ms (Keller et al. 2007). Such group action coordination has recently been studied using the term ‘joint action’, which is defined as a social interaction whereby two or more individuals coordinate their actions in space and time to bring about a change in the environment (Sebanz et al. 2006). However, the behavioral and neural processes underlying such joint action are still poorly understood. Previous studies have identified two key mechanisms subserving temporal coordination. First, some studies from the dynamical systems perspective (Haken et al. 1985) have shown that when two people perform a rhythmic action while they can see each other, in many cases they fall into the same rhythm, known as entrainment (Schmidt et al. 1990; Richardson et al. 2007). Such studies have also indicated that intrapersonal coordination scales up to the interpersonal case (Marsh et al. 2009). However, because such coordination can occur spontaneously between individuals who have no plan to perform actions together, the dynamical systems approach cannot explain how individuals adjust the timing of their actions to others to achieve a common outcome (Knoblich et al. 2011). Second, some studies on joint action have revealed how individuals incorporate others’ actions into their own action planning (Isenhower et al. 2005) and how temporal feedback about others’ actions is used in anticipatory action control (Knoblich and Jordan 2003). Most of the previous studies on entrainment and joint action employed continuous rhythmical tasks in which coordination between two people occurred were based on visual (Richardson et al. 2007), haptic (van der Wel et al. 2011), or auditory information (Keller et al. 2007; Konvalinka et al. 2010). Recently, behavioral (van Schie et al. 2008) and brain imaging (Newman- Norlund et al. 2007) studies on joint action have investigated differences between imitative and complementary actions. As a sample of an imitative action, Brass et al. (2001) asked participants to observe a video of either a lifting movement or a tapping movement on the computer. The participants’ reaction times to perform the lifting or tapping movements were faster following observation of the lifting or tapping movements. Many studies on observation of an action have found that a corresponding representation in the observer’s action system is activated in the premotor

4.1 Two Heads Are Better Than One

157

and parietal cortices pertaining to the ‘mirror neuron system’ (Rizzolatti and Craighero 2004). In a complementary action, Knoblich and Jordan (2003) asked individuals and pairs to perform a motor task keeping a tracker on a target. The target was moved horizontally across a computer monitor by pressing right and left keys that incremented the tracker’s velocity to the right or left, respectively. Although the pairs had a larger error between the target and tracker than the individuals in the initial practice, there was no difference between the pairs and individuals in the error in the final practice. Bosga and Meulenbroek (2007) asked pairs of two participants to perform a virtual lifting task using continuous isometric force with two or four hands. The forces produced by two participants were negatively correlated when visual feedback of their forces was available, indicating the use of complementary forces to control a virtual bar. Neuroimaging studies in humans found that activities in the mirror neuron system areas were greater during complementary action than imitative action (Newman-Norlund et al. 2007). The areas were also more active in the joint condition than in the solo condition during the same virtual bar lifting task (Newman-Norlund et al. 2008). If two people lift and carry an object in real-life situations, they not only produce complementary forces on the object but also walk in synchrony. However, previous studies separately examined complementary (Bosga and Meulenbroek 2007; Knoblich and Jordan 2003) and synchronous (Keller et al. 2007; Konvalinka et al. 2010) strategies in joint actions. This study thus tested the hypothesis that two coordination strategies simultaneously facilitated the action coordination performed by two people. This study examined the complementary strategy in a joint action using a periodic isometric force production such that the sum of the forces produced by two participants was the target force. A periodic motor task with a prescribed movement interval also allows us to analyze the synchronous strategy in the joint action. While Brass et al. (2001) gave participants visual information of the other’s movement, Bosga and Meulenbroek (2007) gave them that of the sum of forces produced by two participants. Presumably, such different types of visual information have different influences on coordination strategy in a joint action. Our study used a motor task such that the sum of the forces produced by two participants was the target force. It is thus anticipated that a joint action is facilitated by visual information of the sum of forces rather than by the information of each participant’s force output. To examine the effects of different types of visual information on complementary and synchronous strategies in a joint action, this study set four conditions of visual information. Experiments were performed using healthy 20 male participants without any apparent neurological disorders (mean age = 22.6 years, standard deviation (SD) = 1.9 years). Handedness was tested using the Edinburgh Handedness Inventory (Oldfield 1971). All 20 participants were right hand dominant. This study consisted of the individual condition performed by one participant and the total-force, both-forces, partner-force, and no-feedback conditions performed by two participants paired randomly. The individual condition was conducted using a half of the setup shown in Fig. 4.1a. In the individual condition, 20 participants were seated facing the load cell, and their palms rested on a support surface 6 cm above the table. In this posture, the participants made periodic isometric pressing move-

158

4 Complementary and Synchronous Force Production in Joint Action

Fig. 4.1 Experimental setup, dependent variables, and feedback conditions. (a) Experimental setup. While the individual condition was performed by one participant using a half of the setup, the other four conditions were conducted by two participants using all of the setup. (b) Definition and measurement of dependent variables. (c) Computer displays under four conditions. The total- force condition displayed the target forces (two black straight lines) and a sum of the forces produced by the two participants (one red sinusoidal line) on a monitor. The both-force condition displayed the force outputs produced by the two participants separately (two red sinusoidal lines) and the personal targets (four black straight lines), which represent 5% and 10% MVC for each participant, on a monitor. While the force output produced by a participant was presented at the top of a monitor, the output produced by his partner was presented at the bottom. The partner-force condition displayed only a partner’s force output (one red sinusoidal line) and his target (two black straight lines) on a monitor. The no-feedback condition removed any visual information from the monitor. Abbreviation. Total: total-force condition, Both: both-forces condition, Partner: partner- force condition, No-FB: no-feedback condition. (Masumoto and Inui 2013, redraw with permissision from the Journal of Neurophysiology)

ments with the right index finger at the metacarpophalangeal joint with a target peak force of 10% maximum voluntary contraction (MVC), a target valley force of 5% MVC, and a target peak-to-peak (PPI) or valley-to-valley interval (VVI) of 1000 ms (Fig. 4.1b). The force output of the load cell was displayed on a monitor screen so that the participants could see the difference between the actual force and target force, both of which were indicated on the screen by two horizontal lines. In the total-force, both-forces, partner-force, and no-feedback conditions, the participants seated on chairs at opposite ends of a table facing the load cell and monitor (Fig. 4.1a). A target peak or valley force was the sum of 10% or 5% MVC produced by the right index fingers of two participants (Fig. 4.1b). Because each participant was required to produce 10% or 5% of their own MVC force, the target was really 10% or 5% of a combined-across-participants MVC under four conditions (Fig. 4.1c): (1) The total-force condition displayed the target forces for the pair and a sum of the forces produced by the two participants on a monitor. (2) The both- forces condition displayed the force outputs produced by the two participants separately and the personal targets, which represent 5% and 10% MVC for each

4.1 Two Heads Are Better Than One

159

participant, on a monitor. While the force output produced by a participant was presented at the top of a monitor, the output produced by his partner was presented at the bottom. (3) The partner-force condition displayed only a partner’s force output and his target on a monitor. (4) The no-feedback condition removed any visual information from the monitor. Under these conditions, the participants were instructed to synchronize their force with the partner’s force. While the participants could not see the other’s action because of the two monitors between them, they were instructed not to verbally communicate to each other. At the start of the experimental session, the participants produced their maximum force to measure the isometric MVC generated at the finger tip of the right index finger. The MVC (mean = 45.1, SD = 2.3 N) was determined by an average of three trials. The order of the conditions performed by the participants was varied randomly. The participants practiced each condition separately with the corresponding test trial following immediately after the practice trials. They pressed their fingers against the load cell for 60 cycles in five practice trials for each condition. During practice trials, the pressing rate was prescribed by means of an audible metronome. The participants were instructed to synchronize finger presses on the load cell with the metronome. On the test trial, although they were given the same visual information as the practice trials, they were instructed to produce the interval acquired during the practice trials by means of self-paced movement without the metronome pulse. The output of the load cell pressed by participants was amplified and recorded by a personal computer after the amplified signal was converted from analogue to digital. The force output was also displayed on a 20-inch computer monitor located approximately 60 cm in front of the participant. Data were sampled at a frequency of 1000 Hz by a 16 bit A/D converter with a low-pass filter for 100 Hz. Figure 4.2 shows a data sample collected from two participants. Peak force, valley force, PPI, and VVI were measured using software for analysis of force and interval. In analyses of the test trials, the dependent measures were the average values corresponding to the dependent measures produced. (1) Complementary strategy, (2) frequency synchrony, (3) phase synchrony, (4) accuracy of force production and interval, and (5) variabilities of force and interval were calculated as follows: (1) to examine the complementary strategy of forces produced by two participants under four conditions, correlation coefficients between the forces they produced were calculated for the peak or valley force. (2) The cross-spectral coherence was calculated to assess the frequency synchrony between movements produced by two participants, which evaluated the correlation of the two force-time series at different frequencies. The coherence was calculated over all force-times series (100 samples/s) with a window length of 500 points (frequency resolution of 0.2 Hz) by using the mscohere command in Octave Forge ver. 3. 6. 1 (John W. Eaton, freeware). Thus, the peak coherence over all frequencies was used as an estimate of the frequency synchrony between forces produced by the two participants. (3) The distribution of relative phase angles between the force-time series produced by the two participants was used to quantify the phase synchrony between their force outputs. The continuous relative phase was first computed using the Hilbert transform (Rosenblum and

160

4 Complementary and Synchronous Force Production in Joint Action

Fig. 4.2 Data samples of the total force (TF) and finger forces of participant A (AF) and B (BF) under the four conditions. In the total-force condition, the all phase difference was 0° and the summed force also matched the target forces nicely. Perfect frequency synchrony and complete complementary sinusoidal force-time functions would result in a constant force output because then the force changes would cancel each other out. In the first two cycles under the no-feedback condition, a 90° phase difference generated the summed force output as required by the task. In the third cycle, the phase difference was reduced to 0° and the summed force also matched the target forces nicely. Then, in cycles 5–7, the anti-phase pattern occurred yielding a small-amplitude summed force-time function. Indeed, with the 180° phase difference, the resulting summed force- time function would be the difference between the individual force-time functions. Conventions are as in Fig. 4.1. (Masumoto and Inui 2013, redraw with permissision from the Journal of Neurophysiology)

Kurths 1998). The Hilbert transform was calculated by using the Hilbert command in Octave Forge. The distribution of relative phase angles examined the concentration of relative phase angles between forces produced by two participants across nine 20° regions of relative phase (0–20°, 21–40°, 41–60°, 61–80°, 81–100°, 101– 120°, 121–140°, 141–160°, 161–180°). The phase synchrony was indicated by a high concentration of relative phase angles near 0°, while an even distribution indicated no phase synchrony. (4) Absolute error (AE) was calculated to assess the accuracy of force production and interval. The AE was calculated by taking the size of each error (the difference between produced and target forces or intervals), regardless of sign, and averaging it over 50 measures. (5) SD for total force (peak or valley force) and interval (PPI or VVI) were calculated to examine the variability of force and timing. Statistical analyses were performed as follows: (1) analysis of complementary strategy: correlation coefficient values were standardized using a Fisher z transformation for averaging across pairs. A 4 (condition: total-force, both-forces, partner- force, vs. no-feedback condition) × 2 (force: peak vs. valley force) analysis of variance (ANOVA) was performed to examine the main effects on the correlation between forces produced by two participants. (2) Analysis of coherence: a one-way ANOVA was performed to examine the main effects on the cross-spectral coherence between forces produced by two participants. (3) Analysis of the distribution of relative phase angles: a 4 (condition) × 9 (phase region: 0–20°, 21–40°, 41–60°,

4.1 Two Heads Are Better Than One

161

61–80°, 81–100°, 101–120°, 121–140°, 141–160°, 161–180°) ANOVA was performed to examine the main effects on phase region. (4) Analysis of accuracy in force and interval: a 5 (condition: total-force, both-forces, partner-force, no- feedback vs. individual condition) × 2 (force or interval: peak vs. valley force or PPI vs. VVI) ANOVA was performed to examine the main effects on the AE of force or interval. (5) Analysis of force and timing variabilities: a 5 (condition) × 2 (force or interval) ANOVA was performed to examine the main effects on the SD of force and interval. When significant overall condition effects were found for a dependent measure, post-hoc multiple comparisons detected differences between conditions using Tukey’s honestly significant difference test. Statistical significance was defined at the p

Interpersonal Coordination

Recommend Stories

Idea Transcript

Helpful Links

Smile Life

Get in touch