WorldWideScience

Sample records for volubility gestural production

  1. Maternal Label and Gesture Use Affects Acquisition of Specific Object Names

    Science.gov (United States)

    Zammit, Maria; Schafer, Graham

    2011-01-01

    Ten mothers were observed prospectively, interacting with their infants aged 0 ; 10 in two contexts (picture description and noun description). Maternal communicative behaviours were coded for volubility, gestural production and labelling style. Verbal labelling events were categorized into three exclusive categories: label only; label plus…

  2. TOT phenomena: Gesture production in younger and older adults.

    Science.gov (United States)

    Theocharopoulou, Foteini; Cocks, Naomi; Pring, Timothy; Dipper, Lucy T

    2015-06-01

    This study explored age-related changes in gesture to better understand the relationship between gesture and word retrieval from memory. The frequency of gestures during tip-of-the-tongue (TOT) states highlights this relationship. There is a lack of evidence describing the form and content of iconic gestures arising spontaneously in such TOT states and a parallel gap addressing age-related variations. In this study, TOT states were induced in 45 participants from 2 age groups (older and younger adults) using a pseudoword paradigm. The type and frequency of gestures produced was recorded during 2 experimental conditions (single-word retrieval and narrative task). We found that both groups experienced a high number of TOT states, during which they gestured. Iconic co-TOT gestures were more common than noniconic gestures. Although there was no age effect on the type of gestures produced, there was a significant, task-specific age difference in the amount of gesturing. That is, younger adults gestured more in the narrative task, whereas older adults generated more gestures in the single-word-retrieval task. Task-specific age differences suggest that there are age-related differences in terms of the cognitive operations involved in TOT gesture production. (c) 2015 APA, all rights reserved.

  3. Hippocampal declarative memory supports gesture production: Evidence from amnesia.

    Science.gov (United States)

    Hilverman, Caitlin; Cook, Susan Wagner; Duff, Melissa C

    2016-12-01

    Spontaneous co-speech hand gestures provide a visuospatial representation of what is being communicated in spoken language. Although it is clear that gestures emerge from representations in memory for what is being communicated (De Ruiter, 1998; Wesp, Hesse, Keutmann, & Wheaton, 2001), the mechanism supporting the relationship between gesture and memory is unknown. Current theories of gesture production posit that action - supported by motor areas of the brain - is key in determining whether gestures are produced. We propose that when and how gestures are produced is determined in part by hippocampally-mediated declarative memory. We examined the speech and gesture of healthy older adults and of memory-impaired patients with hippocampal amnesia during four discourse tasks that required accessing episodes and information from the remote past. Consistent with previous reports of impoverished spoken language in patients with hippocampal amnesia, we predicted that these patients, who have difficulty generating multifaceted declarative memory representations, may in turn have impoverished gesture production. We found that patients gestured less overall relative to healthy comparison participants, and that this was particularly evident in tasks that may rely more heavily on declarative memory. Thus, gestures do not just emerge from the motor representation activated for speaking, but are also sensitive to the representation available in hippocampal declarative memory, suggesting a direct link between memory and gesture production. Copyright © 2016 Elsevier Ltd. All rights reserved.

  4. Gestures make memories, but what kind? Patients with impaired procedural memory display disruptions in gesture production and comprehension.

    Science.gov (United States)

    Klooster, Nathaniel B; Cook, Susan W; Uc, Ergun Y; Duff, Melissa C

    2014-01-01

    Hand gesture, a ubiquitous feature of human interaction, facilitates communication. Gesture also facilitates new learning, benefiting speakers and listeners alike. Thus, gestures must impact cognition beyond simply supporting the expression of already-formed ideas. However, the cognitive and neural mechanisms supporting the effects of gesture on learning and memory are largely unknown. We hypothesized that gesture's ability to drive new learning is supported by procedural memory and that procedural memory deficits will disrupt gesture production and comprehension. We tested this proposal in patients with intact declarative memory, but impaired procedural memory as a consequence of Parkinson's disease (PD), and healthy comparison participants with intact declarative and procedural memory. In separate experiments, we manipulated the gestures participants saw and produced in a Tower of Hanoi (TOH) paradigm. In the first experiment, participants solved the task either on a physical board, requiring high arching movements to manipulate the discs from peg to peg, or on a computer, requiring only flat, sideways movements of the mouse. When explaining the task, healthy participants with intact procedural memory displayed evidence of their previous experience in their gestures, producing higher, more arching hand gestures after solving on a physical board, and smaller, flatter gestures after solving on a computer. In the second experiment, healthy participants who saw high arching hand gestures in an explanation prior to solving the task subsequently moved the mouse with significantly higher curvature than those who saw smaller, flatter gestures prior to solving the task. These patterns were absent in both gesture production and comprehension experiments in patients with procedural memory impairment. These findings suggest that the procedural memory system supports the ability of gesture to drive new learning.

  5. Gestures make memories, but what kind? Patients with impaired procedural memory display disruptions in gesture production and comprehension

    Directory of Open Access Journals (Sweden)

    Nathaniel Bloem Klooster

    2015-01-01

    Full Text Available Hand gesture, a ubiquitous feature of human interaction, facilitates communication. Gesture also facilitates new learning, benefiting speakers and listeners alike. Thus, gestures must impact cognition beyond simply supporting the expression of already-formed ideas. However, the cognitive and neural mechanisms supporting the effects of gesture on learning and memory are largely unknown. We hypothesized that gesture’s ability to drive new learning is supported by procedural memory and that procedural memory deficits will disrupt gesture production and comprehension. We tested this proposal in patients with intact declarative memory, but impaired procedural memory as a consequence of Parkinson’s disease, and healthy comparison participants with intact declarative and procedural memory. In separate experiments, we manipulated the gestures participants saw and produced in a Tower of Hanoi paradigm. In the first experiment, participants solved the task either on a physical board, requiring high arching movements to manipulate the discs from peg to peg, or on a computer, requiring only flat, sideways movements of the mouse. When explaining the task, healthy participants with intact procedural memory displayed evidence of their previous experience in their gestures, producing higher, more arching hand gestures after solving on a physical board, and smaller, flatter gestures after solving on a computer. In the second experiment, healthy participants who saw high arching hand gestures in an explanation prior to solving the task subsequently moved the mouse with significantly higher curvature than those who saw smaller, flatter gestures prior to solving the task. These patterns were absent in both gesture production and comprehension experiments in patients with procedural memory impairment. These findings suggest that the procedural memory system supports the ability of gesture to drive new learning.

  6. A common functional neural network for overt production of speech and gesture.

    Science.gov (United States)

    Marstaller, L; Burianová, H

    2015-01-22

    The perception of co-speech gestures, i.e., hand movements that co-occur with speech, has been investigated by several studies. The results show that the perception of co-speech gestures engages a core set of frontal, temporal, and parietal areas. However, no study has yet investigated the neural processes underlying the production of co-speech gestures. Specifically, it remains an open question whether Broca's area is central to the coordination of speech and gestures as has been suggested previously. The objective of this study was to use functional magnetic resonance imaging to (i) investigate the regional activations underlying overt production of speech, gestures, and co-speech gestures, and (ii) examine functional connectivity with Broca's area. We hypothesized that co-speech gesture production would activate frontal, temporal, and parietal regions that are similar to areas previously found during co-speech gesture perception and that both speech and gesture as well as co-speech gesture production would engage a neural network connected to Broca's area. Whole-brain analysis confirmed our hypothesis and showed that co-speech gesturing did engage brain areas that form part of networks known to subserve language and gesture. Functional connectivity analysis further revealed a functional network connected to Broca's area that is common to speech, gesture, and co-speech gesture production. This network consists of brain areas that play essential roles in motor control, suggesting that the coordination of speech and gesture is mediated by a shared motor control network. Our findings thus lend support to the idea that speech can influence co-speech gesture production on a motoric level. Copyright © 2014 IBRO. Published by Elsevier Ltd. All rights reserved.

  7. An investigation of co-speech gesture production during action description in Parkinson's disease.

    Science.gov (United States)

    Cleary, Rebecca A; Poliakoff, Ellen; Galpin, Adam; Dick, Jeremy P R; Holler, Judith

    2011-12-01

    Parkinson's disease (PD) can impact enormously on speech communication. One aspect of non-verbal behaviour closely tied to speech is co-speech gesture production. In healthy people, co-speech gestures can add significant meaning and emphasis to speech. There is, however, little research into how this important channel of communication is affected in PD. The present study provides a systematic analysis of co-speech gestures which spontaneously accompany the description of actions in a group of PD patients (N = 23, Hoehn and Yahr Stage III or less) and age-matched healthy controls (N = 22). The analysis considers different co-speech gesture types, using established classification schemes from the field of gesture research. The analysis focuses on the rate of these gestures as well as on their qualitative nature. In doing so, the analysis attempts to overcome several methodological shortcomings of research in this area. Contrary to expectation, gesture rate was not significantly affected in our patient group, with relatively mild PD. This indicates that co-speech gestures could compensate for speech problems. However, while gesture rate seems unaffected, the qualitative precision of gestures representing actions was significantly reduced. This study demonstrates the feasibility of carrying out fine-grained, detailed analyses of gestures in PD and offers insights into an as yet neglected facet of communication in patients with PD. Based on the present findings, an important next step is the closer investigation of the qualitative changes in gesture (including different communicative situations) and an analysis of the heterogeneity in co-speech gesture production in PD. Copyright © 2011 Elsevier Ltd. All rights reserved.

  8. Reduction in gesture during the production of repeated references

    NARCIS (Netherlands)

    Hoetjes, M.W.; Koolen, R.M.F.; Goudbeek, M.B.; Krahmer, E.J.; Swerts, M.G.J.

    2015-01-01

    In dialogue, repeated references contain fewer words (which are also acoustically reduced) and fewer gestures than initial ones. In this paper, we describe three experiments studying to what extent gesture reduction is comparable to other forms of linguistic reduction. Since previous studies showed

  9. Gestures make memories, but what kind? Patients with impaired procedural memory display disruptions in gesture production and comprehension

    OpenAIRE

    Klooster, Nathaniel B.; Cook, Susan W.; Uc, Ergun Y.; Duff, Melissa C.

    2015-01-01

    Hand gesture, a ubiquitous feature of human interaction, facilitates communication. Gesture also facilitates new learning, benefiting speakers and listeners alike. Thus, gestures must impact cognition beyond simply supporting the expression of already-formed ideas. However, the cognitive and neural mechanisms supporting the effects of gesture on learning and memory are largely unknown. We hypothesized that gesture's ability to drive new learning is supported by procedural memory and that proc...

  10. Voluble: a space-time diagram of the solar system

    Science.gov (United States)

    Aguilera, Julieta C.; SubbaRao, Mark U.

    2008-02-01

    Voluble is a dynamic space-time diagram of the solar system. Voluble is designed to help users understand the relationship between space and time in the motion of the planets around the sun. Voluble is set in virtual reality to relate these movements to our experience of immediate space. Beyond just the visual, understanding dynamic systems is naturally associated to the articulation of our bodies as we perform a number of complex calculations, albeit unconsciously, to deal with simple tasks. Such capabilities encompass spatial perception and memory. Voluble investigates the balance between the visually abstract and the spatially figurative in immersive development to help illuminate phenomena that are beyond the reach of human scale and time. While most diagrams, even computer-based interactive ones, are flat, three-dimensional real-time virtual reality representations are closer to our experience of space. The representation can be seen as if it was "really there," engaging a larger number of cues pertaining to our everyday spatial experience.

  11. Gestures make memories, but what kind? Patients with impaired procedural memory display disruptions in gesture production and comprehension

    OpenAIRE

    Nathaniel Bloem Klooster; Nathaniel Bloem Klooster; Susan Wagner Cook; Susan Wagner Cook; Ergun Y. Uc; Ergun Y. Uc; Melissa C. Duff; Melissa C. Duff; Melissa C. Duff; Melissa C. Duff

    2015-01-01

    Hand gesture, a ubiquitous feature of human interaction, facilitates communication. Gesture also facilitates new learning, benefiting speakers and listeners alike. Thus, gestures must impact cognition beyond simply supporting the expression of already-formed ideas. However, the cognitive and neural mechanisms supporting the effects of gesture on learning and memory are largely unknown. We hypothesized that gesture’s ability to drive new learning is supported by procedural memory and that proc...

  12. Volubility, consonant, and syllable characteristics in infants and toddlers later diagnosed with childhood apraxia of speech: A pilot study.

    Science.gov (United States)

    Overby, Megan; Caspari, Susan S

    2015-01-01

    This pilot study explored the volubility, consonant singleton acquisition, and syllable structure development between infants and toddlers (birth-24 months) with typical speech sound production (TYP) and those later diagnosed with childhood apraxia of speech (CAS). A retrospective longitudinal between- and within-subjects research design was utilized (TYP N=2; CAS N=4). Vocalizations from participants were analyzed between birth-24 months from home videotapes, volunteered by the children's parents, according to type (nonresonant vs. resonant), volubility, place and manner of consonant singletons, and syllable shape (V, CV, VC, CVC, VCV, CVCV, VCVC, and "Other"). Volubility between groups was not significant but statistically significant differences were found in the number of: resonant and non-resonant productions; different consonant singletons; different place features; different manner classes; and proportional use of fricative, glottal, and voiceless phones. Infants and toddlers in the CAS group also demonstrated difficulty with CVCs, had limited syllable shapes, and possible regression of vowel syllable structure. Data corroborate parent reports that infants and toddlers later diagnosed with CAS present differently than do those with typical speech sound skills. Additional study with infants and toddlers later diagnosed with non-CAS speech sound disorder is needed. Readers will: (1) describe current perspectives on volubility of infants and toddlers later diagnosed with CAS; (2) describe current perspectives of the consonant singleton and syllable shape inventories of infants and toddlers later diagnosed with CAS; and (3) discuss the potential differences between the speech sound development of infants and toddlers later diagnosed with CAS and those with typical speech sound skill. Copyright © 2015 Elsevier Inc. All rights reserved.

  13. Mainstreaming gesture based interfaces

    Directory of Open Access Journals (Sweden)

    David Procházka

    2013-01-01

    Full Text Available Gestures are a common way of interaction with mobile devices. They emerged especially with the iPhone production. Gestures in currently used devices are usually based on the original gestures presented by Apple in its iOS (iPhone Operating System. Therefore, there is a wide agreement on the mobile gesture design. In last years, it is possible to see experiments with gesture usage also in the other areas of consumer electronics and computers. The examples can include televisions, large projections etc. These gestures can be marked as spatial or 3D gestures. They are connected with a natural 3D environment rather than with a flat 2D screen. Nevertheless, it is hard to find a comparable design agreement within the spatial gestures. Various projects are based on completely different gesture sets. This situation is confusing for their users and slows down spatial gesture adoption.This paper is focused on the standardization of spatial gestures. The review of projects focused on spatial gesture usage is provided in the first part. The main emphasis is placed on the usability point-of-view. On the basis of our analysis, we argue that the usability is the key issue enabling the wide adoption. The mobile gesture emergence was possible easily because the iPhone gestures were natural. Therefore, it was not necessary to learn them.The design and implementation of our presentation software, which is controlled by gestures, is outlined in the second part of the paper. Furthermore, the usability testing results are provided as well. We have tested our application on a group of users not instructed in the implemented gestures design. These results were compared with the other ones, obtained with our original implementation. The evaluation can be used as the basis for implementation of similar projects.

  14. Electrophysiological and Kinematic Correlates of Communicative Intent in the Planning and Production of Pointing Gestures and Speech.

    Science.gov (United States)

    Peeters, David; Chu, Mingyuan; Holler, Judith; Hagoort, Peter; Özyürek, Aslı

    2015-12-01

    In everyday human communication, we often express our communicative intentions by manually pointing out referents in the material world around us to an addressee, often in tight synchronization with referential speech. This study investigated whether and how the kinematic form of index finger pointing gestures is shaped by the gesturer's communicative intentions and how this is modulated by the presence of concurrently produced speech. Furthermore, we explored the neural mechanisms underpinning the planning of communicative pointing gestures and speech. Two experiments were carried out in which participants pointed at referents for an addressee while the informativeness of their gestures and speech was varied. Kinematic and electrophysiological data were recorded online. It was found that participants prolonged the duration of the stroke and poststroke hold phase of their gesture to be more communicative, in particular when the gesture was carrying the main informational burden in their multimodal utterance. Frontal and P300 effects in the ERPs suggested the importance of intentional and modality-independent attentional mechanisms during the planning phase of informative pointing gestures. These findings contribute to a better understanding of the complex interplay between action, attention, intention, and language in the production of pointing gestures, a communicative act core to human interaction.

  15. High gamma oscillations in medial temporal lobe during overt production of speech and gestures.

    Science.gov (United States)

    Marstaller, Lars; Burianová, Hana; Sowman, Paul F

    2014-01-01

    The study of the production of co-speech gestures (CSGs), i.e., meaningful hand movements that often accompany speech during everyday discourse, provides an important opportunity to investigate the integration of language, action, and memory because of the semantic overlap between gesture movements and speech content. Behavioral studies of CSGs and speech suggest that they have a common base in memory and predict that overt production of both speech and CSGs would be preceded by neural activity related to memory processes. However, to date the neural correlates and timing of CSG production are still largely unknown. In the current study, we addressed these questions with magnetoencephalography and a semantic association paradigm in which participants overtly produced speech or gesture responses that were either meaningfully related to a stimulus or not. Using spectral and beamforming analyses to investigate the neural activity preceding the responses, we found a desynchronization in the beta band (15-25 Hz), which originated 900 ms prior to the onset of speech and was localized to motor and somatosensory regions in the cortex and cerebellum, as well as right inferior frontal gyrus. Beta desynchronization is often seen as an indicator of motor processing and thus reflects motor activity related to the hand movements that gestures add to speech. Furthermore, our results show oscillations in the high gamma band (50-90 Hz), which originated 400 ms prior to speech onset and were localized to the left medial temporal lobe. High gamma oscillations have previously been found to be involved in memory processes and we thus interpret them to be related to contextual association of semantic information in memory. The results of our study show that high gamma oscillations in medial temporal cortex play an important role in the binding of information in human memory during speech and CSG production.

  16. High gamma oscillations in medial temporal lobe during overt production of speech and gestures.

    Directory of Open Access Journals (Sweden)

    Lars Marstaller

    Full Text Available The study of the production of co-speech gestures (CSGs, i.e., meaningful hand movements that often accompany speech during everyday discourse, provides an important opportunity to investigate the integration of language, action, and memory because of the semantic overlap between gesture movements and speech content. Behavioral studies of CSGs and speech suggest that they have a common base in memory and predict that overt production of both speech and CSGs would be preceded by neural activity related to memory processes. However, to date the neural correlates and timing of CSG production are still largely unknown. In the current study, we addressed these questions with magnetoencephalography and a semantic association paradigm in which participants overtly produced speech or gesture responses that were either meaningfully related to a stimulus or not. Using spectral and beamforming analyses to investigate the neural activity preceding the responses, we found a desynchronization in the beta band (15-25 Hz, which originated 900 ms prior to the onset of speech and was localized to motor and somatosensory regions in the cortex and cerebellum, as well as right inferior frontal gyrus. Beta desynchronization is often seen as an indicator of motor processing and thus reflects motor activity related to the hand movements that gestures add to speech. Furthermore, our results show oscillations in the high gamma band (50-90 Hz, which originated 400 ms prior to speech onset and were localized to the left medial temporal lobe. High gamma oscillations have previously been found to be involved in memory processes and we thus interpret them to be related to contextual association of semantic information in memory. The results of our study show that high gamma oscillations in medial temporal cortex play an important role in the binding of information in human memory during speech and CSG production.

  17. “TOT” phenomena: Gesture production in younger and older adults

    OpenAIRE

    Theochaaropoulou, F.; Cocks, N.; Pring, T.; Dipper, L.

    2015-01-01

    This study explored age-related changes in gesture in order to better understand the relationship between gesture and word retrieval from memory. The frequency of gestures during “Tip-of-the-Tongue” (TOT) states highlights this relationship. There is a lack of evidence describing the form and content of iconic gestures arising spontaneously in such TOT states, and a parallel gap addressing age-related variations. In this study, TOT states were induced in 45 participants from two age groups (o...

  18. Mismatch and lexical retrieval gestures are associated with visual information processing, verbal production, and symptomatology in youth at high risk for psychosis.

    Science.gov (United States)

    Millman, Zachary B; Goss, James; Schiffman, Jason; Mejias, Johana; Gupta, Tina; Mittal, Vijay A

    2014-09-01

    Gesture is integrally linked with language and cognitive systems, and recent years have seen a growing attention to these movements in patients with schizophrenia. To date, however, there have been no investigations of gesture in youth at ultra high risk (UHR) for psychosis. Examining gesture in UHR individuals may help to elucidate other widely recognized communicative and cognitive deficits in this population and yield new clues for treatment development. In this study, mismatch (indicating semantic incongruency between the content of speech and a given gesture) and retrieval (used during pauses in speech while a person appears to be searching for a word or idea) gestures were evaluated in 42 UHR individuals and 36 matched healthy controls. Cognitive functions relevant to gesture production (i.e., speed of visual information processing and verbal production) as well as positive and negative symptomatologies were assessed. Although the overall frequency of cases exhibiting these behaviors was low, UHR individuals produced substantially more mismatch and retrieval gestures than controls. The UHR group also exhibited significantly poorer verbal production performance when compared with controls. In the patient group, mismatch gestures were associated with poorer visual processing speed and elevated negative symptoms, while retrieval gestures were associated with higher speed of visual information-processing and verbal production, but not symptoms. Taken together these findings indicate that gesture abnormalities are present in individuals at high risk for psychosis. While mismatch gestures may be closely related to disease processes, retrieval gestures may be employed as a compensatory mechanism. Copyright © 2014 Elsevier B.V. All rights reserved.

  19. Gestural apraxia.

    Science.gov (United States)

    Etcharry-Bouyx, F; Le Gall, D; Jarry, C; Osiurak, F

    Gestural apraxia was first described in 1905 by Hugo Karl Liepmann. While his description is still used, the actual terms are often confusing. The cognitive approach using models proposes thinking of the condition in terms of production and conceptual knowledge. The underlying cognitive processes are still being debated, as are also the optimal ways to assess them. Several neuroimaging studies have revealed the involvement of a left-lateralized frontoparietal network, with preferential activation of the superior parietal lobe, intraparietal sulcus and inferior parietal cortex. The presence of apraxia after a stroke is prevalent, and the incidence is sufficient to propose rehabilitation. Copyright © 2017. Published by Elsevier Masson SAS.

  20. Do Verbal Children with Autism Comprehend Gesture as Readily as Typically Developing Children?

    OpenAIRE

    Dimitrova, N.; Özçalışkan, Ş.; Adamson, L.B.

    2017-01-01

    Gesture comprehension remains understudied, particularly in children with autism spectrum disorder (ASD) who have difficulties in gesture production. Using a novel gesture comprehension task, Study 1 examined how 2- to 4-year-old typically-developing (TD) children comprehend types of gestures and gesture-speech combinations, and showed better comprehension of deictic gestures and reinforcing gesture-speech combinations than iconic/conventional gestures and supplementary gesture-speech combina...

  1. To beg, or not to beg? That is the question: mangabeys modify their production of requesting gestures in response to human's attentional states.

    Directory of Open Access Journals (Sweden)

    Audrey Maille

    Full Text Available BACKGROUND: Although gestural communication is widespread in primates, few studies focused on the cognitive processes underlying gestures produced by monkeys. METHODOLOGY/PRINCIPAL FINDINGS: The present study asked whether red-capped mangabeys (Cercocebus torquatus trained to produce visually based requesting gestures modify their gestural behavior in response to human's attentional states. The experimenter held a food item and displayed five different attentional states that differed on the basis of body, head and gaze orientation; mangabeys had to request food by extending an arm toward the food item (begging gesture. Mangabeys were sensitive, at least to some extent, to the human's attentional state. They reacted to some postural cues of a human recipient: they gestured more and faster when both the body and the head of the experimenter were oriented toward them than when they were oriented away. However, they did not seem to use gaze cues to recognize an attentive human: monkeys begged at similar levels regardless of the experimenter's eyes state. CONCLUSIONS/SIGNIFICANCE: These results indicate that mangabeys lowered their production of begging gestures when these could not be perceived by the human who had to respond to it. This finding provides important evidence that acquired begging gestures of monkeys might be used intentionally.

  2. Gesture en route to words

    DEFF Research Database (Denmark)

    Jensen de López, Kristine M.

    2010-01-01

    This study explores the communicative production of gestrural and vocal modalities by 8 normally developing children in two different cultures (Danish and Zapotec: Mexican indigenous) 16 to 20 months). We analyzed spontaneous production of gestrures and words in children's transition to the two-word...... the children showed an early preference for the gestural or vocal modality. Through Analyzes of two-element combinations of words and/or gestures, we observd a relative increase in cross-modal (gesture-word and two-word) combinations. The results are discussed in terms understanding gestures as a transition...

  3. Gesture Imitation in Schizophrenia

    Science.gov (United States)

    Matthews, Natasha; Gold, Brian J.; Sekuler, Robert; Park, Sohee

    2013-01-01

    Recent evidence suggests that individuals with schizophrenia (SZ) are impaired in their ability to imitate gestures and movements generated by others. This impairment in imitation may be linked to difficulties in generating and maintaining internal representations in working memory (WM). We used a novel quantitative technique to investigate the relationship between WM and imitation ability. SZ outpatients and demographically matched healthy control (HC) participants imitated hand gestures. In Experiment 1, participants imitated single gestures. In Experiment 2, they imitated sequences of 2 gestures, either while viewing the gesture online or after a short delay that forced the use of WM. In Experiment 1, imitation errors were increased in SZ compared with HC. Experiment 2 revealed a significant interaction between imitation ability and WM. SZ produced more errors and required more time to imitate when that imitation depended upon WM compared with HC. Moreover, impaired imitation from WM was significantly correlated with the severity of negative symptoms but not with positive symptoms. In sum, gesture imitation was impaired in schizophrenia, especially when the production of an imitation depended upon WM and when an imitation entailed multiple actions. Such a deficit may have downstream consequences for new skill learning. PMID:21765171

  4. Gesture imitation in schizophrenia.

    Science.gov (United States)

    Matthews, Natasha; Gold, Brian J; Sekuler, Robert; Park, Sohee

    2013-01-01

    Recent evidence suggests that individuals with schizophrenia (SZ) are impaired in their ability to imitate gestures and movements generated by others. This impairment in imitation may be linked to difficulties in generating and maintaining internal representations in working memory (WM). We used a novel quantitative technique to investigate the relationship between WM and imitation ability. SZ outpatients and demographically matched healthy control (HC) participants imitated hand gestures. In Experiment 1, participants imitated single gestures. In Experiment 2, they imitated sequences of 2 gestures, either while viewing the gesture online or after a short delay that forced the use of WM. In Experiment 1, imitation errors were increased in SZ compared with HC. Experiment 2 revealed a significant interaction between imitation ability and WM. SZ produced more errors and required more time to imitate when that imitation depended upon WM compared with HC. Moreover, impaired imitation from WM was significantly correlated with the severity of negative symptoms but not with positive symptoms. In sum, gesture imitation was impaired in schizophrenia, especially when the production of an imitation depended upon WM and when an imitation entailed multiple actions. Such a deficit may have downstream consequences for new skill learning.

  5. Gesturing by Speakers with Aphasia: How Does It Compare?

    Science.gov (United States)

    Mol, Lisette; Krahmer, Emiel; van de Sandt-Koenderman, Mieke

    2013-01-01

    Purpose: To study the independence of gesture and verbal language production. The authors assessed whether gesture can be semantically compensatory in cases of verbal language impairment and whether speakers with aphasia and control participants use similar depiction techniques in gesture. Method: The informativeness of gesture was assessed in 3…

  6. Comprehensibility and neural substrate of communicative gestures in severe aphasia.

    Science.gov (United States)

    Hogrefe, Katharina; Ziegler, Wolfram; Weidinger, Nicole; Goldenberg, Georg

    2017-08-01

    Communicative gestures can compensate incomprehensibility of oral speech in severe aphasia, but the brain damage that causes aphasia may also have an impact on the production of gestures. We compared the comprehensibility of gestural communication of persons with severe aphasia and non-aphasic persons and used voxel based lesion symptom mapping (VLSM) to determine lesion sites that are responsible for poor gestural expression in aphasia. On group level, persons with aphasia conveyed more information via gestures than controls indicating a compensatory use of gestures in persons with severe aphasia. However, individual analysis showed a broad range of gestural comprehensibility. VLSM suggested that poor gestural expression was associated with lesions in anterior temporal and inferior frontal regions. We hypothesize that likely functional correlates of these localizations are selection of and flexible changes between communication channels as well as between different types of gestures and between features of actions and objects that are expressed by gestures. Copyright © 2017 Elsevier Inc. All rights reserved.

  7. A tale of two hands: Children's early gesture use in narrative production predicts later narrative structure in speech

    Science.gov (United States)

    Demir, Özlem Ece; Levine, Susan C.; Goldin-Meadow, Susan

    2014-01-01

    Speakers of all ages spontaneously gesture as they talk. These gestures predict children's milestones in vocabulary and sentence structure. We ask whether gesture serves a similar role in the development of narrative skill. Children were asked to retell a story conveyed in a wordless cartoon at age 5 and then again at 6, 7, and 8. Children's narrative structure in speech improved across these ages. At age 5, many of the children expressed a character's viewpoint in gesture, and these children were more likely to tell better-structured stories at the later ages than children who did not produce character-viewpoint gestures at age 5. In contrast, framing narratives from a character's perspective in speech at age 5 did not predict later narrative structure in speech. Gesture thus continues to act as a harbinger of change even as it assumes new roles in relation to discourse. PMID:25088361

  8. Individual differences in frequency and saliency of speech-accompanying gestures : the role of cognitive abilities and empathy

    OpenAIRE

    Chu, Mingyuan; Meyer, Antje; Foulkes, Lucy; Kita, Sotaro

    2014-01-01

    The present study concerns individual differences in gesture production. We used correlational and multiple regression analyses to examine the relationship between individuals’ cognitive abilities and empathy levels and their gesture frequency and saliency. We chose predictor variables according to experimental evidence of the functions of gesture in speech production and communication. We examined 3 types of gestures: representational gestures, conduit gestures, and palm-revealing gestures. ...

  9. Gestures Specialized for Dialogue.

    Science.gov (United States)

    Bavelas, Janet Beavin; And Others

    1995-01-01

    Explored how hand gestures help interlocutors coordinate their dialogue. Analysis of dyadic conversations and monologues revealed that requirements of dialogue uniquely affect interactive gestures. Gestures aided the speaker's efforts to include the addressee in the conversation. Gestures also demonstrated the importance of social processes in…

  10. The development of gesture

    OpenAIRE

    Tellier, Marion

    2009-01-01

    Human beings gesture everyday while speaking: they move their hands, their heads, their arms; their whole body is involved in communication. But how does it work? How do we produce gestures and in what purpose? How are gestures connected to speech? When do we begin producing gestures and how do they evolve throughout the life span? These are questions gesture researchers have been trying to answer since the second half of the 20th century. This chapter will first define what a gesture is by d...

  11. Gestures maintain spatial imagery.

    Science.gov (United States)

    Wesp, R; Hesse, J; Keutmann, D; Wheaton, K

    2001-01-01

    Recent theories suggest alternatives to the commonly held belief that the sole role of gestures is to communicate meaning directly to listeners. Evidence suggests that gestures may serve a cognitive function for speakers, possibly acting as lexical primes. We observed that participants gestured more often when describing a picture from memory than when the picture was present and that gestures were not influenced by manipulating eye contact of a listener. We argue that spatial imagery serves a short-term memory function during lexical search and that gestures may help maintain spatial images. When spatial imagery is not necessary, as in conditions of direct visual stimulation, reliance on gestures is reduced or eliminated.

  12. The cortical signature of impaired gesturing: Findings from schizophrenia

    Directory of Open Access Journals (Sweden)

    Petra Verena Viher

    2018-01-01

    Full Text Available Schizophrenia is characterized by deficits in gesturing that is important for nonverbal communication. Research in healthy participants and brain-damaged patients revealed a left-lateralized fronto-parieto-temporal network underlying gesture performance. First evidence from structural imaging studies in schizophrenia corroborates these results. However, as of yet, it is unclear if cortical thickness abnormalities contribute to impairments in gesture performance. We hypothesized that patients with deficits in gesture production show cortical thinning in 12 regions of interest (ROIs of a gesture network relevant for gesture performance and recognition. Forty patients with schizophrenia and 41 healthy controls performed hand and finger gestures as either imitation or pantomime. Group differences in cortical thickness between patients with deficits, patients without deficits, and controls were explored using a multivariate analysis of covariance. In addition, the relationship between gesture recognition and cortical thickness was investigated. Patients with deficits in gesture production had reduced cortical thickness in eight ROIs, including the pars opercularis of the inferior frontal gyrus, the superior and inferior parietal lobes, and the superior and middle temporal gyri. Gesture recognition correlated with cortical thickness in fewer, but mainly the same, ROIs within the patient sample. In conclusion, our results show that impaired gesture production and recognition in schizophrenia is associated with cortical thinning in distinct areas of the gesture network.

  13. Single gaze gestures

    DEFF Research Database (Denmark)

    Møllenbach, Emilie; Lilholm, Martin; Gail, Alastair

    2010-01-01

    This paper examines gaze gestures and their applicability as a generic selection method for gaze-only controlled interfaces. The method explored here is the Single Gaze Gesture (SGG), i.e. gestures consisting of a single point-to-point eye movement. Horizontal and vertical, long and short SGGs were...

  14. Individual differences in frequency and saliency of speech-accompanying gestures: the role of cognitive abilities and empathy.

    Science.gov (United States)

    Chu, Mingyuan; Meyer, Antje; Foulkes, Lucy; Kita, Sotaro

    2014-04-01

    The present study concerns individual differences in gesture production. We used correlational and multiple regression analyses to examine the relationship between individuals' cognitive abilities and empathy levels and their gesture frequency and saliency. We chose predictor variables according to experimental evidence of the functions of gesture in speech production and communication. We examined 3 types of gestures: representational gestures, conduit gestures, and palm-revealing gestures. Higher frequency of representational gestures was related to poorer visual and spatial working memory, spatial transformation ability, and conceptualization ability; higher frequency of conduit gestures was related to poorer visual working memory, conceptualization ability, and higher levels of empathy; and higher frequency of palm-revealing gestures was related to higher levels of empathy. The saliency of all gestures was positively related to level of empathy. These results demonstrate that cognitive abilities and empathy levels are related to individual differences in gesture frequency and saliency.

  15. From Gesture to Speech

    Directory of Open Access Journals (Sweden)

    Maurizio Gentilucci

    2012-11-01

    Full Text Available One of the major problems concerning the evolution of human language is to understand how sounds became associated to meaningful gestures. It has been proposed that the circuit controlling gestures and speech evolved from a circuit involved in the control of arm and mouth movements related to ingestion. This circuit contributed to the evolution of spoken language, moving from a system of communication based on arm gestures. The discovery of the mirror neurons has provided strong support for the gestural theory of speech origin because they offer a natural substrate for the embodiment of language and create a direct link between sender and receiver of a message. Behavioural studies indicate that manual gestures are linked to mouth movements used for syllable emission. Grasping with the hand selectively affected movement of inner or outer parts of the mouth according to syllable pronunciation and hand postures, in addition to hand actions, influenced the control of mouth grasp and vocalization. Gestures and words are also related to each other. It was found that when producing communicative gestures (emblems the intention to interact directly with a conspecific was transferred from gestures to words, inducing modification in voice parameters. Transfer effects of the meaning of representational gestures were found on both vocalizations and meaningful words. It has been concluded that the results of our studies suggest the existence of a system relating gesture to vocalization which was precursor of a more general system reciprocally relating gesture to word.

  16. The role of gestures in spatial working memory and speech.

    Science.gov (United States)

    Morsella, Ezequiel; Krauss, Robert M

    2004-01-01

    Co-speech gestures traditionally have been considered communicative, but they may also serve other functions. For example, hand-arm movements seem to facilitate both spatial working memory and speech production. It has been proposed that gestures facilitate speech indirectly by sustaining spatial representations in working memory. Alternatively, gestures may affect speech production directly by activating embodied semantic representations involved in lexical search. Consistent with the first hypothesis, we found participants gestured more when describing visual objects from memory and when describing objects that were difficult to remember and encode verbally. However, they also gestured when describing a visually accessible object, and gesture restriction produced dysfluent speech even when spatial memory was untaxed, suggesting that gestures can directly affect both spatial memory and lexical retrieval.

  17. Co-verbal gestures among speakers with aphasia: Influence of aphasia severity, linguistic and semantic skills, and hemiplegia on gesture employment in oral discourse.

    Science.gov (United States)

    Kong, Anthony Pak-Hin; Law, Sam-Po; Wat, Watson Ka-Chun; Lai, Christy

    2015-01-01

    The use of co-verbal gestures is common in human communication and has been reported to assist word retrieval and to facilitate verbal interactions. This study systematically investigated the impact of aphasia severity, integrity of semantic processing, and hemiplegia on the use of co-verbal gestures, with reference to gesture forms and functions, by 131 normal speakers, 48 individuals with aphasia and their controls. All participants were native Cantonese speakers. It was found that the severity of aphasia and verbal-semantic impairment was associated with significantly more co-verbal gestures. However, there was no relationship between right-sided hemiplegia and gesture employment. Moreover, significantly more gestures were employed by the speakers with aphasia, but about 10% of them did not gesture. Among those who used gestures, content-carrying gestures, including iconic, metaphoric, deictic gestures, and emblems, served the function of enhancing language content and providing information additional to the language content. As for the non-content carrying gestures, beats were used primarily for reinforcing speech prosody or guiding speech flow, while non-identifiable gestures were associated with assisting lexical retrieval or with no specific functions. The above findings would enhance our understanding of the use of various forms of co-verbal gestures in aphasic discourse production and their functions. Speech-language pathologists may also refer to the current annotation system and the results to guide clinical evaluation and remediation of gestures in aphasia. None. Copyright © 2015 Elsevier Inc. All rights reserved.

  18. Co-verbal gestures among speakers with aphasia: Influence of aphasia severity, linguistic and semantic skills, and hemiplegia on gesture employment in oral discourse

    Science.gov (United States)

    Kong, Anthony Pak-Hin; Law, Sam-Po; Wat, Watson Ka-Chun; Lai, Christy

    2015-01-01

    The use of co-verbal gestures is common in human communication and has been reported to assist word retrieval and to facilitate verbal interactions. This study systematically investigated the impact of aphasia severity, integrity of semantic processing, and hemiplegia on the use of co-verbal gestures, with reference to gesture forms and functions, by 131 normal speakers, 48 individuals with aphasia and their controls. All participants were native Cantonese speakers. It was found that the severity of aphasia and verbal-semantic impairment was associated with significantly more co-verbal gestures. However, there was no relationship between right-sided hemiplegia and gesture employment. Moreover, significantly more gestures were employed by the speakers with aphasia, but about 10% of them did not gesture. Among those who used gestures, content-carrying gestures, including iconic, metaphoric, deictic gestures, and emblems, served the function of enhancing language content and providing information additional to the language content. As for the non-content carrying gestures, beats were used primarily for reinforcing speech prosody or guiding speech flow, while non-identifiable gestures were associated with assisting lexical retrieval or with no specific functions. The above findings would enhance our understanding of the use of various forms of co-verbal gestures in aphasic discourse production and their functions. Speech-language pathologists may also refer to the current annotation system and the results to guide clinical evaluation and remediation of gestures in aphasia. PMID:26186256

  19. Mnemonic Effect of Iconic Gesture and Beat Gesture in Adults and Children: Is Meaning in Gesture Important for Memory Recall?

    Science.gov (United States)

    So, Wing Chee; Chen-Hui, Colin Sim; Wei-Shan, Julie Low

    2012-01-01

    Abundant research has shown that encoding meaningful gesture, such as an iconic gesture, enhances memory. This paper asked whether gesture needs to carry meaning to improve memory recall by comparing the mnemonic effect of meaningful (i.e., iconic gestures) and nonmeaningful gestures (i.e., beat gestures). Beat gestures involve simple motoric…

  20. Hand gestures support word learning in patients with hippocampal amnesia.

    Science.gov (United States)

    Hilverman, Caitlin; Cook, Susan Wagner; Duff, Melissa C

    2018-06-01

    Co-speech hand gesture facilitates learning and memory, yet the cognitive and neural mechanisms supporting this remain unclear. One possibility is that motor information in gesture may engage procedural memory representations. Alternatively, iconic information from gesture may contribute to declarative memory representations mediated by the hippocampus. To investigate these alternatives, we examined gesture's effects on word learning in patients with hippocampal damage and declarative memory impairment, with intact procedural memory, and in healthy and in brain-damaged comparison groups. Participants learned novel label-object pairings while producing gesture, observing gesture, or observing without gesture. After a delay, recall and object identification were assessed. Unsurprisingly, amnesic patients were unable to recall the labels at test. However, they correctly identified objects at above chance levels, but only if they produced a gesture at encoding. Comparison groups performed well above chance at both recall and object identification regardless of gesture. These findings suggest that gesture production may support word learning by engaging nondeclarative (procedural) memory. © 2018 Wiley Periodicals, Inc.

  1. A Comparison of Coverbal Gesture Use in Oral Discourse Among Speakers With Fluent and Nonfluent Aphasia

    Science.gov (United States)

    Law, Sam-Po; Chak, Gigi Wan-Chi

    2017-01-01

    Purpose Coverbal gesture use, which is affected by the presence and degree of aphasia, can be culturally specific. The purpose of this study was to compare gesture use among Cantonese-speaking individuals: 23 neurologically healthy speakers, 23 speakers with fluent aphasia, and 21 speakers with nonfluent aphasia. Method Multimedia data of discourse samples from these speakers were extracted from the Cantonese AphasiaBank. Gestures were independently annotated on their forms and functions to determine how gesturing rate and distribution of gestures differed across speaker groups. A multiple regression was conducted to determine the most predictive variable(s) for gesture-to-word ratio. Results Although speakers with nonfluent aphasia gestured most frequently, the rate of gesture use in counterparts with fluent aphasia did not differ significantly from controls. Different patterns of gesture functions in the 3 speaker groups revealed that gesture plays a minor role in lexical retrieval whereas its role in enhancing communication dominates among the speakers with aphasia. The percentages of complete sentences and dysfluency strongly predicted the gesturing rate in aphasia. Conclusions The current results supported the sketch model of language–gesture association. The relationship between gesture production and linguistic abilities and clinical implications for gesture-based language intervention for speakers with aphasia are also discussed. PMID:28609510

  2. Natural gesture interfaces

    Science.gov (United States)

    Starodubtsev, Illya

    2017-09-01

    The paper describes the implementation of the system of interaction with virtual objects based on gestures. The paper describes the common problems of interaction with virtual objects, specific requirements for the interfaces for virtual and augmented reality.

  3. Gesture and Power

    OpenAIRE

    Covington-Ward, Yolanda

    2016-01-01

    In Gesture and Power Yolanda Covington-Ward examines the everyday embodied practices and performances of the BisiKongo people of the lower Congo to show how their gestures, dances, and spirituality are critical in mobilizing social and political action. Conceiving of the body as the center of analysis, a catalyst for social action, and as a conduit for the social construction of reality, Covington-Ward focuses on specific flashpoints in the last ninety years of Congo's troubled history, when ...

  4. Verbal working memory predicts co-speech gesture: evidence from individual differences.

    Science.gov (United States)

    Gillespie, Maureen; James, Ariel N; Federmeier, Kara D; Watson, Duane G

    2014-08-01

    Gesture facilitates language production, but there is debate surrounding its exact role. It has been argued that gestures lighten the load on verbal working memory (VWM; Goldin-Meadow, Nusbaum, Kelly, & Wagner, 2001), but gestures have also been argued to aid in lexical retrieval (Krauss, 1998). In the current study, 50 speakers completed an individual differences battery that included measures of VWM and lexical retrieval. To elicit gesture, each speaker described short cartoon clips immediately after viewing. Measures of lexical retrieval did not predict spontaneous gesture rates, but lower VWM was associated with higher gesture rates, suggesting that gestures can facilitate language production by supporting VWM when resources are taxed. These data also suggest that individual variability in the propensity to gesture is partly linked to cognitive capacities. Copyright © 2014 Elsevier B.V. All rights reserved.

  5. The Role of Conversational Hand Gestures in a Narrative Task

    Science.gov (United States)

    Jacobs, Naomi; Garnham, Alan

    2007-01-01

    The primary functional role of conversational hand gestures in narrative discourse is disputed. A novel experimental technique investigated whether gestures function primarily to aid speech production by the speaker, or communication to the listener. The experiment involved repeated narration of a cartoon story or stories to a single or multiple…

  6. Gestural Communication in Children with Autism Spectrum Disorders during Mother-Child Interaction

    Science.gov (United States)

    Mastrogiuseppe, Marilina; Capirci, Olga; Cuva, Simone; Venuti, Paola

    2015-01-01

    Children with autism spectrum disorders display atypical development of gesture production, and gesture impairment is one of the determining factors of autism spectrum disorder diagnosis. Despite the obvious importance of this issue for children with autism spectrum disorder, the literature on gestures in autism is scarce and contradictory. The…

  7. Baby Sign but Not Spontaneous Gesture Predicts Later Vocabulary in Children with Down Syndrome

    Science.gov (United States)

    Özçaliskan, Seyda; Adamson, Lauren B.; Dimitrova, Nevena; Bailey, Jhonelle; Schmuck, Lauren

    2016-01-01

    Early spontaneous gesture, specifically deictic gesture, predicts subsequent vocabulary development in typically developing (TD) children. Here, we ask whether deictic gesture plays a similar role in predicting later vocabulary size in children with Down Syndrome (DS), who have been shown to have difficulties in speech production, but strengths in…

  8. Gesture and naming therapy for people with severe aphasia: a group study.

    Science.gov (United States)

    Marshall, Jane; Best, Wendy; Cocks, Naomi; Cruice, Madeline; Pring, Tim; Bulcock, Gemma; Creek, Gemma; Eales, Nancy; Mummery, Alice Lockhart; Matthews, Niina; Caute, Anna

    2012-06-01

    In this study, the authors (a) investigated whether a group of people with severe aphasia could learn a vocabulary of pantomime gestures through therapy and (b) compared their learning of gestures with their learning of words. The authors also examined whether gesture therapy cued word production and whether naming therapy cued gestures. Fourteen people with severe aphasia received 15 hr of gesture and naming treatments. Evaluations comprised repeated measures of gesture and word production, comparing treated and untreated items. Baseline measures were stable but improved significantly following therapy. Across the group, improvements in naming were greater than improvements in gesture. This trend was evident in most individuals' results, although 3 participants made better progress in gesture. Gains were item specific, and there was no evidence of cross-modality cueing. Items that received gesture therapy did not improve in naming, and items that received naming therapy did not improve in gesture. Results show that people with severe aphasia can respond to gesture and naming therapies. Given the unequal gains, naming may be a more productive therapy target than gesture for many (although not all) individuals with severe aphasia. The communicative benefits of therapy were not examined but are addressed in a follow-up article.

  9. [Verbal and gestural communication in interpersonal interaction with Alzheimer's disease patients].

    Science.gov (United States)

    Schiaratura, Loris Tamara; Di Pastena, Angela; Askevis-Leherpeux, Françoise; Clément, Sylvain

    2015-03-01

    Communication can be defined as a verbal and non verbal exchange of thoughts and emotions. While verbal communication deficit in Alzheimer's disease is well documented, very little is known about gestural communication, especially in interpersonal situations. This study examines the production of gestures and its relations with verbal aspects of communication. Three patients suffering from moderately severe Alzheimer's disease were compared to three healthy adults. Each one were given a series of pictures and asked to explain which one she preferred and why. The interpersonal interaction was video recorded. Analyses concerned verbal production (quantity and quality) and gestures. Gestures were either non representational (i.e., gestures of small amplitude punctuating speech or accentuating some parts of utterance) or representational (i.e., referring to the object of the speech). Representational gestures were coded as iconic (depicting of concrete aspects), metaphoric (depicting of abstract meaning) or deictic (pointing toward an object). In comparison with healthy participants, patients revealed a decrease in quantity and quality of speech. Nevertheless, their production of gestures was always present. This pattern is in line with the conception that gestures and speech depend on different communicational systems and look inconsistent with the assumption of a parallel dissolution of gesture and speech. Moreover, analyzing the articulation between verbal and gestural dimensions suggests that representational gestures may compensate for speech deficits. It underlines the importance for the role of gestures in maintaining interpersonal communication.

  10. Gesture Modelling for Linguistic Purposes

    CSIR Research Space (South Africa)

    Olivrin, GJ

    2007-05-01

    Full Text Available The study of sign languages attempts to create a coherent model that binds the expressive nature of signs conveyed in gestures to a linguistic framework. Gesture modelling offers an alternative that provides device independence, scalability...

  11. Bimanual Gesture Imitation in Alzheimer's Disease.

    Science.gov (United States)

    Sanin, G Nter; Benke, Thomas

    2017-01-01

    Unimanual gesture production or imitation has often been studied in Alzheimer's disease (AD) during apraxia testing. In the present study, it was hypothesized that bimanual motor tasks may be a sensitive method to detect impairments of motor cognition in AD due to increased demands on the cognitive system. We investigated bimanual, meaningless gesture imitation in 45 AD outpatients, 38 subjects with mild cognitive impairment (MCI), and 50 normal controls (NC) attending a memory clinic. Participants performed neuropsychological background testing and three tasks: the Interlocking Finger Test (ILF), Imitation of Alternating Hand Movements (AHM), and Bimanual Rhythm Tapping (BRT). The tasks were short and easy to administer. Inter-rater reliability was high across all three tests. AD patients performed significantly poorer than NC and MCI participants; a deficit to imitate bimanual gestures was rarely found in MCI and NC participants. Sensitivity to detect AD ranged from 0.5 and 0.7, specificity beyond 0.9. ROC analyses revealed good diagnostic accuracy (0.77 to 0.92). Impairment to imitate bimanual gestures was mainly predicted by diagnosis and disease severity. Our findings suggest that an impairment to imitate bimanual, meaningless gestures is a valid disease marker of mild to moderate AD and can easily be assessed in memory clinic settings. Based on our preliminary findings, it appears to be a separate impairment which can be distinguished from other cognitive deficits.

  12. A unique gesture of sharing

    International Nuclear Information System (INIS)

    Mustafa, T.

    1985-01-01

    The Atoms for Peace program was a unique gesture of sharing on the part of the leading industrialized nation, and has very few parallels in modern history. The author says one of the major advantages of the program for developing nations was the much needed stimulation of their indigenous science and technology efforts and the awakening of their governments to the multifaceted benefits of atomic energy. The author discusses how the program benefited Pakistan in the production of electrical energy and in the application of nuclear techniques in the fields of agriculture and medicine, which help to alleviate hunger and combat disease

  13. Gesture in the Developing Brain

    Science.gov (United States)

    Dick, Anthony Steven; Goldin-Meadow, Susan; Solodkin, Ana; Small, Steven L.

    2012-01-01

    Speakers convey meaning not only through words, but also through gestures. Although children are exposed to co-speech gestures from birth, we do not know how the developing brain comes to connect meaning conveyed in gesture with speech. We used functional magnetic resonance imaging (fMRI) to address this question and scanned 8- to 11-year-old…

  14. Neural correlates of gesture processing across human development.

    Science.gov (United States)

    Wakefield, Elizabeth M; James, Thomas W; James, Karin H

    2013-01-01

    Co-speech gesture facilitates learning to a greater degree in children than in adults, suggesting that the mechanisms underlying the processing of co-speech gesture differ as a function of development. We suggest that this may be partially due to children's lack of experience producing gesture, leading to differences in the recruitment of sensorimotor networks when comparing adults to children. Here, we investigated the neural substrates of gesture processing in a cross-sectional sample of 5-, 7.5-, and 10-year-old children and adults and focused on relative recruitment of a sensorimotor system that included the precentral gyrus (PCG) and the posterior middle temporal gyrus (pMTG). Children and adults were presented with videos in which communication occurred through different combinations of speech and gesture during a functional magnetic resonance imaging (fMRI) session. Results demonstrated that the PCG and pMTG were recruited to different extents in the two populations. We interpret these novel findings as supporting the idea that gesture perception (pMTG) is affected by a history of gesture production (PCG), revealing the importance of considering gesture processing as a sensorimotor process.

  15. Individual Differences in Frequency and Saliency of Speech-Accompanying Gestures: The Role of Cognitive Abilities and Empathy

    NARCIS (Netherlands)

    Chu, M.; Meyer, A.S.; Foulkes, L.; Kita, S.

    2014-01-01

    The present study concerns individual differences in gesture production. We used correlational and multiple regression analyses to examine the relationship between individuals’ cognitive abilities and empathy levels and their gesture frequency and saliency. We chose predictor variables according to

  16. Eye-based head gestures

    DEFF Research Database (Denmark)

    Mardanbegi, Diako; Witzner Hansen, Dan; Pederson, Thomas

    2012-01-01

    A novel method for video-based head gesture recognition using eye information by an eye tracker has been proposed. The method uses a combination of gaze and eye movement to infer head gestures. Compared to other gesture-based methods a major advantage of the method is that the user keeps the gaze...... mobile phone screens. The user study shows that the method detects a set of defined gestures reliably.......A novel method for video-based head gesture recognition using eye information by an eye tracker has been proposed. The method uses a combination of gaze and eye movement to infer head gestures. Compared to other gesture-based methods a major advantage of the method is that the user keeps the gaze...

  17. Gestures Enhance Foreign Language Learning

    Directory of Open Access Journals (Sweden)

    Manuela Macedonia

    2012-11-01

    Full Text Available Language and gesture are highly interdependent systems that reciprocally influence each other. For example, performing a gesture when learning a word or a phrase enhances its retrieval compared to pure verbal learning. Although the enhancing effects of co-speech gestures on memory are known to be robust, the underlying neural mechanisms are still unclear. Here, we summarize the results of behavioral and neuroscientific studies. They indicate that the neural representation of words consists of complex multimodal networks connecting perception and motor acts that occur during learning. In this context, gestures can reinforce the sensorimotor representation of a word or a phrase, making it resistant to decay. Also, gestures can favor embodiment of abstract words by creating it from scratch. Thus, we propose the use of gesture as a facilitating educational tool that integrates body and mind.

  18. Gestures and multimodal input

    OpenAIRE

    Keates, Simeon; Robinson, Peter

    1999-01-01

    For users with motion impairments, the standard keyboard and mouse arrangement for computer access often presents problems. Other approaches have to be adopted to overcome this. In this paper, we will describe the development of a prototype multimodal input system based on two gestural input channels. Results from extensive user trials of this system are presented. These trials showed that the physical and cognitive loads on the user can quickly become excessive and detrimental to the interac...

  19. Pantomimic gestures for human-robot interaction

    CSIR Research Space (South Africa)

    Burke, Michael G

    2015-10-01

    Full Text Available -1 IEEE TRANSACTIONS ON ROBOTICS 1 Pantomimic Gestures for Human-Robot Interaction Michael Burke, Student Member, IEEE, and Joan Lasenby Abstract This work introduces a pantomimic gesture interface, which classifies human hand gestures using...

  20. Gesturing Makes Memories that Last

    Science.gov (United States)

    Cook, Susan Wagner; Yip, Terina KuangYi; Goldin-Meadow, Susan

    2010-01-01

    When people are asked to perform actions, they remember those actions better than if they are asked to talk about the same actions. But when people talk, they often gesture with their hands, thus adding an action component to talking. The question we asked in this study was whether producing gesture along with speech makes the information encoded…

  1. The development of co-speech gesture and its semantic integration with speech in 6- to 12-year-old children with autism spectrum disorders.

    Science.gov (United States)

    So, Wing-Chee; Wong, Miranda Kit-Yi; Lui, Ming; Yip, Virginia

    2015-11-01

    Previous work leaves open the question of whether children with autism spectrum disorders aged 6-12 years have delay in producing gestures compared to their typically developing peers. This study examined gestural production among school-aged children in a naturalistic context and how their gestures are semantically related to the accompanying speech. Delay in gestural production was found in children with autism spectrum disorders through their middle to late childhood. Compared to their typically developing counterparts, children with autism spectrum disorders gestured less often and used fewer types of gestures, in particular markers, which carry culture-specific meaning. Typically developing children's gestural production was related to language and cognitive skills, but among children with autism spectrum disorders, gestural production was more strongly related to the severity of socio-communicative impairment. Gesture impairment also included the failure to integrate speech with gesture: in particular, supplementary gestures are absent in children with autism spectrum disorders. The findings extend our understanding of gestural production in school-aged children with autism spectrum disorders during spontaneous interaction. The results can help guide new therapies for gestural production for children with autism spectrum disorders in middle and late childhood. © The Author(s) 2014.

  2. Designing Gestural Interfaces Touchscreens and Interactive Devices

    CERN Document Server

    Saffer, Dan

    2008-01-01

    If you want to get started in new era of interaction design, this is the reference you need. Packed with informative illustrations and photos, Designing Gestural Interfaces provides you with essential information about kinesiology, sensors, ergonomics, physical computing, touchscreen technology, and new interface patterns -- information you need to augment your existing skills in traditional" websites, software, or product development. This book will help you enter this new world of possibilities."

  3. Gesture-Based Controls for Robots: Overview and Implications for Use by Soldiers

    Science.gov (United States)

    2016-07-01

    application is that of the Kinect system, where camera-based interpretation of user body posture and movements serves to control videogame features...modalities will benefit from respective advantages. The combination of deictic gestures to support human-human interactions has been well...used in spontaneous gesture production were to clarify speech utterances. Studies have shown demonstrable benefits from use of gestures to support

  4. Gestures and Insight in Advanced Mathematical Thinking

    Science.gov (United States)

    Yoon, Caroline; Thomas, Michael O. J.; Dreyfus, Tommy

    2011-01-01

    What role do gestures play in advanced mathematical thinking? We argue that the role of gestures goes beyond merely communicating thought and supporting understanding--in some cases, gestures can help generate new mathematical insights. Gestures feature prominently in a case study of two participants working on a sequence of calculus activities.…

  5. A cross-species study of gesture and its role in symbolic development: Implications for the gestural theory of language evolution

    Directory of Open Access Journals (Sweden)

    Kristen eGillespie-Lynch

    2013-06-01

    Full Text Available Using a naturalistic video database, we examined whether gestures scaffolded the symbolic development of a language-enculturated chimpanzee, a language-enculturated bonobo, and a human child during the second year of life. These three species constitute a complete clade: species possessing a common immediate ancestor. A basic finding was the functional and formal similarity of many gestures between chimpanzee, bonobo, and human child. The child’s symbols were spoken words; the apes’ symbols were lexigrams, noniconic visual signifiers. A developmental pattern in which gestural representation of a referent preceded symbolic representation of the same referent appeared in all three species (but was statistically significant only for the child. Nonetheless, across species, the ratio of symbol to gesture increased significantly with age. But even though their symbol production increased, the apes continued to communicate more frequently by gesture than by symbol. In contrast, by15-18 months of age, the child used symbols more frequently than gestures. This ontogenetic sequence from gesture to symbol, present across the clade but more pronounced in child than ape, provides support for the role of gesture in language evolution. In all three species, the overwhelming majority of gestures were communicative (paired with eye-contact, vocalization, and/or persistence. However, vocalization was rare for the apes, but accompanied the majority of the child’s communicative gestures. This finding suggests the co-evolution of speech and gesture after the evolutionary divergence of the hominid line. Multimodal expressions of communicative intent (e.g., vocalization plus persistence were normative for the child, but less common for the apes. This finding suggests that multimodal expression of communicative intent was also strengthened after hominids diverged from apes.

  6. Machine Learning of Musical Gestures

    OpenAIRE

    Caramiaux, Baptiste; Tanaka, Atau

    2013-01-01

    We present an overview of machine learning (ML) techniques and theirapplication in interactive music and new digital instruments design. We firstgive to the non-specialist reader an introduction to two ML tasks,classification and regression, that are particularly relevant for gesturalinteraction. We then present a review of the literature in current NIMEresearch that uses ML in musical gesture analysis and gestural sound control.We describe the ways in which machine learning is useful for cre...

  7. Kazakh Traditional Dance Gesture Recognition

    Science.gov (United States)

    Nussipbekov, A. K.; Amirgaliyev, E. N.; Hahn, Minsoo

    2014-04-01

    Full body gesture recognition is an important and interdisciplinary research field which is widely used in many application spheres including dance gesture recognition. The rapid growth of technology in recent years brought a lot of contribution in this domain. However it is still challenging task. In this paper we implement Kazakh traditional dance gesture recognition. We use Microsoft Kinect camera to obtain human skeleton and depth information. Then we apply tree-structured Bayesian network and Expectation Maximization algorithm with K-means clustering to calculate conditional linear Gaussians for classifying poses. And finally we use Hidden Markov Model to detect dance gestures. Our main contribution is that we extend Kinect skeleton by adding headwear as a new skeleton joint which is calculated from depth image. This novelty allows us to significantly improve the accuracy of head gesture recognition of a dancer which in turn plays considerable role in whole body gesture recognition. Experimental results show the efficiency of the proposed method and that its performance is comparable to the state-of-the-art system performances.

  8. The importance of considering gestures in the study of current spoken Yucatec Maya.

    Directory of Open Access Journals (Sweden)

    Olivier Le Guen

    2018-02-01

    Full Text Available For centuries, linguistic description has been somehow limited because it was not possible to record audio and video. For this reason, the intrinsic multimodal nature of human language has been left out, putting aside various types of information both prosodic and visual. This work analyzes the ways in which gestures complement speech, taking into account several levels of analysis: pragmatic, semantic and syntactic; but also how some gestures can be considered linguistic signs. In order to exemplify the argumentation, I will consider the Yucatec Maya language using examples of spontaneous productions. Although certain processes presented in this work are specific to Yucatec Maya, most can be found in various languages. This paper first presents a definition of language, speech and gestures, and how one can study the way in which speech and gestures are integrated in a composite utterance. Subsequently, I analyze examples of different types of gestures in various areas of communication in Yucatec Maya, such as deictic gestures, the use of expressive gestures, metaphors and the integration of gestures at the pragmatic level. Finally, I explain how gestures can become linguistic signs in Yucatec Maya.

  9. Effects of hand gestures on auditory learning of second-language vowel length contrasts.

    Science.gov (United States)

    Hirata, Yukari; Kelly, Spencer D; Huang, Jessica; Manansala, Michael

    2014-12-01

    Research has shown that hand gestures affect comprehension and production of speech at semantic, syntactic, and pragmatic levels for both native language and second language (L2). This study investigated a relatively less explored question: Do hand gestures influence auditory learning of an L2 at the segmental phonology level? To examine auditory learning of phonemic vowel length contrasts in Japanese, 88 native English-speaking participants took an auditory test before and after one of the following 4 types of training in which they (a) observed an instructor in a video speaking Japanese words while she made syllabic-rhythm hand gesture, (b) produced this gesture with the instructor, (c) observed the instructor speaking those words and her moraic-rhythm hand gesture, or (d) produced the moraic-rhythm gesture with the instructor. All of the training types yielded similar auditory improvement in identifying vowel length contrast. However, observing the syllabic-rhythm hand gesture yielded the most balanced improvement between word-initial and word-final vowels and between slow and fast speaking rates. The overall effect of hand gesture on learning of segmental phonology is limited. Implications for theories of hand gesture are discussed in terms of the role it plays at different linguistic levels.

  10. The use of hand gestures to communicate about nonpresent objects in mind among children with autism spectrum disorder.

    Science.gov (United States)

    So, Wing-Chee; Lui, Ming; Wong, Tze-Kiu; Sit, Long-Tin

    2015-04-01

    The current study examined whether children with autism spectrum disorder (ASD), in comparison with typically developing children, perceive and produce gestures to identify nonpresent objects (i.e., referent-identifying gestures), which is crucial for communicating ideas in a discourse. An experimenter described the uses of daily-life objects to 6- to 12-year-old children both orally and with gestures. The children were then asked to describe how they performed daily activities using those objects. All children gestured. A gesture identified a nonpresent referent if it was produced in the same location that had previously been established by the experimenter. Children with ASD gestured at the specific locations less often than typically developing children. Verbal and spatial memory were positively correlated with the ability to produce referent-identifying gestures for all children. However, the positive correlation between Raven's Children Progressive Matrices score and the production of referent-identifying gestures was found only in children with ASD. Children with ASD might be less able to perceive and produce referent-identifying gestures and may rely more heavily on visual-spatial skills in producing referent-identifying gestures. The results have clinical implications for designing an intervention program to enhance the ability of children with ASD to communicate about nonpresent objects with gestures.

  11. Using Robot Animation to Promote Gestural Skills in Children with Autism Spectrum Disorders

    Science.gov (United States)

    So, W.-C.; Wong, M. K.-Y.; Cabibihan, J.-J.; Lam, C. K.-Y.; Chan, R. Y.-Y.; Qian, H.-H.

    2016-01-01

    School-aged children with autism spectrum disorders (ASDs) have delayed gestural development, in comparison with age-matched typically developing children. In this study, an intervention program taught children with low-functioning ASD gestural comprehension and production using video modelling (VM) by a computer-generated robot animation. Six to…

  12. The Development of Co-Speech Gesture and Its Semantic Integration with Speech in 6- to 12-Year-Old Children with Autism Spectrum Disorders

    Science.gov (United States)

    So, Wing-Chee; Wong, Miranda Kit-Yi; Lui, Ming; Yip, Virginia

    2015-01-01

    Previous work leaves open the question of whether children with autism spectrum disorders aged 6-12?years have delay in producing gestures compared to their typically developing peers. This study examined gestural production among school-aged children in a naturalistic context and how their gestures are semantically related to the accompanying…

  13. Non Audio-Video gesture recognition system

    DEFF Research Database (Denmark)

    Craciunescu, Razvan; Mihovska, Albena Dimitrova; Kyriazakos, Sofoklis

    2016-01-01

    Gesture recognition is a topic in computer science and language technology with the goal of interpreting human gestures via mathematical algorithms. Gestures can originate from any bodily motion or state but commonly originate from the face or hand. Current research focus includes on the emotion...... recognition from the face and hand gesture recognition. Gesture recognition enables humans to communicate with the machine and interact naturally without any mechanical devices. This paper investigates the possibility to use non-audio/video sensors in order to design a low-cost gesture recognition device...

  14. Pantomimes are special gestures which rely on working memory.

    Science.gov (United States)

    Bartolo, A; Cubelli, R; Della Sala, S; Drei, S

    2003-12-01

    The case of a patient is reported who presented consistently with overt deficits in producing pantomimes in the absence of any other deficits in producing meaningful gestures. This pattern of spared and impaired abilities is difficult to reconcile with the current layout of cognitive models for praxis. This patient also showed clear impairment in a dual-task paradigm, a test taxing the co-ordination aspect of working memory, though performed normally in a series of other neuropsychological measures assessing language, visuo-spatial functions, reasoning function, and executive function. A specific working memory impairment associated with a deficit of pantomiming in the absence of any other disorders in the production of meaningful gestures suggested a way to modify the model to account for the data. Pantomimes are a particular category of gestures, meaningful, yet novel. We posit that by their very nature they call for the intervention of a mechanism to integrate and synthesise perceptual inputs together with information made available from the action semantics (knowledge about objects and functions) and the output lexicon (stored procedural programmes). This processing stage conceived as a temporary workspace where gesture information is actively manipulated, would generate new motor programmes to carry out pantomimes. The model of gesture production is refined to include this workspace.

  15. Crossover learning of gestures in two ideomotor apraxia patients: A single case experimental design study.

    Science.gov (United States)

    Shimizu, Daisuke; Tanemura, Rumi

    2017-06-01

    Crossover learning may aid rehabilitation in patients with neurological disorders. Ideomotor apraxia (IMA) is a common sequela of left-brain damage that comprises a deficit in the ability to perform gestures to verbal commands or by imitation. This study elucidated whether crossover learning occurred in two post-stroke IMA patients without motor paralysis after gesture training approximately 2 months after stroke onset. We quantitatively analysed the therapeutic intervention history and investigated whether revised action occurred during gesture production. Treatment intervention was to examine how to influence improvement and generalisation of the ability to produce the gesture. This study used an alternating treatments single-subject design, and the intervention method was errorless learning. Results indicated crossover learning in both patients. Qualitative analysis indicated that revised action occurred during the gesture-production process in one patient and that there were two types of post-revised action gestures: correct and incorrect gestures. We also discovered that even when a comparably short time had elapsed since stroke onset, generalisation was difficult. Information transfer between the left and right hemispheres of the brain via commissural fibres is important in crossover learning. In conclusion, improvements in gesture-production skill should be made with reference to the left cerebral hemisphere disconnection hypothesis.

  16. Contribution of the pre-SMA to the production of words and non-speech oral motor gestures, as revealed by repetitive transcranial magnetic stimulation (rTMS).

    Science.gov (United States)

    Tremblay, Pascale; Gracco, Vincent L

    2009-05-01

    An emerging theoretical perspective, largely based on neuroimaging studies, suggests that the pre-SMA is involved in planning cognitive aspects of motor behavior and language, such as linguistic and non-linguistic response selection. Neuroimaging studies, however, cannot indicate whether a brain region is equally important to all tasks in which it is activated. In the present study, we tested the hypothesis that the pre-SMA is an important component of response selection, using an interference technique. High frequency repetitive TMS (10 Hz) was used to interfere with the functioning of the pre-SMA during tasks requiring selection of words and oral gestures under different selection modes (forced, volitional) and attention levels (high attention, low attention). Results show that TMS applied to the pre-SMA interferes selectively with the volitional selection condition, resulting in longer RTs. The low- and high-attention forced selection conditions were unaffected by TMS, demonstrating that the pre-SMA is sensitive to selection mode but not attentional demands. TMS similarly affected the volitional selection of words and oral gestures, reflecting the response-independent nature of the pre-SMA contribution to response selection. The implications of these results are discussed.

  17. Ethics, Gesture and the Western

    Directory of Open Access Journals (Sweden)

    Michael Minden

    2017-06-01

    Full Text Available This paper relates the Western Movie to Agamben’s implied gestural zone between intention and act. Film is important in the realisation of this zone because it was the first means of representation to capture the body in movement. The Western movie explores the space of ethical indistinction between the acts of individual fighters and the establishment of a rule of law, or putting this another way, between violence and justice. Two classic examples of an archetypal Western plot (Shane, 1953 and Unforgiven, 1991 that particularly embodies this are cited. In both a gunfighter who has forsworn violence at the start is led by the circumstances of the plot to take it up once more at the conclusion. In these terms all the gestures contained between these beginning- and end-points are analysable as an ethics of gesture because, captured as gestures, they occupy the human space between abstraction and action, suspended between them, and reducible to neither.  David Foster Wallace's definition of this narrative arc in Infinite Jest (and embodied in it is adduced in order to suggest a parallel between Agamben's notion of an ethics of gesture, and an ethics of genre.

  18. How do gestures influence thinking and speaking? The gesture-for-conceptualization hypothesis

    OpenAIRE

    Kita, Sotaro; Alibali, M. W.; Chu, Mingyuan

    2017-01-01

    People spontaneously produce gestures during speaking and thinking. The authors focus here on gestures that depict or indicate information related to the contents of concurrent speech or thought (i.e., representational gestures). Previous research indicates that such gestures have not only communicative functions, but also self-oriented cognitive functions. In this article, the authors propose a new theoretical framework, the gesture-for-conceptualization hypothesis, which explains the self-o...

  19. An in-situ trainable gesture classifier

    NARCIS (Netherlands)

    van Diepen, A.; Cox, M.G.H.; de Vries, A.; Duivesteijn, W.; Pechenizkiy, M.; Fletcher, G.H.L.

    2017-01-01

    Gesture recognition, i.e., the recognition of pre-defined gestures by arm or hand movements, enables a natural extension of the way we currently interact with devices (Horsley, 2016). Commercially available gesture recognition systems are usually pre-trained: the developers specify a set of

  20. Gesture Activated Mobile Edutainment (GAME)

    DEFF Research Database (Denmark)

    Rehm, Matthias; Leichtenstern, Karin; Plomer, Joerg

    2010-01-01

    An approach to intercultural training of nonverbal behavior is presented that draws from research on role-plays with virtual agents and ideas from situated learning. To this end, a mobile serious game is realized where the user acquires knowledge about German emblematic gestures and tries them out...... in role-plays with virtual agents. Gesture performance is evaluated making use of build-in acceleration sensors of smart phones. After an account of the theoretical background covering diverse areas like virtual agents, situated learning and intercultural training, the paper presents the GAME approach...... along with details on the gesture recognition and content authoring. By its experience-based role plays with virtual characters, GAME brings together ideas from situated learning and intercultural training in an integrated approach and paves the way for new m-learning concepts....

  1. Does brain injury impair speech and gesture differently?

    Directory of Open Access Journals (Sweden)

    Tilbe Göksun

    2016-09-01

    Full Text Available People often use spontaneous gestures when talking about space, such as when giving directions. In a recent study from our lab, we examined whether focal brain-injured individuals’ naming motion event components of manner and path (represented in English by verbs and prepositions, respectively are impaired selectively, and whether gestures compensate for impairment in speech. Left or right hemisphere damaged patients and elderly control participants were asked to describe motion events (e.g., walking around depicted in brief videos. Results suggest that producing verbs and prepositions can be separately impaired in the left hemisphere and gesture production compensates for naming impairments when damage involves specific areas in the left temporal cortex.

  2. Evaluación agronómica de líneas avanzadas de fríjol voluble Phaseolus vulgaris L. en Paipa, Boyacá

    Directory of Open Access Journals (Sweden)

    Diana Katherine Ríos

    2014-06-01

    Full Text Available En Boyacá la oferta de germoplasma de frijol mejorado es baja. El objetivo de este estudio fue evaluar el comportamiento agronómico de 18 líneas avanzadas volubles de frijol, procedentes del CIAT. Se utilizó el diseño bloques completos al azar, con tres repeticiones, incluyendo Bola Roja, Bolívar y Cargamanto como testigos. Se evaluó días a floración, vigor de crecimiento, días a madurez, incidencia de enfermedades, número de vainas/planta, número de granos/vaina, peso de 100 granos y rendimiento. Los genotipos MAC27, MAC13, MBC26, MBC28 y MBC37 no se diferenciaron del Cargamanto pero sí mostraron ser más precoces que Bola Roja y Bolívar. En días a madurez MAC27 y MBC39 fueron más tempranos que los testigos. Todos los genotipos se comportaron como resistentes a enfermedades, excepto MBC30 que resultó susceptible a Uromyces phaseoli y MBC33 a Phoma exigua. En número de vainas por planta, número de granos por vaina, peso de 100 semillas y rendimiento se destacaron MAC27 y LAS399, sin diferencia significativa frente a Cargamanto pero si superando a Bola Roja y Bolívar. Pese al efecto de las heladas en la fase final del ciclo, LAS399, MAC27 y Cargamanto produjeron rendimientos de 1300, 1038 y 950 kg/ha, respectivamente, mostrando las ventajas de su precocidad, en comparación a Bola Roja y Bolívar que son tardíos. Se concluye que LAS399 y MAC27, por su alta precocidad y resistencia a patógenos, son las líneas más promisorias para continuar la evaluación respecto a la posibilidad de ampliar la oferta de germoplasma de frijol voluble en Boyacá.

  3. Gesture, sign, and language: The coming of age of sign language and gesture studies.

    Science.gov (United States)

    Goldin-Meadow, Susan; Brentari, Diane

    2017-01-01

    How does sign language compare with gesture, on the one hand, and spoken language on the other? Sign was once viewed as nothing more than a system of pictorial gestures without linguistic structure. More recently, researchers have argued that sign is no different from spoken language, with all of the same linguistic structures. The pendulum is currently swinging back toward the view that sign is gestural, or at least has gestural components. The goal of this review is to elucidate the relationships among sign language, gesture, and spoken language. We do so by taking a close look not only at how sign has been studied over the past 50 years, but also at how the spontaneous gestures that accompany speech have been studied. We conclude that signers gesture just as speakers do. Both produce imagistic gestures along with more categorical signs or words. Because at present it is difficult to tell where sign stops and gesture begins, we suggest that sign should not be compared with speech alone but should be compared with speech-plus-gesture. Although it might be easier (and, in some cases, preferable) to blur the distinction between sign and gesture, we argue that distinguishing between sign (or speech) and gesture is essential to predict certain types of learning and allows us to understand the conditions under which gesture takes on properties of sign, and speech takes on properties of gesture. We end by calling for new technology that may help us better calibrate the borders between sign and gesture.

  4. Signers and co-speech gesturers adopt similar strategies for portraying viewpoint in narratives.

    Science.gov (United States)

    Quinto-Pozos, David; Parrill, Fey

    2015-01-01

    Gestural viewpoint research suggests that several dimensions determine which perspective a narrator takes, including properties of the event described. Events can evoke gestures from the point of view of a character (CVPT), an observer (OVPT), or both perspectives. CVPT and OVPT gestures have been compared to constructed action (CA) and classifiers (CL) in signed languages. We ask how CA and CL, as represented in ASL productions, compare to previous results for CVPT and OVPT from English-speaking co-speech gesturers. Ten ASL signers described cartoon stimuli from Parrill (2010). Events shown by Parrill to elicit a particular gestural strategy (CVPT, OVPT, both) were coded for signers' instances of CA and CL. CA was divided into three categories: CA-torso, CA-affect, and CA-handling. Signers used CA-handling the most when gesturers used CVPT exclusively. Additionally, signers used CL the most when gesturers used OVPT exclusively and CL the least when gesturers used CVPT exclusively. Copyright © 2014 Cognitive Science Society, Inc.

  5. Maternal Mental State Talk and Infants' Early Gestural Communication

    Science.gov (United States)

    Slaughter, Virginia; Peterson, Candida C.; Carpenter, Malinda

    2009-01-01

    Twenty-four infants were tested monthly for the production of imperative and declarative gestures between 0 ; 9 and 1 ; 3 and concurrent mother-infant free-play sessions were conducted at 0 ; 9, 1 ; 0 and 1 ; 3 (Carpenter, Nagell & Tomasello, 1998). Free-play transcripts were subsequently coded for maternal talk about mental states. Results…

  6. Investigation of the Reationship between Hand Gestures and Speech in Adults Who Stutter

    Directory of Open Access Journals (Sweden)

    Ali Barikrou

    2008-12-01

    Full Text Available Objective: Gestures of the hands and arms have long been observed to accompany speech in spontaneous conversation. However, the way in which these two modes of expression are related in production is not yet fully understood. So, the present study aims to investigate the spontaneous gestures that accompany speech in adults who stutter in comparison to fluent controls.  Materials & Methods: In this cross-sectional and comparative research, ten adults who stutter were selected randomly from speech and language pathology clinics and compared with ten healthy persons as control group who were matched with stutterers according to sex, age and education. The cartoon story-retelling task used to elicit spontaneous gestures that accompany speech. Participants were asked to watch the animation carefully and then retell the storyline in as much detail as possible to a listener sitting across from him or her and his or her narration was video recorded simultaneously. Then recorded utterances and gestures were analyzed. The statistical methods such as Kolmogorov- Smirnov and Independent t-test were used for data analyzing. Results: The results indicated that stutterers in comparison to controls in average use fewer iconic gestures in their narration (P=0.005. Also, stutterers in comparison to controls in average use fewer iconic gestures per each utterance and word (P=0.019. Furthermore, the execution of gesture production during moments of dysfluency revealed that more than 70% of the gestures produced with stuttering were frozen or abandoned at the moment of dysfluency. Conclusion: It seems gesture and speech have such an intricate and deep association that show similar frequency and timing patterns and move completely parallel to each other in such a way that deficit in speech results in deficiency in hand gesture.

  7. Thirty years of great ape gestures.

    Science.gov (United States)

    Tomasello, Michael; Call, Josep

    2018-02-21

    We and our colleagues have been doing studies of great ape gestural communication for more than 30 years. Here we attempt to spell out what we have learned. Some aspects of the process have been reliably established by multiple researchers, for example, its intentional structure and its sensitivity to the attentional state of the recipient. Other aspects are more controversial. We argue here that it is a mistake to assimilate great ape gestures to the species-typical displays of other mammals by claiming that they are fixed action patterns, as there are many differences, including the use of attention-getters. It is also a mistake, we argue, to assimilate great ape gestures to human gestures by claiming that they are used referentially and declaratively in a human-like manner, as apes' "pointing" gesture has many limitations and they do not gesture iconically. Great ape gestures constitute a unique form of primate communication with their own unique qualities.

  8. Gesturing Gives Children New Ideas About Math

    Science.gov (United States)

    Goldin-Meadow, Susan; Cook, Susan Wagner; Mitchell, Zachary A.

    2009-01-01

    How does gesturing help children learn? Gesturing might encourage children to extract meaning implicit in their hand movements. If so, children should be sensitive to the particular movements they produce and learn accordingly. Alternatively, all that may matter is that children move their hands. If so, they should learn regardless of which movements they produce. To investigate these alternatives, we manipulated gesturing during a math lesson. We found that children required to produce correct gestures learned more than children required to produce partially correct gestures, who learned more than children required to produce no gestures. This effect was mediated by whether children took information conveyed solely in their gestures and added it to their speech. The findings suggest that body movements are involved not only in processing old ideas, but also in creating new ones. We may be able to lay foundations for new knowledge simply by telling learners how to move their hands. PMID:19222810

  9. How do gestures influence thinking and speaking? The gesture-for-conceptualization hypothesis.

    Science.gov (United States)

    Kita, Sotaro; Alibali, Martha W; Chu, Mingyuan

    2017-04-01

    People spontaneously produce gestures during speaking and thinking. The authors focus here on gestures that depict or indicate information related to the contents of concurrent speech or thought (i.e., representational gestures). Previous research indicates that such gestures have not only communicative functions, but also self-oriented cognitive functions. In this article, the authors propose a new theoretical framework, the gesture-for-conceptualization hypothesis, which explains the self-oriented functions of representational gestures. According to this framework, representational gestures affect cognitive processes in 4 main ways: gestures activate, manipulate, package, and explore spatio-motoric information for speaking and thinking. These four functions are shaped by gesture's ability to schematize information, that is, to focus on a small subset of available information that is potentially relevant to the task at hand. The framework is based on the assumption that gestures are generated from the same system that generates practical actions, such as object manipulation; however, gestures are distinct from practical actions in that they represent information. The framework provides a novel, parsimonious, and comprehensive account of the self-oriented functions of gestures. The authors discuss how the framework accounts for gestures that depict abstract or metaphoric content, and they consider implications for the relations between self-oriented and communicative functions of gestures. (PsycINFO Database Record (c) 2017 APA, all rights reserved).

  10. Gesture Interaction at a Distance

    NARCIS (Netherlands)

    Fikkert, F.W.

    2010-01-01

    The aim of this work is to explore, from a perspective of human behavior, which gestures are suited to control large display surfaces from a short distance away; why that is so; and, equally important, how such an interface can be made a reality. A well-known example of the type of interface that is

  11. Common neural substrates support speech and non-speech vocal tract gestures

    OpenAIRE

    Chang, Soo-Eun; Kenney, Mary Kay; Loucks, Torrey M.J.; Poletto, Christopher J.; Ludlow, Christy L.

    2009-01-01

    The issue of whether speech is supported by the same neural substrates as non-speech vocal-tract gestures has been contentious. In this fMRI study we tested whether producing non-speech vocal tract gestures in humans shares the same functional neuroanatomy as non-sense speech syllables. Production of non-speech vocal tract gestures, devoid of phonological content but similar to speech in that they had familiar acoustic and somatosensory targets, were compared to the production of speech sylla...

  12. Communicative Gestures Facilitate Problem Solving for Both Communicators and Recipients

    Science.gov (United States)

    Lozano, Sandra C.; Tversky, Barbara

    2006-01-01

    Gestures are a common, integral part of communication. Here, we investigate the roles of gesture and speech in explanations, both for communicators and recipients. Communicators explained how to assemble a simple object, using either speech with gestures or gestures alone. Gestures used for explaining included pointing and exhibiting to indicate…

  13. Hand Matters: Left-Hand Gestures Enhance Metaphor Explanation

    Science.gov (United States)

    Argyriou, Paraskevi; Mohr, Christine; Kita, Sotaro

    2017-01-01

    Research suggests that speech-accompanying gestures influence cognitive processes, but it is not clear whether the gestural benefit is specific to the gesturing hand. Two experiments tested the "(right/left) hand-specificity" hypothesis for self-oriented functions of gestures: gestures with a particular hand enhance cognitive processes…

  14. What Iconic Gesture Fragments Reveal about Gesture-Speech Integration: When Synchrony Is Lost, Memory Can Help

    Science.gov (United States)

    Obermeier, Christian; Holle, Henning; Gunter, Thomas C.

    2011-01-01

    The present series of experiments explores several issues related to gesture-speech integration and synchrony during sentence processing. To be able to more precisely manipulate gesture-speech synchrony, we used gesture fragments instead of complete gestures, thereby avoiding the usual long temporal overlap of gestures with their coexpressive…

  15. How early do children understand gesture-speech combinations with iconic gestures?

    Science.gov (United States)

    Stanfield, Carmen; Williamson, Rebecca; Ozçalişkan, Seyda

    2014-03-01

    Children understand gesture+speech combinations in which a deictic gesture adds new information to the accompanying speech by age 1;6 (Morford & Goldin-Meadow, 1992; 'push'+point at ball). This study explores how early children understand gesture+speech combinations in which an iconic gesture conveys additional information not found in the accompanying speech (e.g., 'read'+BOOK gesture). Our analysis of two- to four-year-old children's responses in a gesture+speech comprehension task showed that children grasp the meaning of iconic co-speech gestures by age three and continue to improve their understanding with age. Overall, our study highlights the important role gesture plays in language comprehension as children learn to unpack increasingly complex communications addressed to them at the early ages.

  16. Gestures Towards the Digital Maypole

    Directory of Open Access Journals (Sweden)

    Christian McRea

    2005-01-01

    Full Text Available To paraphrase Blanchot: We are not learned; we are not ignorant. We have known joys. That is saying too little: We are alive, and this life gives us the greatest pleasure. The intensities afforded by mobile communication can be thought of as an extension of the styles and gestures already materialised by multiple maypole cultures, pre-digital community forms and the very clustered natures of speech and being. In his Critique of Judgment, Kant argues that the information selection process at the heart of communication is one of the fundamental activities of any aesthetically produced knowledge form. From this radial point, "Gestures Towards The Digital Maypole" begins the process of reorganising conceptions of modalities of communication around the absent centre and the affective realms that form through the movement of information-energy, like sugar in a hurricane.

  17. Gestures in an Intelligent User Interface

    Science.gov (United States)

    Fikkert, Wim; van der Vet, Paul; Nijholt, Anton

    In this chapter we investigated which hand gestures are intuitive to control a large display multimedia interface from a user's perspective. Over the course of two sequential user evaluations, we defined a simple gesture set that allows users to fully control a large display multimedia interface, intuitively. First, we evaluated numerous gesture possibilities for a set of commands that can be issued to the interface. These gestures were selected from literature, science fiction movies, and a previous exploratory study. Second, we implemented a working prototype with which the users could interact with both hands and the preferred hand gestures with 2D and 3D visualizations of biochemical structures. We found that the gestures are influenced to significant extent by the fast paced developments in multimedia interfaces such as the Apple iPhone and the Nintendo Wii and to no lesser degree by decades of experience with the more traditional WIMP-based interfaces.

  18. Initial experiments with Multiple Musical Gestures

    DEFF Research Database (Denmark)

    Jensen, Kristoffer; Graugaard, Lars

    2005-01-01

    The classic orchestra has a diminishing role in society, while hard-disc recorded music plays a predominant role today. A simple to use pointer interface in 2D for producing music is presented as a means for playing in a social situation. The sounds of the music are produced by a low-level...... synthesizer, and the music is produced by simple gestures that are repeated easily. The gestures include left-to-right and right-to-left motion shapes for spectral envelope and temporal envelope of the sounds, with optional backwards motion for the addition of noise; downward motion for note onset and several...... other manipulation gestures. The initial position controls which parameter is being affected, the notes intensity is controlled by the downward gesture speed, and a sequence is finalized instantly with one upward gesture. The synthesis employs a novel interface structure, the multiple musical gesture...

  19. Acid chat: gestural interface design

    OpenAIRE

    Gökhan, Ali Oytun; Gokhan, Ali Oytun

    2005-01-01

    AcidChat is an experimental design project that aims to create an innovative computer software interface for Internet chat software using today's well known technologies; Adobe Photoshop, Macromedia Freehand and digital photography. The aim of the project is to create new understandings of interface and it's usage, by adding new conceptions to chat based interfaces which creates a totally new look at the computer software and application. One of the key features is to add a gestural approach ...

  20. Gesture analysis for physics education researchers

    Directory of Open Access Journals (Sweden)

    Rachel E. Scherr

    2008-01-01

    Full Text Available Systematic observations of student gestures can not only fill in gaps in students’ verbal expressions, but can also offer valuable information about student ideas, including their source, their novelty to the speaker, and their construction in real time. This paper provides a review of the research in gesture analysis that is most relevant to physics education researchers and illustrates gesture analysis for the purpose of better understanding student thinking about physics.

  1. Hand Gesture Recognition with Leap Motion

    OpenAIRE

    Du, Youchen; Liu, Shenglan; Feng, Lin; Chen, Menghui; Wu, Jie

    2017-01-01

    The recent introduction of depth cameras like Leap Motion Controller allows researchers to exploit the depth information to recognize hand gesture more robustly. This paper proposes a novel hand gesture recognition system with Leap Motion Controller. A series of features are extracted from Leap Motion tracking data, we feed these features along with HOG feature extracted from sensor images into a multi-class SVM classifier to recognize performed gesture, dimension reduction and feature weight...

  2. Grounded Blends and Mathematical Gesture Spaces: Developing Mathematical Understandings via Gestures

    Science.gov (United States)

    Yoon, Caroline; Thomas, Michael O. J.; Dreyfus, Tommy

    2011-01-01

    This paper examines how a person's gesture space can become endowed with mathematical meaning associated with mathematical spaces and how the resulting mathematical gesture space can be used to communicate and interpret mathematical features of gestures. We use the theory of grounded blends to analyse a case study of two teachers who used gestures…

  3. Language, Gesture, Action! A Test of the Gesture as Simulated Action Framework

    Science.gov (United States)

    Hostetter, Autumn B.; Alibali, Martha W.

    2010-01-01

    The Gesture as Simulated Action (GSA) framework (Hostetter & Alibali, 2008) holds that representational gestures are produced when actions are simulated as part of thinking and speaking. Accordingly, speakers should gesture more when describing images with which they have specific physical experience than when describing images that are less…

  4. Gesture facilitates the syntactic analysis of speech

    Directory of Open Access Journals (Sweden)

    Henning eHolle

    2012-03-01

    Full Text Available Recent research suggests that the brain routinely binds together information from gesture and speech. However, most of this research focused on the integration of representational gestures with the semantic content of speech. Much less is known about how other aspects of gesture, such as emphasis, influence the interpretation of the syntactic relations in a spoken message. Here, we investigated whether beat gestures alter which syntactic structure is assigned to ambiguous spoken German sentences. The P600 component of the Event Related Brain Potential indicated that the more complex syntactic structure is easier to process when the speaker emphasizes the subject of a sentence with a beat. Thus, a simple flick of the hand can change our interpretation of who has been doing what to whom in a spoken sentence. We conclude that gestures and speech are an integrated system. Unlike previous studies, which have shown that the brain effortlessly integrates semantic information from gesture and speech, our study is the first to demonstrate that this integration also occurs for syntactic information. Moreover, the effect appears to be gesture-specific and was not found for other stimuli that draw attention to certain parts of speech, including prosodic emphasis, or a moving visual stimulus with the same trajectory as the gesture. This suggests that only visual emphasis produced with a communicative intention in mind (that is, beat gestures influences language comprehension, but not a simple visual movement lacking such an intention.

  5. Research on Interaction-oriented Gesture Recognition

    Directory of Open Access Journals (Sweden)

    Lu Huang

    2014-01-01

    Full Text Available This thesis designs a series of gesture interaction with the features of the natural human-machine interaction; besides, it utilizes the 3D acceleration sensors as interactive input. Afterwards, it builds the Discrete Hidden Markov Model to make gesture recognition by introducing the collection proposal of gesture interaction based on the acceleration sensors and pre-handling the gesture acceleration signal obtained in the collection. In the end, the thesis proofs the design proposal workable and effective according to the experiments.

  6. Gesture and Speech in Interaction - 4th edition (GESPIN 4)

    OpenAIRE

    Ferré , Gaëlle; Mark , Tutton

    2015-01-01

    International audience; The fourth edition of Gesture and Speech in Interaction (GESPIN) was held in Nantes, France. With more than 40 papers, these proceedings show just what a flourishing field of enquiry gesture studies continues to be. The keynote speeches of the conference addressed three different aspects of multimodal interaction:gesture and grammar, gesture acquisition, and gesture and social interaction. In a talk entitled Qualitiesof event construal in speech and gesture: Aspect and...

  7. A comparison of sung and spoken phonation onset gestures using high-speed digital imaging.

    Science.gov (United States)

    Freeman, Ena; Woo, Peak; Saxman, John H; Murry, Thomas

    2012-03-01

    Phonation onset is important in the maintenance of healthy vocal production for speech and singing. The purpose of this preliminary study was to examine differences in vocal fold vibratory behavior between sung and spoken phonation onset gestures. Given the greater degree of precision required for the abrupt onset sung gestures, we hypothesize that differences exist in the timing and coordination of the vocal fold adductory gesture with the onset of vocal fold vibration. Staccato and German (a modified glottal plosive, so named for its occurrence in German classical singing) onset gestures were compared with breathy, normal, and hard onset gestures, using high-speed digital imaging. Samples were obtained from two subjects with no history of voice disorders (a female trained singer and a male nonsinger). Simultaneous capture of acoustical data confirmed the distinction among gestures. Image data were compared for glottal area configurations, degree of adductory positioning, number of small-amplitude prephonatory oscillations (PPOs), and timing of onset gesture events, the latter marked by maximum vocal fold abduction, maximum adduction, beginning of PPOs, and beginning of steady-state oscillation. Results reveal closer adductory positioning of the vocal folds for the staccato and German gestures. The data also suggest a direct relationship between the degree of adductory positioning and the number of PPOs. Results for the timing of onset gesture events suggest a relationship between discrete adductory positioning and more evenly spaced PPOs. By contrast, the overlapping of prephonatory adductory positioning with vibration onset revealed more unevenly spaced PPOs. This may support an existing hypothesis that less well-defined boundaries interfere with normal modes of vibration of the vocal fold tissue. Copyright © 2012 The Voice Foundation. Published by Mosby, Inc. All rights reserved.

  8. Enhacing spectral sintesis techniques with performance gestures using the violin as a case study

    OpenAIRE

    Pérez Carrillo, Alfonso Antonio

    2009-01-01

    In this work we investigate new sound synthesis techniques for imitating musical instruments using the violin as a case study. It is a multidisciplinary research, covering several fields such as spectral modeling, machine learning, analysis of musical gestures or musical acoustics. It addresses sound production with a very empirical approach, based on the analysis of performance gestures as well as on the measurement of acoustical properties of the violin. Based on the characteristics of the ...

  9. Hand Gesture Recognition Using Ultrasonic Waves

    KAUST Repository

    AlSharif, Mohammed Hussain

    2016-04-01

    Gesturing is a natural way of communication between people and is used in our everyday conversations. Hand gesture recognition systems are used in many applications in a wide variety of fields, such as mobile phone applications, smart TVs, video gaming, etc. With the advances in human-computer interaction technology, gesture recognition is becoming an active research area. There are two types of devices to detect gestures; contact based devices and contactless devices. Using ultrasonic waves for determining gestures is one of the ways that is employed in contactless devices. Hand gesture recognition utilizing ultrasonic waves will be the focus of this thesis work. This thesis presents a new method for detecting and classifying a predefined set of hand gestures using a single ultrasonic transmitter and a single ultrasonic receiver. This method uses a linear frequency modulated ultrasonic signal. The ultrasonic signal is designed to meet the project requirements such as the update rate, the range of detection, etc. Also, it needs to overcome hardware limitations such as the limited output power, transmitter, and receiver bandwidth, etc. The method can be adapted to other hardware setups. Gestures are identified based on two main features; range estimation of the moving hand and received signal strength (RSS). These two factors are estimated using two simple methods; channel impulse response (CIR) and cross correlation (CC) of the reflected ultrasonic signal from the gesturing hand. A customized simple hardware setup was used to classify a set of hand gestures with high accuracy. The detection and classification were done using methods of low computational cost. This makes the proposed method to have a great potential for the implementation in many devices including laptops and mobile phones. The predefined set of gestures can be used for many control applications.

  10. GESTURE'S ROLE IN CREATING AND LEARNING LANGUAGE.

    Science.gov (United States)

    Goldin-Meadow, Susan

    2010-09-22

    Imagine a child who has never seen or heard language. Would such a child be able to invent a language? Despite what one might guess, the answer is "yes". This chapter describes children who are congenitally deaf and cannot learn the spoken language that surrounds them. In addition, the children have not been exposed to sign language, either by their hearing parents or their oral schools. Nevertheless, the children use their hands to communicate--they gesture--and those gestures take on many of the forms and functions of language (Goldin-Meadow 2003a). The properties of language that we find in these gestures are just those properties that do not need to be handed down from generation to generation, but can be reinvented by a child de novo. They are the resilient properties of language, properties that all children, deaf or hearing, come to language-learning ready to develop. In contrast to these deaf children who are inventing language with their hands, hearing children are learning language from a linguistic model. But they too produce gestures, as do all hearing speakers (Feyereisen and de Lannoy 1991; Goldin-Meadow 2003b; Kendon 1980; McNeill 1992). Indeed, young hearing children often use gesture to communicate before they use words. Interestingly, changes in a child's gestures not only predate but also predict changes in the child's early language, suggesting that gesture may be playing a role in the language-learning process. This chapter begins with a description of the gestures the deaf child produces without speech. These gestures assume the full burden of communication and take on a language-like form--they are language. This phenomenon stands in contrast to the gestures hearing speakers produce with speech. These gestures share the burden of communication with speech and do not take on a language-like form--they are part of language.

  11. Nature and Specificity of Gestural Disorder in Children with Developmental Coordination Disorder: A Multiple Case Study

    Directory of Open Access Journals (Sweden)

    Orianne Costini

    2017-07-01

    Full Text Available Aim: Praxis assessment in children with developmental coordination disorder (DCD is usually based on tests of adult apraxia, by comparing across types of gestures and input modalities. However, the cognitive models of adult praxis processing are rarely used in a comprehensive and critical interpretation. These models generally involve two systems: a conceptual system and a production system. Heterogeneity of deficits is consistently reported in DCD, involving other cognitive skills such as executive or visual-perceptual and visuospatial functions. Surprisingly, few researches examined the impact of these functions in gestural production. Our study aimed at discussing the nature and specificity of the gestural deficit in DCD using a multiple case study approach.Method: Tasks were selected and adapted from protocols proposed in adult apraxia, in order to enable a comprehensive assessment of gestures. This included conceptual tasks (knowledge about tool functions and actions; recognition of gestures, representational (transitive, intransitive, and non-representational gestures (imitation of meaningless postures. We realized an additional assessment of constructional abilities and other cognitive domains (executive functions, visual-perceptual and visuospatial functions. Data from 27 patients diagnosed with DCD were collected. Neuropsychological profiles were classified using an inferential clinical analysis based on the modified t-test, by comparison with 100 typically developing children divided into five age groups (from 7 to 13 years old.Results: Among the 27 DCD patients, we first classified profiles that are characterized by impairment in tasks assessing perceptual visual or visuospatial skills (n = 8. Patients with a weakness in executive functions (n = 6 were then identified, followed by those with an impaired performance in conceptual knowledge tasks (n = 4. Among the nine remaining patients, six could be classified as having a visual

  12. Individual differences in the gesture effect on working memory.

    Science.gov (United States)

    Marstaller, Lars; Burianová, Hana

    2013-06-01

    Co-speech gestures have been shown to interact with working memory (WM). However, no study has investigated whether there are individual differences in the effect of gestures on WM. Combining a novel gesture/no-gesture task and an operation span task, we examined the differences in WM accuracy between individuals who gestured and individuals who did not gesture in relation to their WM capacity. Our results showed individual differences in the gesture effect on WM. Specifically, only individuals with low WM capacity showed a reduced WM accuracy when they did not gesture. Individuals with low WM capacity who did gesture, as well as high-capacity individuals (irrespective of whether they gestured or not), did not show the effect. Our findings show that the interaction between co-speech gestures and WM is affected by an individual's WM load.

  13. Functional neuroanatomy of gesture-speech integration in children varies with individual differences in gesture processing.

    Science.gov (United States)

    Demir-Lira, Özlem Ece; Asaridou, Salomi S; Raja Beharelle, Anjali; Holt, Anna E; Goldin-Meadow, Susan; Small, Steven L

    2018-03-08

    Gesture is an integral part of children's communicative repertoire. However, little is known about the neurobiology of speech and gesture integration in the developing brain. We investigated how 8- to 10-year-old children processed gesture that was essential to understanding a set of narratives. We asked whether the functional neuroanatomy of gesture-speech integration varies as a function of (1) the content of speech, and/or (2) individual differences in how gesture is processed. When gestures provided missing information not present in the speech (i.e., disambiguating gesture; e.g., "pet" + flapping palms = bird), the presence of gesture led to increased activity in inferior frontal gyri, the right middle temporal gyrus, and the left superior temporal gyrus, compared to when gesture provided redundant information (i.e., reinforcing gesture; e.g., "bird" + flapping palms = bird). This pattern of activation was found only in children who were able to successfully integrate gesture and speech behaviorally, as indicated by their performance on post-test story comprehension questions. Children who did not glean meaning from gesture did not show differential activation across the two conditions. Our results suggest that the brain activation pattern for gesture-speech integration in children overlaps with-but is broader than-the pattern in adults performing the same task. Overall, our results provide a possible neurobiological mechanism that could underlie children's increasing ability to integrate gesture and speech over childhood, and account for individual differences in that integration. © 2018 John Wiley & Sons Ltd.

  14. Nonsymbolic Gestural Interaction for Ambient Intelligence

    DEFF Research Database (Denmark)

    Rehm, Matthias

    2010-01-01

    the addressee with subtle clues about personality or cultural background. Gestures are an extremly rich source of communication-specific and contextual information for interactions in ambient intelligence environments. This chapter reviews the semantic layers of gestural interaction, focusing on the layer...

  15. Gesture Analysis for Physics Education Researchers

    Science.gov (United States)

    Scherr, Rachel E.

    2008-01-01

    Systematic observations of student gestures can not only fill in gaps in students' verbal expressions, but can also offer valuable information about student ideas, including their source, their novelty to the speaker, and their construction in real time. This paper provides a review of the research in gesture analysis that is most relevant to…

  16. Enhancing Communication through Gesture and Naming Therapy

    Science.gov (United States)

    Caute, Anna; Pring, Tim; Cocks, Naomi; Cruice, Madeline; Best, Wendy; Marshall, Jane

    2013-01-01

    Purpose: In this study, the authors investigated whether gesture, naming, and strategic treatment improved the communication skills of 14 people with severe aphasia. Method: All participants received 15 hr of gesture and naming treatment (reported in a companion article [Marshall et al., 2012]). Half the group received a further 15 hr of strategic…

  17. Pitch Gestures in Generative Modeling of Music

    DEFF Research Database (Denmark)

    Jensen, Kristoffer

    2011-01-01

    Generative models of music are in need of performance and gesture additions, i.e. inclusions of subtle temporal and dynamic alterations, and gestures so as to render the music musical. While much of the research regarding music generation is based on music theory, the work presented here is based...

  18. Towards a Description of East African Gestures

    Science.gov (United States)

    Creider, Chet A.

    1977-01-01

    This paper describes the gestural behavior of four tribal groups, Kipsigis, Luo, Gusii, and Samburu, observed and elicted in the course of two and one-half years of field work in Western Kenya in 1970-72. The gestures are grouped into four categories: (1) initiators and finalizers of interaction; (2) imperatives; (3) responses; (4) qualifiers.…

  19. Aspects of the Multiple Musical Gestures

    DEFF Research Database (Denmark)

    Jensen, Karl Kristoffer

    2006-01-01

    is finalized instantly with one upward gesture. Several synthesis methods are presented and the control mechanisms are mapped into the multiple musical gesture interface. This enables a number of performers to interact on the same interface, either by each playing the same musical instruments simultaneously...

  20. The comprehension of gesture and speech

    NARCIS (Netherlands)

    Willems, R.M.; Özyürek, A.; Hagoort, P.

    2005-01-01

    Although generally studied in isolation, action observation and speech comprehension go hand in hand during everyday human communication. That is, people gesture while they speak. From previous research it is known that a tight link exists between spoken language and such hand gestures. This study

  1. The ontogenetic ritualization of bonobo gestures.

    Science.gov (United States)

    Halina, Marta; Rossano, Federico; Tomasello, Michael

    2013-07-01

    Great apes communicate with gestures in flexible ways. Based on several lines of evidence, Tomasello and colleagues have posited that many of these gestures are learned via ontogenetic ritualization-a process of mutual anticipation in which particular social behaviors come to function as intentional communicative signals. Recently, Byrne and colleagues have argued that all great ape gestures are basically innate. In the current study, for the first time, we attempted to observe the process of ontogenetic ritualization as it unfolds over time. We focused on one communicative function between bonobo mothers and infants: initiation of "carries" for joint travel. We observed 1,173 carries in ten mother-infant dyads. These were initiated by nine different gesture types, with mothers and infants using many different gestures in ways that reflected their different roles in the carry interaction. There was also a fair amount of variability among the different dyads, including one idiosyncratic gesture used by one infant. This gestural variation could not be attributed to sampling effects alone. These findings suggest that ontogenetic ritualization plays an important role in the origin of at least some great ape gestures.

  2. Integration of speech and gesture in aphasia.

    Science.gov (United States)

    Cocks, Naomi; Byrne, Suzanne; Pritchard, Madeleine; Morgan, Gary; Dipper, Lucy

    2018-02-07

    Information from speech and gesture is often integrated to comprehend a message. This integration process requires the appropriate allocation of cognitive resources to both the gesture and speech modalities. People with aphasia are likely to find integration of gesture and speech difficult. This is due to a reduction in cognitive resources, a difficulty with resource allocation or a combination of the two. Despite it being likely that people who have aphasia will have difficulty with integration, empirical evidence describing this difficulty is limited. Such a difficulty was found in a single case study by Cocks et al. in 2009, and is replicated here with a greater number of participants. To determine whether individuals with aphasia have difficulties understanding messages in which they have to integrate speech and gesture. Thirty-one participants with aphasia (PWA) and 30 control participants watched videos of an actor communicating a message in three different conditions: verbal only, gesture only, and verbal and gesture message combined. The message related to an action in which the name of the action (e.g., 'eat') was provided verbally and the manner of the action (e.g., hands in a position as though eating a burger) was provided gesturally. Participants then selected a picture that 'best matched' the message conveyed from a choice of four pictures which represented a gesture match only (G match), a verbal match only (V match), an integrated verbal-gesture match (Target) and an unrelated foil (UR). To determine the gain that participants obtained from integrating gesture and speech, a measure of multimodal gain (MMG) was calculated. The PWA were less able to integrate gesture and speech than the control participants and had significantly lower MMG scores. When the PWA had difficulty integrating, they more frequently selected the verbal match. The findings suggest that people with aphasia can have difficulty integrating speech and gesture in order to obtain

  3. The gesture in Physical Culture career teaching

    Directory of Open Access Journals (Sweden)

    Alina Bestard-Revilla

    2015-04-01

    Full Text Available The research is in charge of gesture interpretation of Physical Culture Career's teacherr with the objective of revealing the senses that underlie in the pedagogic al interaction between the teacher and the students. It also tends to the analysis and understanding of the teacher's gestures during their pedagogic al interactions. The research answers the following question s: How to take advantage s from the Physical Culture university teachers for a greater quality of his lessons ?, and it precisely looks for the gesture inter pretation, analyzes what underlies in a gesture in a teaching learning space; reveals the meanings contained in a glance, the hands signalizations, the corporal postures, the approaches, the smiles, among other important expressions in the teachers communi cative situations in correspondence with the students gestures.

  4. Role of maternal gesture use in speech use by children with fragile X syndrome.

    Science.gov (United States)

    Hahn, Laura J; Zimmer, B Jean; Brady, Nancy C; Swinburne Romine, Rebecca E; Fleming, Kandace K

    2014-05-01

    The purpose of this study was to investigate how maternal gesture relates to speech production by children with fragile X syndrome (FXS). Participants were 27 young children with FXS (23 boys, 4 girls) and their mothers. Videotaped home observations were conducted between the ages of 25 and 37 months (toddler period) and again between the ages of 60 and 71 months (child period). The videos were later coded for types of maternal utterances and maternal gestures that preceded child speech productions. Children were also assessed with the Mullen Scales of Early Learning at both ages. Maternal gesture use in the toddler period was positively related to expressive language scores at both age periods and was related to receptive language scores in the child period. Maternal proximal pointing, in comparison to other gestures, evoked more speech responses from children during the mother-child interactions, particularly when combined with wh-questions. This study adds to the growing body of research on the importance of contextual variables, such as maternal gestures, in child language development. Parental gesture use may be an easily added ingredient to parent-focused early language intervention programs.

  5. Common neural substrates support speech and non-speech vocal tract gestures.

    Science.gov (United States)

    Chang, Soo-Eun; Kenney, Mary Kay; Loucks, Torrey M J; Poletto, Christopher J; Ludlow, Christy L

    2009-08-01

    The issue of whether speech is supported by the same neural substrates as non-speech vocal tract gestures has been contentious. In this fMRI study we tested whether producing non-speech vocal tract gestures in humans shares the same functional neuroanatomy as non-sense speech syllables. Production of non-speech vocal tract gestures, devoid of phonological content but similar to speech in that they had familiar acoustic and somatosensory targets, was compared to the production of speech syllables without meaning. Brain activation related to overt production was captured with BOLD fMRI using a sparse sampling design for both conditions. Speech and non-speech were compared using voxel-wise whole brain analyses, and ROI analyses focused on frontal and temporoparietal structures previously reported to support speech production. Results showed substantial activation overlap between speech and non-speech function in regions. Although non-speech gesture production showed greater extent and amplitude of activation in the regions examined, both speech and non-speech showed comparable left laterality in activation for both target perception and production. These findings posit a more general role of the previously proposed "auditory dorsal stream" in the left hemisphere--to support the production of vocal tract gestures that are not limited to speech processing.

  6. Young Children Create Iconic Gestures to Inform Others

    Science.gov (United States)

    Behne, Tanya; Carpenter, Malinda; Tomasello, Michael

    2014-01-01

    Much is known about young children's use of deictic gestures such as pointing. Much less is known about their use of other types of communicative gestures, especially iconic or symbolic gestures. In particular, it is unknown whether children can create iconic gestures on the spot to inform others. Study 1 provided 27-month-olds with the…

  7. Introduction: Towards an Ethics of Gesture

    Directory of Open Access Journals (Sweden)

    Lucia Ruprecht

    2017-06-01

    Full Text Available The introduction to this special section of Performance Philosophy takes Giorgio Agamben’s remarks about the mediality and potentiality of gesture as a starting point to rethink gesture’s nexus with ethics. Shifting the emphasis from philosophical reflection to corporeal practice, it defines gestural ethics as an acting-otherwise which comes into being in the particularities of singular gestural practice, its forms, kinetic qualities, temporal displacements and calls for response. Gestural acting-otherwise is illustrated in a number of ways: We might talk of a gestural ethics when gesturality becomes an object for dedicated analytical exploration and reflection on sites where it is not taken for granted, but exhibited, on stage or on screen, in its mediality, in the ways it quotes, signifies and departs from signification, but also in the ways in which it follows a forward-looking agenda driven by adaptability and inventiveness. It interrupts or modifies operative continua that might be geared towards violence; it appears in situations that are suspended between the possibility of malfunction and the potential of room for play; and it emerges in the ways in which gestures act on their own implication in the signifying structures of gender, sexuality, race, and class, on how these structures play out relationally across time and space, and between historically and locally situated human beings.

  8. Hand gesture recognition by analysis of codons

    Science.gov (United States)

    Ramachandra, Poornima; Shrikhande, Neelima

    2007-09-01

    The problem of recognizing gestures from images using computers can be approached by closely understanding how the human brain tackles it. A full fledged gesture recognition system will substitute mouse and keyboards completely. Humans can recognize most gestures by looking at the characteristic external shape or the silhouette of the fingers. Many previous techniques to recognize gestures dealt with motion and geometric features of hands. In this thesis gestures are recognized by the Codon-list pattern extracted from the object contour. All edges of an image are described in terms of sequence of Codons. The Codons are defined in terms of the relationship between maxima, minima and zeros of curvature encountered as one traverses the boundary of the object. We have concentrated on a catalog of 24 gesture images from the American Sign Language alphabet (Letter J and Z are ignored as they are represented using motion) [2]. The query image given as an input to the system is analyzed and tested against the Codon-lists, which are shape descriptors for external parts of a hand gesture. We have used the Weighted Frequency Indexing Transform (WFIT) approach which is used in DNA sequence matching for matching the Codon-lists. The matching algorithm consists of two steps: 1) the query sequences are converted to short sequences and are assigned weights and, 2) all the sequences of query gestures are pruned into match and mismatch subsequences by the frequency indexing tree based on the weights of the subsequences. The Codon sequences with the most weight are used to determine the most precise match. Once a match is found, the identified gesture and corresponding interpretation are shown as output.

  9. Device Control Using Gestures Sensed from EMG

    Science.gov (United States)

    Wheeler, Kevin R.

    2003-01-01

    In this paper we present neuro-electric interfaces for virtual device control. The examples presented rely upon sampling Electromyogram data from a participants forearm. This data is then fed into pattern recognition software that has been trained to distinguish gestures from a given gesture set. The pattern recognition software consists of hidden Markov models which are used to recognize the gestures as they are being performed in real-time. Two experiments were conducted to examine the feasibility of this interface technology. The first replicated a virtual joystick interface, and the second replicated a keyboard.

  10. Gestural Control Of Wavefield synthesis

    DEFF Research Database (Denmark)

    Grani, Francesco; Di Carlo, Diego; Portillo, Jorge Madrid

    2016-01-01

    We present a report covering our preliminary research on the control of spatial sound sources in wavefield synthesis through gesture based interfaces. After a short general introduction on spatial sound and few basic concepts on wavefield synthesis, we presents a graphical application called sp......AAce which let users to con- trol real-time movements of sound sources by drawing tra- jectories on a screen. The first prototype of this application has been developed bound to WFSCollider, an open-source software based on Supercollider which let users control wavefield synthesis. The spAAce application has...... been im- plemented using Processing, a programming language for sketches and prototypes within the context of visual arts, and communicates with WFSCollider through the Open Sound Control protocol. This application aims to create a new way of interaction for live performance of spatial composition...

  11. Evaluación por rendimiento de 12 genotipos promisorios de fríjol voluble (Phaseolus vulgaris L. tipo Bola roja y Reventón para las zonas frías de Colombia

    Directory of Open Access Journals (Sweden)

    López Jesús Edgardo

    2006-12-01

    Full Text Available

    El fríjol común (Phaseolus vulgaris L. es un alimento básico en la región Andina por ser una fuente rica en proteína y de bajo costo. La investigación para incrementar rendimientos en esta leguminosa es una opción para mejorar la competitividad en el mercado mundial. El objetivo principal de este trabajo fue evaluar por rendimiento los genotipos promisorios de fríjol voluble, tipos Bola roja y Reventón, para las zonas frías de Colombia mediante el análisis de sendero. Se realizó un diseño de bloques completos al azar con tres réplicas para evaluar 10 genotipos promisorios de fríjol voluble. El análisis de sendero para el rendimiento por planta y las correlaciones entre el rendimiento y sus componentes mostraron que el carácter número de vainas por planta es el de mayor importancia sobre la determinación del rendimiento, en comparación con los caracteres peso de 100 semillas y número de semillas por vaina, tanto en los genotipos de fríjol voluble tipo Bola roja como tipo Reventón. 

  12. Gesture recognition for an exergame prototype

    NARCIS (Netherlands)

    Gacem, Brahim; Vergouw, Robert; Verbiest, Harm; Cicek, Emrullah; Kröse, Ben; van Oosterhout, Tim; Bakkes, S.C.J.

    2011-01-01

    We will demonstrate a prototype exergame aimed at the serious domain of elderly fitness. The exergame incorporates straightforward means to gesture recognition, and utilises a Kinect camera to obtain 2.5D sensory data of the human user.

  13. Hand Gesture Recognition Using Ultrasonic Waves

    KAUST Repository

    AlSharif, Mohammed Hussain

    2016-01-01

    estimation of the moving hand and received signal strength (RSS). These two factors are estimated using two simple methods; channel impulse response (CIR) and cross correlation (CC) of the reflected ultrasonic signal from the gesturing hand. A customized

  14. Evolutionary Sound Synthesis Controlled by Gestural Data

    Directory of Open Access Journals (Sweden)

    Jose Fornari

    2011-05-01

    Full Text Available This article focuses on the interdisciplinary research involving Computer Music and Generative Visual Art. We describe the implementation of two interactive artistic systems based on principles of Gestural Data (WILSON, 2002 retrieval and self-organization (MORONI, 2003, to control an Evolutionary Sound Synthesis method (ESSynth. The first implementation uses, as gestural data, image mapping of handmade drawings. The second one uses gestural data from dynamic body movements of dance. The resulting computer output is generated by an interactive system implemented in Pure Data (PD. This system uses principles of Evolutionary Computation (EC, which yields the generation of a synthetic adaptive population of sound objects. Considering that music could be seen as “organized sound” the contribution of our study is to develop a system that aims to generate "self-organized sound" – a method that uses evolutionary computation to bridge between gesture, sound and music.

  15. Gesture and form in the Neolithic graphic expression

    Directory of Open Access Journals (Sweden)

    Philippe HAMEAU

    2011-10-01

    Full Text Available The parietal painted sign keeps on the memory of gesture that produces it. It is one of the distinctive features of this particular artefact. However, it is not possible to reconstruct this gesture if we do not give the context of the sign, if we do not present the numerous physical and cultural parameters which are in charge of their production. About schematic paintings of Neolithic age, we must take the union of criterions into account such as the parietal and site topography, the cultural constraints that appoint the location of figures and the ritual practices originally the graphical expression. The painter perceives, adapts and behaves according to this spatial and social environment. We refer here to several strategies: the attention for the parietal microtopography in accordance with the signs to draw, the respect of some criterions that specify the choice of the site like the hygrophily of places and the rubefaction of rock walls, the need to paint at the limits of the accessibility of site and wall, the use of drawing-tools for increase the capacities of the body. The efficiency of the gesture consists in realizing a sign bearing a meaning because in harmony with the features of its support.

  16. Gestural interaction in a virtual environment

    Science.gov (United States)

    Jacoby, Richard H.; Ferneau, Mark; Humphries, Jim

    1994-04-01

    This paper discusses the use of hand gestures (i.e., changing finger flexion) within a virtual environment (VE). Many systems now employ static hand postures (i.e., static finger flexion), often coupled with hand translations and rotations, as a method of interacting with a VE. However, few systems are currently using dynamically changing finger flexion for interacting with VEs. In our system, the user wears an electronically instrumented glove. We have developed a simple algorithm for recognizing gestures for use in two applications: automotive design and visualization of atmospheric data. In addition to recognizing the gestures, we also calculate the rate at which the gestures are made and the rate and direction of hand movement while making the gestures. We report on our experiences with the algorithm design and implementation, and the use of the gestures in our applications. We also talk about our background work in user calibration of the glove, as well as learned and innate posture recognition (postures recognized with and without training, respectively).

  17. Make Gestures to Learn: Reproducing Gestures Improves the Learning of Anatomical Knowledge More than Just Seeing Gestures

    Directory of Open Access Journals (Sweden)

    Mélaine Cherdieu

    2017-10-01

    Full Text Available Manual gestures can facilitate problem solving but also language or conceptual learning. Both seeing and making the gestures during learning seem to be beneficial. However, the stronger activation of the motor system in the second case should provide supplementary cues to consolidate and re-enact the mental traces created during learning. We tested this hypothesis in the context of anatomy learning by naïve adult participants. Anatomy is a challenging topic to learn and is of specific interest for research on embodied learning, as the learning content can be directly linked to learners' body. Two groups of participants were asked to look at a video lecture on the forearm anatomy. The video included a model making gestures related to the content of the lecture. Both groups see the gestures but only one also imitate the model. Tests of knowledge were run just after learning and few days later. The results revealed that imitating gestures improves the recall of structures names and their localization on a diagram. This effect was however significant only in long-term assessments. This suggests that: (1 the integration of motor actions and knowledge may require sleep; (2 a specific activation of the motor system during learning may improve the consolidation and/or the retrieval of memories.

  18. Make Gestures to Learn: Reproducing Gestures Improves the Learning of Anatomical Knowledge More than Just Seeing Gestures

    Science.gov (United States)

    Cherdieu, Mélaine; Palombi, Olivier; Gerber, Silvain; Troccaz, Jocelyne; Rochet-Capellan, Amélie

    2017-01-01

    Manual gestures can facilitate problem solving but also language or conceptual learning. Both seeing and making the gestures during learning seem to be beneficial. However, the stronger activation of the motor system in the second case should provide supplementary cues to consolidate and re-enact the mental traces created during learning. We tested this hypothesis in the context of anatomy learning by naïve adult participants. Anatomy is a challenging topic to learn and is of specific interest for research on embodied learning, as the learning content can be directly linked to learners' body. Two groups of participants were asked to look at a video lecture on the forearm anatomy. The video included a model making gestures related to the content of the lecture. Both groups see the gestures but only one also imitate the model. Tests of knowledge were run just after learning and few days later. The results revealed that imitating gestures improves the recall of structures names and their localization on a diagram. This effect was however significant only in long-term assessments. This suggests that: (1) the integration of motor actions and knowledge may require sleep; (2) a specific activation of the motor system during learning may improve the consolidation and/or the retrieval of memories. PMID:29062287

  19. Make Gestures to Learn: Reproducing Gestures Improves the Learning of Anatomical Knowledge More than Just Seeing Gestures.

    Science.gov (United States)

    Cherdieu, Mélaine; Palombi, Olivier; Gerber, Silvain; Troccaz, Jocelyne; Rochet-Capellan, Amélie

    2017-01-01

    Manual gestures can facilitate problem solving but also language or conceptual learning. Both seeing and making the gestures during learning seem to be beneficial. However, the stronger activation of the motor system in the second case should provide supplementary cues to consolidate and re-enact the mental traces created during learning. We tested this hypothesis in the context of anatomy learning by naïve adult participants. Anatomy is a challenging topic to learn and is of specific interest for research on embodied learning, as the learning content can be directly linked to learners' body. Two groups of participants were asked to look at a video lecture on the forearm anatomy. The video included a model making gestures related to the content of the lecture. Both groups see the gestures but only one also imitate the model. Tests of knowledge were run just after learning and few days later. The results revealed that imitating gestures improves the recall of structures names and their localization on a diagram. This effect was however significant only in long-term assessments. This suggests that: (1) the integration of motor actions and knowledge may require sleep; (2) a specific activation of the motor system during learning may improve the consolidation and/or the retrieval of memories.

  20. Co-Thought and Co-Speech Gestures Are Generated by the Same Action Generation Process

    Science.gov (United States)

    Chu, Mingyuan; Kita, Sotaro

    2016-01-01

    People spontaneously gesture when they speak (co-speech gestures) and when they solve problems silently (co-thought gestures). In this study, we first explored the relationship between these 2 types of gestures and found that individuals who produced co-thought gestures more frequently also produced co-speech gestures more frequently (Experiments…

  1. Different visual exploration of tool-related gestures in left hemisphere brain damaged patients is associated with poor gestural imitation.

    Science.gov (United States)

    Vanbellingen, Tim; Schumacher, Rahel; Eggenberger, Noëmi; Hopfner, Simone; Cazzoli, Dario; Preisig, Basil C; Bertschi, Manuel; Nyffeler, Thomas; Gutbrod, Klemens; Bassetti, Claudio L; Bohlhalter, Stephan; Müri, René M

    2015-05-01

    According to the direct matching hypothesis, perceived movements automatically activate existing motor components through matching of the perceived gesture and its execution. The aim of the present study was to test the direct matching hypothesis by assessing whether visual exploration behavior correlate with deficits in gestural imitation in left hemisphere damaged (LHD) patients. Eighteen LHD patients and twenty healthy control subjects took part in the study. Gesture imitation performance was measured by the test for upper limb apraxia (TULIA). Visual exploration behavior was measured by an infrared eye-tracking system. Short videos including forty gestures (20 meaningless and 20 communicative gestures) were presented. Cumulative fixation duration was measured in different regions of interest (ROIs), namely the face, the gesturing hand, the body, and the surrounding environment. Compared to healthy subjects, patients fixated significantly less the ROIs comprising the face and the gesturing hand during the exploration of emblematic and tool-related gestures. Moreover, visual exploration of tool-related gestures significantly correlated with tool-related imitation as measured by TULIA in LHD patients. Patients and controls did not differ in the visual exploration of meaningless gestures, and no significant relationships were found between visual exploration behavior and the imitation of emblematic and meaningless gestures in TULIA. The present study thus suggests that altered visual exploration may lead to disturbed imitation of tool related gestures, however not of emblematic and meaningless gestures. Consequently, our findings partially support the direct matching hypothesis. Copyright © 2015 Elsevier Ltd. All rights reserved.

  2. Spontaneous gestures influence strategy choices in problem solving.

    Science.gov (United States)

    Alibali, Martha W; Spencer, Robert C; Knox, Lucy; Kita, Sotaro

    2011-09-01

    Do gestures merely reflect problem-solving processes, or do they play a functional role in problem solving? We hypothesized that gestures highlight and structure perceptual-motor information, and thereby make such information more likely to be used in problem solving. Participants in two experiments solved problems requiring the prediction of gear movement, either with gesture allowed or with gesture prohibited. Such problems can be correctly solved using either a perceptual-motor strategy (simulation of gear movements) or an abstract strategy (the parity strategy). Participants in the gesture-allowed condition were more likely to use perceptual-motor strategies than were participants in the gesture-prohibited condition. Gesture promoted use of perceptual-motor strategies both for participants who talked aloud while solving the problems (Experiment 1) and for participants who solved the problems silently (Experiment 2). Thus, spontaneous gestures influence strategy choices in problem solving.

  3. Gesture Commanding of a Robot with EVA Gloves

    Data.gov (United States)

    National Aeronautics and Space Administration — Gestures commands allow a human operator to directly interact with a robot without the use of intermediary hand controllers. There are two main types of hand gesture...

  4. Spatial reference in a bonobo gesture.

    Science.gov (United States)

    Genty, Emilie; Zuberbühler, Klaus

    2014-07-21

    Great apes frequently produce gestures during social interactions to communicate in flexible, goal-directed ways [1-3], a feature with considerable relevance for the ongoing debate over the evolutionary origins of human language [1, 4]. But despite this shared feature with language, there has been a lack of evidence for semantic content in ape gestures. According to one authoritative view, ape gestures thus do not have any specific referential, iconic, or deictic content, a fundamental difference versus human gestures and spoken language [1, 5] that suggests these features have a more recent origin in human evolution, perhaps caused by a fundamental transition from ape-like individual intentionality to human-like shared intentionality [6]. Here, we revisit this human uniqueness claim with a study of a previously undescribed human-like beckoning gesture in bonobos that has potentially both deictic and iconic character. We analyzed beckoning in two groups of bonobos, kept under near natural environmental and social conditions at the Lola Ya Bonobo sanctuary near Kinshasa, Democratic Republic of Congo, in terms of its linguistic content and underlying communicative intention. Copyright © 2014 Elsevier Ltd. All rights reserved.

  5. Tactile Feedback for Above-Device Gesture Interfaces

    OpenAIRE

    Freeman, Euan; Brewster, Stephen; Lantz, Vuokko

    2014-01-01

    Above-device gesture interfaces let people interact in the space above mobile devices using hand and finger movements. For example, users could gesture over a mobile phone or wearable without having to use the touchscreen. We look at how above-device interfaces can also give feedback in the space over the device. Recent haptic and wearable technologies give new ways to provide tactile feedback while gesturing, letting touchless gesture interfaces give touch feedback. In this paper we take a f...

  6. Narrative Processing in Typically Developing Children and Children with Early Unilateral Brain Injury: Seeing Gesture Matters

    Science.gov (United States)

    Demir, Özlem Ece; Fisher, Joan A.; Goldin-Meadow, Susan; Levine, Susan C.

    2014-01-01

    Narrative skill in kindergarteners has been shown to be a reliable predictor of later reading comprehension and school achievement. However, we know little about how to scaffold children’s narrative skill. Here we examine whether the quality of kindergarten children’s narrative retellings depends on the kind of narrative elicitation they are given. We asked this question in typically developing (TD) kindergarten children and in children with pre- or perinatal unilateral brain injury (PL), a group that has been shown to have difficulty with narrative production. We compared children’s skill in story retellings under four different elicitation formats: (1) wordless cartoons, (2) stories told by a narrator through the auditory modality, (3) stories told by a narrator through the audiovisual modality without co-speech gestures, and (4) stories told by a narrator in the audiovisual modality with co-speech gestures. We found that children told better structured narratives in the fourth, audiovisual + gesture elicitation format than in the other three elicitation formats, consistent with findings that co-speech gestures can scaffold other aspects of language and memory. The audiovisual + gesture elicitation format was particularly beneficial to children who had the most difficulty telling a well-structured narrative, a group that included children with larger lesions associated with cerebrovascular infarcts. PMID:24127729

  7. Asymmetric coupling between gestures and speech during reasoning

    NARCIS (Netherlands)

    Hoekstra, Lisette

    2017-01-01

    When children learn, insights displayed in gestures typically precede insights displayed in speech. In this study, we investigated how this leading role of gestures in cognitive development is evident in (and emerges from) the dynamic coupling between gestures and speech during one task. We

  8. Iconic Gestures as Undervalued Representations during Science Teaching

    Science.gov (United States)

    Chue, Shien; Lee, Yew-Jin; Tan, Kim Chwee Daniel

    2015-01-01

    Iconic gestures that are ubiquitous in speech are integral to human meaning-making. However, few studies have attempted to map out the role of these gestures in science teaching. This paper provides a review of existing literature in everyday communication and education to articulate potential contributions of iconic gestures for science teaching.…

  9. Adaptation in Gesture: Converging Hands or Converging Minds?

    Science.gov (United States)

    Mol, Lisette; Krahmer, Emiel; Maes, Alfons; Swerts, Marc

    2012-01-01

    Interlocutors sometimes repeat each other's co-speech hand gestures. In three experiments, we investigate to what extent the copying of such gestures' form is tied to their meaning in the linguistic context, as well as to interlocutors' representations of this meaning at the conceptual level. We found that gestures were repeated only if they could…

  10. Recognizing Stress Using Semantics and Modulation of Speech and Gestures

    NARCIS (Netherlands)

    Lefter, I.; Burghouts, G.J.; Rothkrantz, L.J.M.

    2016-01-01

    This paper investigates how speech and gestures convey stress, and how they can be used for automatic stress recognition. As a first step, we look into how humans use speech and gestures to convey stress. In particular, for both speech and gestures, we distinguish between stress conveyed by the

  11. Benefits and limitations of computer gesture therapy for the rehabilitation of severe aphasia

    Directory of Open Access Journals (Sweden)

    Abi Roper

    2016-11-01

    Full Text Available Aphasia intervention has made increasing use of technology in recent years. The evidence base, which is largely limited to the investigation of spoken language outcomes, indicates positive treatment effects for people with mild to moderate levels of aphasia. Outcomes for those with severe aphasia however, are less well documented and - where reported - present less consistent gains for measures of spoken output. This study investigates the effects of a purpose-built gesture therapy technology for people with severe aphasia: GeST+. Study outcomes show significant improvement in gesture production abilities for adults with severe aphasia following computer intervention. They indicate no transfer of effects into naming gains or interactive gesture. Outcomes offer encouraging results for computer therapy methods within this hitherto under-researched population but indicate a need for further refinement of interventions in order to maximize persistence of effects and generalization into everyday communication.

  12. Exploring the Use of Discrete Gestures for Authentication

    Science.gov (United States)

    Chong, Ming Ki; Marsden, Gary

    Research in user authentication has been a growing field in HCI. Previous studies have shown that peoples’ graphical memory can be used to increase password memorability. On the other hand, with the increasing number of devices with built-in motion sensors, kinesthetic memory (or muscle memory) can also be exploited for authentication. This paper presents a novel knowledge-based authentication scheme, called gesture password, which uses discrete gestures as password elements. The research presents a study of multiple password retention using PINs and gesture passwords. The study reports that although participants could use kinesthetic memory to remember gesture passwords, retention of PINs is far superior to retention of gesture passwords.

  13. Touch and Gesture-Based Language Learning: Some Possible Avenues for Research and Classroom Practice

    Science.gov (United States)

    Reinders, Hayo

    2014-01-01

    Our interaction with digital resources is becoming increasingly based on touch, gestures, and now also eye movement. Many everyday consumer electronics products already include touch-based interfaces, from e-book readers to tablets, and from the last personal computers to the GPS system in your car. What implications do these new forms of…

  14. Communicating Epistemic Stance: How Speech and Gesture Patterns Reflect Epistemicity and Evidentiality

    Science.gov (United States)

    Roseano, Paolo; González, Montserrat; Borràs-Comes, Joan; Prieto, Pilar

    2016-01-01

    This study investigates how epistemic stance is encoded and perceived in face-to-face communication when language is regarded as comprised by speech and gesture. Two studies were conducted with this goal in mind. The first study consisted of a production task in which participants performed opinion reports. Results showed that speakers communicate…

  15. When gestures show us the way: Co-speech gestures selectively facilitate navigation and spatial memory.

    OpenAIRE

    Galati, Alexia; Weisberg, Steven M.; Newcombe, Nora S.; Avraamides, Marios N.

    2017-01-01

    How does gesturing during route learning relate to subsequent spatial performance? We examined the relationship between gestures produced spontaneously while studying route directions and spatial representations of the navigated environment. Participants studied route directions, then navigated those routes from memory in a virtual environment, and finally had their memory of the environment assessed. We found that, for navigators with low spatial perspective-taking pe...

  16. From action to abstraction: Gesture as a mechanism of change.

    Science.gov (United States)

    Goldin-Meadow, Susan

    2015-12-01

    Piaget was a master at observing the routine behaviors children produce as they go from knowing less to knowing more about at a task, and making inferences not only about how the children understood the task at each point, but also about how they progressed from one point to the next. In this paper, I examine a routine behavior that Piaget overlooked-the spontaneous gestures speakers produce as they explain their solutions to a problem. These gestures are not mere hand waving. They reflect ideas that the speaker has about the problem, often ideas that are not found in that speaker's talk. But gesture can do more than reflect ideas-it can also change them. In this sense, gesture behaves like any other action; both gesture and action on objects facilitate learning problems on which training was given. However, only gesture promotes transferring the knowledge gained to problems that require generalization. Gesture is, in fact, a special kind of action in that it represents the world rather than directly manipulating the world (gesture does not move objects around). The mechanisms by which gesture and action promote learning may therefore differ-gesture is able to highlight components of an action that promote abstract learning while leaving out details that could tie learning to a specific context. Because it is both an action and a representation, gesture can serve as a bridge between the two and thus be a powerful tool for learning abstract ideas.

  17. Hand use and gestural communication in chimpanzees (Pan troglodytes).

    Science.gov (United States)

    Hopkins, W D; Leavens, D A

    1998-03-01

    Hand use in gestural communication was examined in 115 captive chimpanzees (Pan troglodytes). Hand use was measured in subjects while they gestured to food placed out of their reach. The distribution of hand use was examined in relation to sex, age, rearing history, gesture type, and whether the subjects vocalized while gesturing. Overall, significantly more chimpanzees, especially females and adults, gestured with their right than with their left hand. Foods begs were more lateralized to the right hand than pointing, and a greater prevalence of right-hand gesturing was found in subjects who simultaneously vocalized than those who did not. Taken together, these data suggest that referential, intentional communicative behaviors, in the form of gestures, are lateralized to the left hemisphere in chimpanzees.

  18. Humanoid Upper Torso Complexity for Displaying Gestures

    Directory of Open Access Journals (Sweden)

    Robert Richardson

    2008-11-01

    Full Text Available Body language is an important part of human-to-human communication; therefore body language in humanoid robots is very important for successful communication and social interaction with humans. The number of degrees of freedom (d.o.f necessary to achieve realistic body language in robots has been investigated. Using animation, three robots were simulated performing body language gestures; the complex model was given 25 d.o.f, the simplified model 18 d.o.f and the basic model 10 d.o.f. A subjective survey was created online using these animations, to obtain people's opinions on the realism of the gestures and to see if they could recognise the emotions portrayed. It was concluded that the basic system was the least realistic, complex system the most realistic, and the simplified system was only slightly less realistic than the human. Modular robotic joints were then fabricated so that the gestures could be implemented experimentally. The experimental results demonstrate that through simplification of the required degrees of freedom, the gestures can be experimentally reproduced.

  19. A Prelinguistic Gestural Universal of Human Communication

    Science.gov (United States)

    Liszkowski, Ulf; Brown, Penny; Callaghan, Tara; Takada, Akira; de Vos, Conny

    2012-01-01

    Several cognitive accounts of human communication argue for a language-independent, prelinguistic basis of human communication and language. The current study provides evidence for the universality of a prelinguistic gestural basis for human communication. We used a standardized, semi-natural elicitation procedure in seven very different cultures…

  20. A Brief Overview of Gesture Control Architectures

    Directory of Open Access Journals (Sweden)

    Gheorghe Gîlcă

    2014-12-01

    Full Text Available This papers deals with a detailed study of the literature about artificial vision systems and the applications where they can be used, such as: gesture interpretation for robot control, telephone control and the video control as well as presenting the structure of two vision systems: one for face recognition and the second to achieve the detection of multiple-touch finger.

  1. The Authentic Teacher: Gestures of Behavior.

    Science.gov (United States)

    Shimabukuro, Gini

    1998-01-01

    Stresses the importance for Catholic school educators to reveal the Christian message through every gesture of behavior and foster an experiential faith in students' lives. States that this demands a great deal of skill, knowledge, and self-awareness on the teacher's part, and requires self-esteem, authentic caring, humility, and communication…

  2. Grids and Gestures: A Comics Making Exercise

    Science.gov (United States)

    Sousanis, Nick

    2015-01-01

    Grids and Gestures is an exercise intended to offer participants insight into a comics maker's decision-making process for composing the entire page through the hands-on activity of making an abstract comic. It requires no prior drawing experience and serves to help reexamine what it means to draw. In addition to a description of how to proceed…

  3. The Gestural Theory of Language Origins

    Science.gov (United States)

    Armstrong, David F.

    2008-01-01

    The idea that iconic visible gesture had something to do with the origin of language, particularly speech, is a frequent element in speculation about this phenomenon and appears early in its history. Socrates hypothesizes about the origins of Greek words in Plato's satirical dialogue, "Cratylus", and his speculation includes a possible…

  4. Gesture-speech integration in children with specific language impairment.

    Science.gov (United States)

    Mainela-Arnold, Elina; Alibali, Martha W; Hostetter, Autumn B; Evans, Julia L

    2014-11-01

    Previous research suggests that speakers are especially likely to produce manual communicative gestures when they have relative ease in thinking about the spatial elements of what they are describing, paired with relative difficulty organizing those elements into appropriate spoken language. Children with specific language impairment (SLI) exhibit poor expressive language abilities together with within-normal-range nonverbal IQs. This study investigated whether weak spoken language abilities in children with SLI influence their reliance on gestures to express information. We hypothesized that these children would rely on communicative gestures to express information more often than their age-matched typically developing (TD) peers, and that they would sometimes express information in gestures that they do not express in the accompanying speech. Participants were 15 children with SLI (aged 5;6-10;0) and 18 age-matched TD controls. Children viewed a wordless cartoon and retold the story to a listener unfamiliar with the story. Children's gestures were identified and coded for meaning using a previously established system. Speech-gesture combinations were coded as redundant if the information conveyed in speech and gesture was the same, and non-redundant if the information conveyed in speech was different from the information conveyed in gesture. Children with SLI produced more gestures than children in the TD group; however, the likelihood that speech-gesture combinations were non-redundant did not differ significantly across the SLI and TD groups. In both groups, younger children were significantly more likely to produce non-redundant speech-gesture combinations than older children. The gesture-speech integration system functions similarly in children with SLI and TD, but children with SLI rely more on gesture to help formulate, conceptualize or express the messages they want to convey. This provides motivation for future research examining whether interventions

  5. Using the Hands to Represent Objects in Space: Gesture as a Substrate for Signed Language Acquisition.

    Science.gov (United States)

    Janke, Vikki; Marshall, Chloë R

    2017-01-01

    An ongoing issue of interest in second language research concerns what transfers from a speaker's first language to their second. For learners of a sign language, gesture is a potential substrate for transfer. Our study provides a novel test of gestural production by eliciting silent gesture from novices in a controlled environment. We focus on spatial relationships, which in sign languages are represented in a very iconic way using the hands, and which one might therefore predict to be easy for adult learners to acquire. However, a previous study by Marshall and Morgan (2015) revealed that this was only partly the case: in a task that required them to express the relative locations of objects, hearing adult learners of British Sign Language (BSL) could represent objects' locations and orientations correctly, but had difficulty selecting the correct handshapes to represent the objects themselves. If hearing adults are indeed drawing upon their gestural resources when learning sign languages, then their difficulties may have stemmed from their having in manual gesture only a limited repertoire of handshapes to draw upon, or, alternatively, from having too broad a repertoire. If the first hypothesis is correct, the challenge for learners is to extend their handshape repertoire, but if the second is correct, the challenge is instead to narrow down to the handshapes appropriate for that particular sign language. 30 sign-naïve hearing adults were tested on Marshall and Morgan's task. All used some handshapes that were different from those used by native BSL signers and learners, and the set of handshapes used by the group as a whole was larger than that employed by native signers and learners. Our findings suggest that a key challenge when learning to express locative relations might be reducing from a very large set of gestural resources, rather than supplementing a restricted one, in order to converge on the conventionalized classifier system that forms part of the

  6. Hand movements with a phase structure and gestures that depict action stem from a left hemispheric system of conceptualization.

    Science.gov (United States)

    Helmich, I; Lausberg, H

    2014-10-01

    The present study addresses the previously discussed controversy on the contribution of the right and left cerebral hemispheres to the production and conceptualization of spontaneous hand movements and gestures. Although it has been shown that each hemisphere contains the ability to produce hand movements, results of left hemispherically lateralized motor functions challenge the view of a contralateral hand movement production system. To examine hemispheric specialization in hand movement and gesture production, ten right-handed participants were tachistoscopically presented pictures of everyday life actions. The participants were asked to demonstrate with their hands, but without speaking what they had seen on the drawing. Two independent blind raters evaluated the videotaped hand movements and gestures employing the Neuropsychological Gesture Coding System. The results showed that the overall frequency of right- and left-hand movements is equal independent of stimulus lateralization. When hand movements were analyzed considering their Structure, the presentation of the action stimuli to the left hemisphere resulted in more hand movements with a phase structure than the presentation to the right hemisphere. Furthermore, the presentation to the left hemisphere resulted in more right and left-hand movements with a phase structure, whereas the presentation to the right hemisphere only increased contralateral left-hand movements with a phase structure as compared to hand movements without a phase structure. Gestures that depict action were primarily displayed in response to stimuli presented in the right visual field than in the left one. The present study shows that both hemispheres possess the faculty to produce hand movements in response to action stimuli. However, the left hemisphere dominates the production of hand movements with a phase structure and gestures that depict action. We therefore conclude that hand movements with a phase structure and gestures that

  7. Gestures, but Not Meaningless Movements, Lighten Working Memory Load when Explaining Math

    Science.gov (United States)

    Cook, Susan Wagner; Yip, Terina Kuangyi; Goldin-Meadow, Susan

    2012-01-01

    Gesturing is ubiquitous in communication and serves an important function for listeners, who are able to glean meaningful information from the gestures they see. But gesturing also functions for speakers, whose own gestures reduce demands on their working memory. Here we ask whether gesture's beneficial effects on working memory stem from its…

  8. Meaningful gesture in monkeys? Investigating whether mandrills create social culture.

    Directory of Open Access Journals (Sweden)

    Mark E Laidre

    Full Text Available BACKGROUND: Human societies exhibit a rich array of gestures with cultural origins. Often these gestures are found exclusively in local populations, where their meaning has been crafted by a community into a shared convention. In nonhuman primates like African monkeys, little evidence exists for such culturally-conventionalized gestures. METHODOLOGY/PRINCIPAL FINDINGS: Here I report a striking gesture unique to a single community of mandrills (Mandrillus sphinx among nineteen studied across North America, Africa, and Europe. The gesture was found within a community of 23 mandrills where individuals old and young, female and male covered their eyes with their hands for periods which could exceed 30 min, often while simultaneously raising their elbow prominently into the air. This 'Eye covering' gesture has been performed within the community for a decade, enduring deaths, removals, and births, and it persists into the present. Differential responses to Eye covering versus controls suggested that the gesture might have a locally-respected meaning, potentially functioning over a distance to inhibit interruptions as a 'do not disturb' sign operates. CONCLUSIONS/SIGNIFICANCE: The creation of this gesture by monkeys suggests that the ability to cultivate shared meanings using novel manual acts may be distributed more broadly beyond the human species. Although logistically difficult with primates, the translocation of gesturers between communities remains critical to experimentally establishing the possible cultural origin and transmission of nonhuman gestures.

  9. Musical Shaping Gestures: Considerations about Terminology and Methodology

    Directory of Open Access Journals (Sweden)

    Elaine King

    2013-12-01

    Full Text Available Fulford and Ginsborg's investigation into non-verbal communication during music rehearsal-talk between performers with and without hearing impairments extends existing research in the field of gesture studies by contributing significantly to our understanding of musicians' physical gestures as well as opening up discussion about the relationship between speech, sign and gesture in discourse about music. Importantly, the authors weigh up the possibility of an emerging sign language about music. This commentary focuses on three key considerations in response to their paper: first, use of terminology in the study of gesture, specifically about 'musical shaping gestures' (MSGs; second, methodological issues about capturing physical gestures; and third, evaluation of the application of gesture research beyond the rehearsal context. While the difficulties of categorizing gestures in observational research are acknowledged, I indicate that the consistent application of terminology from outside and within the study is paramount. I also suggest that the classification of MSGs might be based upon a set of observed physical characteristics within a single gesture, including size, duration, speed, plane and handedness, leading towards an alternative taxonomy for interpreting these data. Finally, evaluation of the application of gesture research in education and performance arenas is provided.

  10. Gesture's role in speaking, learning, and creating language.

    Science.gov (United States)

    Goldin-Meadow, Susan; Alibali, Martha Wagner

    2013-01-01

    When speakers talk, they gesture. The goal of this review is to investigate the contribution that these gestures make to how we communicate and think. Gesture can play a role in communication and thought at many timespans. We explore, in turn, gesture's contribution to how language is produced and understood in the moment; its contribution to how we learn language and other cognitive skills; and its contribution to how language is created over generations, over childhood, and on the spot. We find that the gestures speakers produce when they talk are integral to communication and can be harnessed in a number of ways. (a) Gesture reflects speakers' thoughts, often their unspoken thoughts, and thus can serve as a window onto cognition. Encouraging speakers to gesture can thus provide another route for teachers, clinicians, interviewers, etc., to better understand their communication partners. (b) Gesture can change speakers' thoughts. Encouraging gesture thus has the potential to change how students, patients, witnesses, etc., think about a problem and, as a result, alter the course of learning, therapy, or an interchange. (c) Gesture provides building blocks that can be used to construct a language. By watching how children and adults who do not already have a language put those blocks together, we can observe the process of language creation. Our hands are with us at all times and thus provide researchers and learners with an ever-present tool for understanding how we talk and think.

  11. Releasing the constraints on aphasia therapy: the positive impact of gesture and multimodality treatments.

    Science.gov (United States)

    Rose, Miranda L

    2013-05-01

    There is a 40-year history of interest in the use of arm and hand gestures in treatments that target the reduction of aphasic linguistic impairment and compensatory methods of communication (Rose, 2006). Arguments for constraining aphasia treatment to the verbal modality have arisen from proponents of constraint-induced aphasia therapy (Pulvermüller et al., 2001). Confusion exists concerning the role of nonverbal treatments in treating people with aphasia. The central argument of this paper is that given the state of the empirical evidence and the strong theoretical accounts of modality interactions in human communication, gesture-based and multimodality aphasia treatments are at least as legitimate an option as constraint-based aphasia treatment. Theoretical accounts of modality interactions in human communication and the gesture production abilities of individuals with aphasia that are harnessed in treatments are reviewed. The negative effects on word retrieval of restricting gesture production are also reviewed, and an overview of the neurological architecture subserving language processing is provided as rationale for multimodality treatments. The evidence for constrained and unconstrained treatments is critically reviewed. Together, these data suggest that constraint treatments and multimodality treatments are equally efficacious, and there is limited support for constraining client responses to the spoken modality.

  12. 3D Hand Gesture Analysis through a Real-Time Gesture Search Engine

    Directory of Open Access Journals (Sweden)

    Shahrouz Yousefi

    2015-06-01

    Full Text Available 3D gesture recognition and tracking are highly desired features of interaction design in future mobile and smart environments. Specifically, in virtual/augmented reality applications, intuitive interaction with the physical space seems unavoidable and 3D gestural interaction might be the most effective alternative for the current input facilities such as touchscreens. In this paper, we introduce a novel solution for real-time 3D gesture-based interaction by finding the best match from an extremely large gesture database. This database includes images of various articulated hand gestures with the annotated 3D position/orientation parameters of the hand joints. Our unique matching algorithm is based on the hierarchical scoring of the low-level edge-orientation features between the query frames and database and retrieving the best match. Once the best match is found from the database in each moment, the pre-recorded 3D motion parameters can instantly be used for natural interaction. The proposed bare-hand interaction technology performs in real time with high accuracy using an ordinary camera.

  13. Human computer interaction using hand gestures

    CERN Document Server

    Premaratne, Prashan

    2014-01-01

    Human computer interaction (HCI) plays a vital role in bridging the 'Digital Divide', bringing people closer to consumer electronics control in the 'lounge'. Keyboards and mouse or remotes do alienate old and new generations alike from control interfaces. Hand Gesture Recognition systems bring hope of connecting people with machines in a natural way. This will lead to consumers being able to use their hands naturally to communicate with any electronic equipment in their 'lounge.' This monograph will include the state of the art hand gesture recognition approaches and how they evolved from their inception. The author would also detail his research in this area for the past 8 years and how the future might turn out to be using HCI. This monograph will serve as a valuable guide for researchers (who would endeavour into) in the world of HCI.

  14. Gestures recognition based on wavelet and LLE

    International Nuclear Information System (INIS)

    Ai, Qingsong; Liu, Quan; Lu, Ying; Yuan, Tingting

    2013-01-01

    Wavelet analysis is a time–frequency, non-stationary method while the largest Lyapunov exponent (LLE) is used to judge the non-linear characteristic of systems. Because surface electromyography signal (SEMGS) is a complex signal that is characterized by non-stationary and non-linear properties. This paper combines wavelet coefficient and LLE together as the new feature of SEMGS. The proposed method not only reflects the non-stationary and non-linear characteristics of SEMGS, but also is suitable for its classification. Then, the BP (back propagation) neural network is employed to implement the identification of six gestures (fist clench, fist extension, wrist extension, wrist flexion, radial deviation, ulnar deviation). The experimental results indicate that based on the proposed method, the identification of these six gestures can reach an average rate of 97.71 %.

  15. Working memory for meaningless manual gestures.

    Science.gov (United States)

    Rudner, Mary

    2015-03-01

    Effects on working memory performance relating to item similarity have been linked to prior categorisation of representations in long-term memory. However, there is evidence from gesture processing that this link may not be obligatory. The present study investigated whether working memory for incidentally generated meaningless manual gestures is influenced by formational similarity and whether this effect is modulated by working-memory load. Results showed that formational similarity did lower performance, demonstrating that similarity effects are not dependent on prior categorisation. However, this effect was only found when working-memory load was low, supporting a flexible resource allocation model according to which it is the quality rather than quantity of working memory representations that determines performance. This interpretation is in line with proposals suggesting language modality specific allocation of resources in working memory. (PsycINFO Database Record (c) 2015 APA, all rights reserved).

  16. Recognition of Gestures using Artifical Neural Network

    Directory of Open Access Journals (Sweden)

    Marcel MORE

    2013-12-01

    Full Text Available Sensors for motion measurements are now becoming more widespread. Thanks to their parameters and affordability they are already used not only in the professional sector, but also in devices intended for daily use or entertainment. One of their applications is in control of devices by gestures. Systems that can determine type of gesture from measured motion have many uses. Some are for example in medical practice, but they are still more often used in devices such as cell phones, where they serve as a non-standard form of input. Today there are already several approaches for solving this problem, but building sufficiently reliable system is still a challenging task. In our project we are developing solution based on artificial neural network. In difference to other solutions, this one doesn’t require building model for each measuring system and thus it can be used in combination with various sensors just with minimal changes in his structure.

  17. Spatial analogies pervade complex relational reasoning: Evidence from spontaneous gestures.

    Science.gov (United States)

    Cooperrider, Kensy; Gentner, Dedre; Goldin-Meadow, Susan

    2016-01-01

    How do people think about complex phenomena like the behavior of ecosystems? Here we hypothesize that people reason about such relational systems in part by creating spatial analogies, and we explore this possibility by examining spontaneous gestures. In two studies, participants read a written lesson describing positive and negative feedback systems and then explained the differences between them. Though the lesson was highly abstract and people were not instructed to gesture, people produced spatial gestures in abundance during their explanations. These gestures used space to represent simple abstract relations (e.g., increase ) and sometimes more complex relational structures (e.g., negative feedback ). Moreover, over the course of their explanations, participants' gestures often cohered into larger analogical models of relational structure. Importantly, the spatial ideas evident in the hands were largely unaccompanied by spatial words. Gesture thus suggests that spatial analogies are pervasive in complex relational reasoning, even when language does not.

  18. Virtual sculpting with advanced gestural interface

    OpenAIRE

    Kılıboz, Nurettin Çağrı

    2013-01-01

    Ankara : The Department of Computer Engineering and the Graduate School of Engineering and Science of Bilkent University, 2013. Thesis (Master's) -- Bilkent University, 2013. Includes bibliographical references leaves 54-58. In this study, we propose a virtual reality application that can be utilized to design preliminary/conceptual models similar to real world clay sculpting. The proposed system makes use of the innovative gestural interface that enhances the experience of...

  19. Distinguishing the communicative functions of gestures

    DEFF Research Database (Denmark)

    Jokinen, Kristiina; Navarretta, Costanza; Paggio, Patrizia

    2008-01-01

    This paper deals with the results of a machine learning experiment conducted on annotated gesture data from two case studies (Danish and Estonian). The data concern mainly facial displays, that are annotated with attributes relating to shape and dynamics, as well as communicative function....... The results of the experiments show that the granularity of the attributes used seems appropriate for the task of distinguishing the desired communicative functions. This is a promising result in view of a future automation of the annotation task....

  20. Towards successful user interaction with systems: focusing on user-derived gestures for smart home systems.

    Science.gov (United States)

    Choi, Eunjung; Kwon, Sunghyuk; Lee, Donghun; Lee, Hogin; Chung, Min K

    2014-07-01

    Various studies that derived gesture commands from users have used the frequency ratio to select popular gestures among the users. However, the users select only one gesture from a limited number of gestures that they could imagine during an experiment, and thus, the selected gesture may not always be the best gesture. Therefore, two experiments including the same participants were conducted to identify whether the participants maintain their own gestures after observing other gestures. As a result, 66% of the top gestures were different between the two experiments. Thus, to verify the changed gestures between the two experiments, a third experiment including another set of participants was conducted, which showed that the selected gestures were similar to those from the second experiment. This finding implies that the method of using the frequency in the first step does not necessarily guarantee the popularity of the gestures. Copyright © 2014 Elsevier Ltd and The Ergonomics Society. All rights reserved.

  1. Does training with beat gestures favour children's narrative discourse abilities?

    OpenAIRE

    Vilà Giménez, Ingrid

    2016-01-01

    There is consensus evidence that gestures and prosody are important precursors of children’s early language abilities and development. Previous literature has investigated the beneficial role of beat gestures in the recall of information by preschoolers (Igualada, Esteve-Gibert, & Prieto, under review; Austin & Sweller, 2014). However, to our knowledge, little is known about whether the use of beat gestures can promote children’s later linguistic abilities and specifically whether training wi...

  2. Effects of the restriction of hand gestures on disfluency.

    OpenAIRE

    Finlayson, Sheena; Forrest, Victoria; Lickley, Robin; Beck, Janet M

    2003-01-01

    This paper describes an experimental pilot study of disfluency and gesture rates in spontaneous speech where speakers perform a communication task in three conditions: hands free, one arm immobilized, both arms immobilized. Previous work suggests that the restriction of the ability to gesture can have an impact on the fluency of speech. In particular, it has been found that the inability to produce iconic gestures, which depict actions and objects, results in a higher rate of disfluency. Mode...

  3. Beat gestures and prosodic prominence: impact on learning

    OpenAIRE

    Kushch, Olga

    2018-01-01

    Previous research has shown that gestures are beneficial for language learning. This doctoral thesis centers on the effects of beat gestures– i.e., hand and arm gestures that are typically associated with prosodically prominent positions in speech - on such processes. Little is known about how the two central properties of beat gestures, namely how they mark both information focus and rhythmic positions in speech, can be beneficial for learning either a first or a second language. The main go...

  4. GESTURE-VERBAL UTTERANCES FROM THE COGNITIVE PERSPECTIVE

    OpenAIRE

    Martynyuk, Alla

    2016-01-01

    The article develops the idea of speech and gesture as an integral system of generation of meaning viewing an individual’s cognitive system as a dynamic, evolving semantic lattice organising semantic items of propositional and imagistic modes around a core meaning: linguistic items (propositions) are linked to ideas, concepts and beliefs as well as to specific feelings, mental states, images of gestures and stereotypic patterns of behaviour. Since gesture and speech are equally engaged in gen...

  5. Adult Gesture in Collaborative Mathematics Reasoning in Different Ages

    Science.gov (United States)

    Noto, M. S.; Harisman, Y.; Harun, L.; Amam, A.; Maarif, S.

    2017-09-01

    This article describes the case study on postgraduate students by using descriptive method. A problem is designed to facilitate the reasoning in the topic of Chi-Square test. The problem was given to two male students with different ages to investigate the gesture pattern and it will be related to their reasoning process. The indicators in reasoning problem can obtain the conclusion of analogy and generalization, and arrange the conjectures. This study refers to some questions—whether unique gesture is for every individual or to identify the pattern of the gesture used by the students with different ages. Reasoning problem was employed to collect the data. Two students were asked to collaborate to reason the problem. The discussion process recorded in using video tape to observe the gestures. The video recorded are explained clearly in this writing. Prosodic cues such as time, conversation text, gesture that appears, might help in understanding the gesture. The purpose of this study is to investigate whether different ages influences the maturity in collaboration observed from gesture perspective. The finding of this study shows that age is not a primary factor that influences the gesture in that reasoning process. In this case, adult gesture or gesture performed by order student does not show that he achieves, maintains, and focuses on the problem earlier on. Adult gesture also does not strengthen and expand the meaning if the student’s words or the language used in reasoning is not familiar for younger student. Adult gesture also does not affect cognitive uncertainty in mathematics reasoning. The future research is suggested to take more samples to find the consistency from that statement.

  6. Patients with hippocampal amnesia successfully integrate gesture and speech.

    Science.gov (United States)

    Hilverman, Caitlin; Clough, Sharice; Duff, Melissa C; Cook, Susan Wagner

    2018-06-19

    During conversation, people integrate information from co-speech hand gestures with information in spoken language. For example, after hearing the sentence, "A piece of the log flew up and hit Carl in the face" while viewing a gesture directed at the nose, people tend to later report that the log hit Carl in the nose (information only in gesture) rather than in the face (information in speech). The cognitive and neural mechanisms that support the integration of gesture with speech are unclear. One possibility is that the hippocampus - known for its role in relational memory and information integration - is necessary for integrating gesture and speech. To test this possibility, we examined how patients with hippocampal amnesia and healthy and brain-damaged comparison participants express information from gesture in a narrative retelling task. Participants watched videos of an experimenter telling narratives that included hand gestures that contained supplementary information. Participants were asked to retell the narratives and their spoken retellings were assessed for the presence of information from gesture. For features that had been accompanied by supplementary gesture, patients with amnesia retold fewer of these features overall and fewer retellings that matched the speech from the narrative. Yet their retellings included features that contained information that had been present uniquely in gesture in amounts that were not reliably different from comparison groups. Thus, a functioning hippocampus is not necessary for gesture-speech integration over short timescales. Providing unique information in gesture may enhance communication for individuals with declarative memory impairment, possibly via non-declarative memory mechanisms. Copyright © 2018. Published by Elsevier Ltd.

  7. A prelinguistic gestural universal of human communication.

    Science.gov (United States)

    Liszkowski, Ulf; Brown, Penny; Callaghan, Tara; Takada, Akira; de Vos, Conny

    2012-01-01

    Several cognitive accounts of human communication argue for a language-independent, prelinguistic basis of human communication and language. The current study provides evidence for the universality of a prelinguistic gestural basis for human communication. We used a standardized, semi-natural elicitation procedure in seven very different cultures around the world to test for the existence of preverbal pointing in infants and their caregivers. Results were that by 10-14 months of age, infants and their caregivers pointed in all cultures in the same basic situation with similar frequencies and the same proto-typical morphology of the extended index finger. Infants' pointing was best predicted by age and caregiver pointing, but not by cultural group. Further analyses revealed a strong relation between the temporal unfolding of caregivers' and infants' pointing events, uncovering a structure of early prelinguistic gestural conversation. Findings support the existence of a gestural, language-independent universal of human communication that forms a culturally shared, prelinguistic basis for diversified linguistic communication. Copyright © 2012 Cognitive Science Society, Inc.

  8. Gestures, vocalizations, and memory in language origins.

    Science.gov (United States)

    Aboitiz, Francisco

    2012-01-01

    THIS ARTICLE DISCUSSES THE POSSIBLE HOMOLOGIES BETWEEN THE HUMAN LANGUAGE NETWORKS AND COMPARABLE AUDITORY PROJECTION SYSTEMS IN THE MACAQUE BRAIN, IN AN ATTEMPT TO RECONCILE TWO EXISTING VIEWS ON LANGUAGE EVOLUTION: one that emphasizes hand control and gestures, and the other that emphasizes auditory-vocal mechanisms. The capacity for language is based on relatively well defined neural substrates whose rudiments have been traced in the non-human primate brain. At its core, this circuit constitutes an auditory-vocal sensorimotor circuit with two main components, a "ventral pathway" connecting anterior auditory regions with anterior ventrolateral prefrontal areas, and a "dorsal pathway" connecting auditory areas with parietal areas and with posterior ventrolateral prefrontal areas via the arcuate fasciculus and the superior longitudinal fasciculus. In humans, the dorsal circuit is especially important for phonological processing and phonological working memory, capacities that are critical for language acquisition and for complex syntax processing. In the macaque, the homolog of the dorsal circuit overlaps with an inferior parietal-premotor network for hand and gesture selection that is under voluntary control, while vocalizations are largely fixed and involuntary. The recruitment of the dorsal component for vocalization behavior in the human lineage, together with a direct cortical control of the subcortical vocalizing system, are proposed to represent a fundamental innovation in human evolution, generating an inflection point that permitted the explosion of vocal language and human communication. In this context, vocal communication and gesturing have a common history in primate communication.

  9. Angle-of-arrival-based gesture recognition using ultrasonic multi-frequency signals

    KAUST Repository

    Chen, Hui

    2017-11-02

    Hand gestures are tools for conveying information, expressing emotion, interacting with electronic devices or even serving disabled people as a second language. A gesture can be recognized by capturing the movement of the hand, in real time, and classifying the collected data. Several commercial products such as Microsoft Kinect, Leap Motion Sensor, Synertial Gloves and HTC Vive have been released and new solutions have been proposed by researchers to handle this task. These systems are mainly based on optical measurements, inertial measurements, ultrasound signals and radio signals. This paper proposes an ultrasonic-based gesture recognition system using AOA (Angle of Arrival) information of ultrasonic signals emitted from a wearable ultrasound transducer. The 2-D angles of the moving hand are estimated using multi-frequency signals captured by a fixed receiver array. A simple redundant dictionary matching classifier is designed to recognize gestures representing the numbers from `0\\' to `9\\' and compared with a neural network classifier. Average classification accuracies of 95.5% and 94.4% are obtained, respectively, using the two classification methods.

  10. Gliding and Saccadic Gaze Gesture Recognition in Real Time

    DEFF Research Database (Denmark)

    Rozado, David; San Agustin, Javier; Rodriguez, Francisco

    2012-01-01

    , and their corresponding real-time recognition algorithms, Hierarchical Temporal Memory networks and the Needleman-Wunsch algorithm for sequence alignment. Our results show how a specific combination of gaze gesture modality, namely saccadic gaze gestures, and recognition algorithm, Needleman-Wunsch, allows for reliable...... usage of intentional gaze gestures to interact with a computer with accuracy rates of up to 98% and acceptable completion speed. Furthermore, the gesture recognition engine does not interfere with otherwise standard human-machine gaze interaction generating therefore, very low false positive rates...

  11. Gesturing more diminishes recall of abstract words when gesture is allowed and concrete words when it is taboo.

    Science.gov (United States)

    Matthews-Saugstad, Krista M; Raymakers, Erik P; Kelty-Stephen, Damian G

    2017-07-01

    Gesture during speech can promote or diminish recall for conversation content. We explored effects of cognitive load on this relationship, manipulating it at two scales: individual-word abstractness and social constraints to prohibit gestures. Prohibited gestures can diminish recall but more so for abstract-word recall. Insofar as movement planning adds to cognitive load, movement amplitude may moderate gesture effects on memory, with greater permitted- and prohibited-gesture movements reducing abstract-word recall and concrete-word recall, respectively. We tested these effects in a dyadic game in which 39 adult participants described words to confederates without naming the word or five related words. Results supported our expectations and indicated that memory effects of gesturing depend on social, cognitive, and motoric aspects of discourse.

  12. Give me a hand: Differential effects of gesture type in guiding young children's problem-solving

    OpenAIRE

    Vallotton, Claire; Fusaro, Maria; Hayden, Julia; Decker, Kalli; Gutowski, Elizabeth

    2015-01-01

    Adults’ gestures support children's learning in problem-solving tasks, but gestures may be differentially useful to children of different ages, and different features of gestures may make them more or less useful to children. The current study investigated parents’ use of gestures to support their young children (1.5 – 6 years) in a block puzzle task (N = 126 parent-child dyads), and identified patterns in parents’ gesture use indicating different gestural strategies. Further, we examined the...

  13. Foundational Issues in Touch-Screen Stroke Gesture Design - An Integrative Review

    OpenAIRE

    Zhai , Shumin; Kristensson , Per Ola; Appert , Caroline; Andersen , Tue Haste; Cao , Xiang

    2012-01-01

    International audience; The potential for using stroke gestures to enter, retrieve and select commands and text has been recently unleashed by the popularity of touchscreen devices. This monograph provides a state-of-the-art inte- grative review of a body of human-computer interaction research on stroke gestures. It begins with an analysis of the design dimensions of stroke gestures as an interaction medium. The analysis classifies gestures into analogue versus abstract gestures, gestures for...

  14. Archetypal Gesture and Everyday Gesture: a fundamental binomial in Delsartean theory

    Directory of Open Access Journals (Sweden)

    Elena Randi

    2012-11-01

    Full Text Available This text presents François Delsarte’s system from a historical-exploratory viewpoint, focusing on some particular aspects of the work of the French master and the interpretation of his work by some of his main disciples. The article describes the status of the body and its importance in the Delsarte system, taking the notions of archetypal gesture and everyday gesture as the bases of this system. Indeed, the text highlights both historical facts obtained from the Delsarte archive, and arguments questioning the authorship of exercises attributed to Delsarte, which, according to the text, may have been created by his students.

  15. Beat gestures help preschoolers recall and comprehend discourse information.

    Science.gov (United States)

    Llanes-Coromina, Judith; Vilà-Giménez, Ingrid; Kushch, Olga; Borràs-Comes, Joan; Prieto, Pilar

    2018-08-01

    Although the positive effects of iconic gestures on word recall and comprehension by children have been clearly established, less is known about the benefits of beat gestures (rhythmic hand/arm movements produced together with prominent prosody). This study investigated (a) whether beat gestures combined with prosodic information help children recall contrastively focused words as well as information related to those words in a child-directed discourse (Experiment 1) and (b) whether the presence of beat gestures helps children comprehend a narrative discourse (Experiment 2). In Experiment 1, 51 4-year-olds were exposed to a total of three short stories with contrastive words presented in three conditions, namely with prominence in both speech and gesture, prominence in speech only, and nonprominent speech. Results of a recall task showed that (a) children remembered more words when exposed to prominence in both speech and gesture than in either of the other two conditions and that (b) children were more likely to remember information related to those words when the words were associated with beat gestures. In Experiment 2, 55 5- and 6-year-olds were presented with six narratives with target items either produced with prosodic prominence but no beat gestures or produced with both prosodic prominence and beat gestures. Results of a comprehension task demonstrated that stories told with beat gestures were comprehended better by children. Together, these results constitute evidence that beat gestures help preschoolers not only to recall discourse information but also to comprehend it. Copyright © 2018 Elsevier Inc. All rights reserved.

  16. From mouth to hand: gesture, speech, and the evolution of right-handedness.

    Science.gov (United States)

    Corballis, Michael C

    2003-04-01

    The strong predominance of right-handedness appears to be a uniquely human characteristic, whereas the left-cerebral dominance for vocalization occurs in many species, including frogs, birds, and mammals. Right-handedness may have arisen because of an association between manual gestures and vocalization in the evolution of language. I argue that language evolved from manual gestures, gradually incorporating vocal elements. The transition may be traced through changes in the function of Broca's area. Its homologue in monkeys has nothing to do with vocal control, but contains the so-called "mirror neurons," the code for both the production of manual reaching movements and the perception of the same movements performed by others. This system is bilateral in monkeys, but predominantly left-hemispheric in humans, and in humans is involved with vocalization as well as manual actions. There is evidence that Broca's area is enlarged on the left side in Homo habilis, suggesting that a link between gesture and vocalization may go back at least two million years, although other evidence suggests that speech may not have become fully autonomous until Homo sapiens appeared some 170,000 years ago, or perhaps even later. The removal of manual gesture as a necessary component of language may explain the rapid advance of technology, allowing late migrations of Homo sapiens from Africa to replace all other hominids in other parts of the world, including the Neanderthals in Europe and Homo erectus in Asia. Nevertheless, the long association of vocalization with manual gesture left us a legacy of right-handedness.

  17. Gesture, Landscape and Embrace: A Phenomenological Analysis of ...

    African Journals Online (AJOL)

    The 'radical reflection' on the 'flesh of the world' to which this analysis aspires in turn bears upon the general field of gestural reciprocities and connections, providing the insight that intimate gestures of the flesh, such as the embrace, are primordial attunements, motions of rhythm and reciprocity, that emanate from the world ...

  18. Comprehension of iconic gestures by chimpanzees and human children.

    Science.gov (United States)

    Bohn, Manuel; Call, Josep; Tomasello, Michael

    2016-02-01

    Iconic gestures-communicative acts using hand or body movements that resemble their referent-figure prominently in theories of language evolution and development. This study contrasted the abilities of chimpanzees (N=11) and 4-year-old human children (N=24) to comprehend novel iconic gestures. Participants learned to retrieve rewards from apparatuses in two distinct locations, each requiring a different action. In the test, a human adult informed the participant where to go by miming the action needed to obtain the reward. Children used the iconic gestures (more than arbitrary gestures) to locate the reward, whereas chimpanzees did not. Some children also used arbitrary gestures in the same way, but only after they had previously shown comprehension for iconic gestures. Over time, chimpanzees learned to associate iconic gestures with the appropriate location faster than arbitrary gestures, suggesting at least some recognition of the iconicity involved. These results demonstrate the importance of iconicity in referential communication. Copyright © 2015 Elsevier Inc. All rights reserved.

  19. Communicative Effectiveness of Pantomime Gesture in People with Aphasia

    Science.gov (United States)

    Rose, Miranda L.; Mok, Zaneta; Sekine, Kazuki

    2017-01-01

    Background: Human communication occurs through both verbal and visual/motoric modalities. Simultaneous conversational speech and gesture occurs across all cultures and age groups. When verbal communication is compromised, more of the communicative load can be transferred to the gesture modality. Although people with aphasia produce meaning-laden…

  20. Does gesture add to the comprehensibility of people with aphasia?

    NARCIS (Netherlands)

    van Nispen, Karin; Sekine, Kazuki; Rose, Miranda; Ferré, Gaëlle; Tutton, Mark

    2015-01-01

    Gesture can convey information co-occurring with and in the absence of speech. As such, it seems a useful strategy for people with aphasia (PWA) to compensate for their impaired speech. To find out whether gestures used by PWA add to the comprehensibility of their communication we looked at the

  1. Enhancement of naming in nonfluent aphasia through gesture.

    Science.gov (United States)

    Hanlon, R E; Brown, J W; Gerstman, L J

    1990-02-01

    In a number of studies that have examined the gestural disturbance in aphasia and the utility of gestural interventions in aphasia therapy, a variable degree of facilitation of verbalization during gestural activity has been reported. The present study examined the effect of different unilateral gestural movements on simultaneous oral-verbal expression, specifically naming to confrontation. It was hypothesized that activation of the phylogenetically older proximal motor system of the hemiplegic right arm in the execution of a communicative but nonrepresentational pointing gesture would have a facilitatory effect on naming ability. Twenty-four aphasic patients, representing five aphasic subtypes, including Broca's, Transcortical Motor, Anomic, Global, and Wernicke's aphasics were assessed under three gesture/naming conditions. The findings indicated that gestures produced through activation of the proximal (shoulder) musculature of the right paralytic limb differentially facilitated naming performance in the nonfluent subgroup, but not in the Wernicke's aphasics. These findings may be explained on the view that functional activation of the archaic proximal motor system of the hemiplegic limb, in the execution of a communicative gesture, permits access to preliminary stages in the formative process of the anterior action microgeny, which ultimately emerges in vocal articulation.

  2. Gestures as Semiotic Resources in the Mathematics Classroom

    Science.gov (United States)

    Arzarello, Ferdinando; Paola, Domingo; Robutti, Ornella; Sabena, Cristina

    2009-01-01

    In this paper, we consider gestures as part of the resources activated in the mathematics classroom: speech, inscriptions, artifacts, etc. As such, gestures are seen as one of the semiotic tools used by students and teacher in mathematics teaching-learning. To analyze them, we introduce a suitable model, the "semiotic bundle." It allows focusing…

  3. The Effects of Prohibiting Gestures on Children's Lexical Retrieval Ability

    Science.gov (United States)

    Pine, Karen J.; Bird, Hannah; Kirk, Elizabeth

    2007-01-01

    Two alternative accounts have been proposed to explain the role of gestures in thinking and speaking. The Information Packaging Hypothesis (Kita, 2000) claims that gestures are important for the conceptual packaging of information before it is coded into a linguistic form for speech. The Lexical Retrieval Hypothesis (Rauscher, Krauss & Chen, 1996)…

  4. Mothers' labeling responses to infants' gestures predict vocabulary outcomes.

    Science.gov (United States)

    Olson, Janet; Masur, Elise Frank

    2015-11-01

    Twenty-nine infants aged 1;1 and their mothers were videotaped while interacting with toys for 18 minutes. Six experimental stimuli were presented to elicit infant communicative bids in two communicative intent contexts - proto-declarative and proto-imperative. Mothers' verbal responses to infants' gestural and non-gestural communicative bids were coded for object and action labels. Relations between maternal labeling responses and infants' vocabularies at 1;1 and 1;5 were examined. Mothers' labeling responses to infants' gestural communicative bids were concurrently and predictively related to infants' vocabularies, whereas responses to non-gestural communicative bids were not. Mothers' object labeling following gestures in the proto-declarative context mediated the association from infants' gesturing in the proto-declarative context to concurrent noun lexicons and was the strongest predictor of subsequent noun lexicons. Mothers' action labeling after infants' gestural bids in the proto-imperative context predicted infants' acquisition of action words at 1;5. Findings show that mothers' responsive labeling explain specific relations between infants' gestures and their vocabulary development.

  5. Gesture and Identity in the Teaching and Learning of Italian

    Science.gov (United States)

    Peltier, Ilaria Nardotto; McCafferty, Steven G.

    2010-01-01

    This study investigated the use of mimetic gestures of identity by foreign language teachers of Italian and their students in college classes as a form of meaning-making. All four of the teachers were found to use a variety of Italian gestures as a regular aspect of their teaching and presentation of self. Students and teachers also were found to…

  6. A Hierarchical Model for Continuous Gesture Recognition Using Kinect

    DEFF Research Database (Denmark)

    Jensen, Søren Kejser; Moesgaard, Christoffer; Nielsen, Christoffer Samuel

    2013-01-01

    Human gesture recognition is an area, which has been studied thoroughly in recent years,and close to100% recognition rates in restricted environments have been achieved, often either with single separated gestures in the input stream, or with computationally intensive systems. The results are unf...

  7. Diagram, Gesture, Agency: Theorizing Embodiment in the Mathematics Classroom

    Science.gov (United States)

    de Freitas, Elizabeth; Sinclair, Nathalie

    2012-01-01

    In this paper, we use the work of philosopher Gilles Chatelet to rethink the gesture/diagram relationship and to explore the ways mathematical agency is constituted through it. We argue for a fundamental philosophical shift to better conceptualize the relationship between gesture and diagram, and suggest that such an approach might open up new…

  8. Body in Mind: How Gestures Empower Foreign Language Learning

    Science.gov (United States)

    Macedonia, Manuela; Knosche, Thomas R.

    2011-01-01

    It has previously been demonstrated that enactment (i.e., performing representative gestures during encoding) enhances memory for concrete words, in particular action words. Here, we investigate the impact of enactment on abstract word learning in a foreign language. We further ask if learning novel words with gestures facilitates sentence…

  9. Probing the Mental Representation of Gesture: Is Handwaving Spatial?

    Science.gov (United States)

    Wagner, Susan M.; Nusbaum, Howard; Goldin-Meadow, Susan

    2004-01-01

    What type of mental representation underlies the gestures that accompany speech? We used a dual-task paradigm to compare the demands gesturing makes on visuospatial and verbal working memories. Participants in one group remembered a string of letters (verbal working memory group) and those in a second group remembered a visual grid pattern…

  10. View Invariant Gesture Recognition using 3D Motion Primitives

    DEFF Research Database (Denmark)

    Holte, Michael Boelstoft; Moeslund, Thomas B.

    2008-01-01

    This paper presents a method for automatic recognition of human gestures. The method works with 3D image data from a range camera to achieve invariance to viewpoint. The recognition is based solely on motion from characteristic instances of the gestures. These instances are denoted 3D motion...

  11. Gestural acquisition in great apes: the Social Negotiation Hypothesis.

    Science.gov (United States)

    Pika, Simone; Fröhlich, Marlen

    2018-01-24

    Scientific interest in the acquisition of gestural signalling dates back to the heroic figure of Charles Darwin. More than a hundred years later, we still know relatively little about the underlying evolutionary and developmental pathways involved. Here, we shed new light on this topic by providing the first systematic, quantitative comparison of gestural development in two different chimpanzee (Pan troglodytes verus and Pan troglodytes schweinfurthii) subspecies and communities living in their natural environments. We conclude that the three most predominant perspectives on gestural acquisition-Phylogenetic Ritualization, Social Transmission via Imitation, and Ontogenetic Ritualization-do not satisfactorily explain our current findings on gestural interactions in chimpanzees in the wild. In contrast, we argue that the role of interactional experience and social exposure on gestural acquisition and communicative development has been strongly underestimated. We introduce the revised Social Negotiation Hypothesis and conclude with a brief set of empirical desiderata for instigating more research into this intriguing research domain.

  12. Barack Obama’s pauses and gestures in humorous speeches

    DEFF Research Database (Denmark)

    Navarretta, Costanza

    2017-01-01

    The main aim of this paper is to investigate speech pauses and gestures as means to engage the audience and present the humorous message in an effective way. The data consist of two speeches by the USA president Barack Obama at the 2011 and 2016 Annual White House Correspondents’ Association Dinner...... produced significantly more hand gestures in 2016 than in 2011. An analysis of the hand gestures produced by Barack Obama in two political speeches held at the United Nations in 2011 and 2016 confirms that the president produced significantly less communicative co-speech hand gestures during his speeches...... and they emphasise the speech segment which they follow or precede. We also found a highly significant correlation between Obama’s speech pauses and audience response. Obama produces numerous head movements, facial expressions and hand gestures and their functions are related to both discourse content and structure...

  13. Predicting an Individual’s Gestures from the Interlocutor’s Co-occurring Gestures and Related Speech

    DEFF Research Database (Denmark)

    Navarretta, Costanza

    2016-01-01

    to the prediction of gestures of the same type of the other subject. In this work, we also want to determine whether the speech segments to which these gestures are related to contribute to the prediction. The results of our pilot experiments show that a Naive Bayes classifier trained on the duration and shape...

  14. Conductor gestures influence evaluations of ensemble performance.

    Science.gov (United States)

    Morrison, Steven J; Price, Harry E; Smedley, Eric M; Meals, Cory D

    2014-01-01

    Previous research has found that listener evaluations of ensemble performances vary depending on the expressivity of the conductor's gestures, even when performances are otherwise identical. It was the purpose of the present study to test whether this effect of visual information was evident in the evaluation of specific aspects of ensemble performance: articulation and dynamics. We constructed a set of 32 music performances that combined auditory and visual information and were designed to feature a high degree of contrast along one of two target characteristics: articulation and dynamics. We paired each of four music excerpts recorded by a chamber ensemble in both a high- and low-contrast condition with video of four conductors demonstrating high- and low-contrast gesture specifically appropriate to either articulation or dynamics. Using one of two equivalent test forms, college music majors and non-majors (N = 285) viewed sixteen 30 s performances and evaluated the quality of the ensemble's articulation, dynamics, technique, and tempo along with overall expressivity. Results showed significantly higher evaluations for performances featuring high rather than low conducting expressivity regardless of the ensemble's performance quality. Evaluations for both articulation and dynamics were strongly and positively correlated with evaluations of overall ensemble expressivity.

  15. Conductor gestures influence evaluations of ensemble performance

    Directory of Open Access Journals (Sweden)

    Steven eMorrison

    2014-07-01

    Full Text Available Previous research has found that listener evaluations of ensemble performances vary depending on the expressivity of the conductor’s gestures, even when performances are otherwise identical. It was the purpose of the present study to test whether this effect of visual information was evident in the evaluation of specific aspects of ensemble performance, articulation and dynamics. We constructed a set of 32 music performances that combined auditory and visual information and were designed to feature a high degree of contrast along one of two target characteristics: articulation and dynamics. We paired each of four music excerpts recorded by a chamber ensemble in both a high- and low-contrast condition with video of four conductors demonstrating high- and low-contrast gesture specifically appropriate to either articulation or dynamics. Using one of two equivalent test forms, college music majors and nonmajors (N = 285 viewed sixteen 30-second performances and evaluated the quality of the ensemble’s articulation, dynamics, technique and tempo along with overall expressivity. Results showed significantly higher evaluations for performances featuring high rather than low conducting expressivity regardless of the ensemble’s performance quality. Evaluations for both articulation and dynamics were strongly and positively correlated with evaluations of overall ensemble expressivity.

  16. View invariant gesture recognition using the CSEMSwissRanger SR-2 camera

    DEFF Research Database (Denmark)

    Holte, Michael Boelstoft; Moeslund, Thomas B.; Fihl, Preben

    2008-01-01

    by a hysteresis bandpass filter. Gestures are represented by concatenating harmonic shape contexts over time. This representation allows for a view invariant matching of the gestures. The system is trained on gestures from one viewpoint and evaluated on gestures from other viewpoints. The results show...

  17. Depth camera-based 3D hand gesture controls with immersive tactile feedback for natural mid-air gesture interactions.

    Science.gov (United States)

    Kim, Kwangtaek; Kim, Joongrock; Choi, Jaesung; Kim, Junghyun; Lee, Sangyoun

    2015-01-08

    Vision-based hand gesture interactions are natural and intuitive when interacting with computers, since we naturally exploit gestures to communicate with other people. However, it is agreed that users suffer from discomfort and fatigue when using gesture-controlled interfaces, due to the lack of physical feedback. To solve the problem, we propose a novel complete solution of a hand gesture control system employing immersive tactile feedback to the user's hand. For this goal, we first developed a fast and accurate hand-tracking algorithm with a Kinect sensor using the proposed MLBP (modified local binary pattern) that can efficiently analyze 3D shapes in depth images. The superiority of our tracking method was verified in terms of tracking accuracy and speed by comparing with existing methods, Natural Interaction Technology for End-user (NITE), 3D Hand Tracker and CamShift. As the second step, a new tactile feedback technology with a piezoelectric actuator has been developed and integrated into the developed hand tracking algorithm, including the DTW (dynamic time warping) gesture recognition algorithm for a complete solution of an immersive gesture control system. The quantitative and qualitative evaluations of the integrated system were conducted with human subjects, and the results demonstrate that our gesture control with tactile feedback is a promising technology compared to a vision-based gesture control system that has typically no feedback for the user's gesture inputs. Our study provides researchers and designers with informative guidelines to develop more natural gesture control systems or immersive user interfaces with haptic feedback.

  18. Depth Camera-Based 3D Hand Gesture Controls with Immersive Tactile Feedback for Natural Mid-Air Gesture Interactions

    Directory of Open Access Journals (Sweden)

    Kwangtaek Kim

    2015-01-01

    Full Text Available Vision-based hand gesture interactions are natural and intuitive when interacting with computers, since we naturally exploit gestures to communicate with other people. However, it is agreed that users suffer from discomfort and fatigue when using gesture-controlled interfaces, due to the lack of physical feedback. To solve the problem, we propose a novel complete solution of a hand gesture control system employing immersive tactile feedback to the user’s hand. For this goal, we first developed a fast and accurate hand-tracking algorithm with a Kinect sensor using the proposed MLBP (modified local binary pattern that can efficiently analyze 3D shapes in depth images. The superiority of our tracking method was verified in terms of tracking accuracy and speed by comparing with existing methods, Natural Interaction Technology for End-user (NITE, 3D Hand Tracker and CamShift. As the second step, a new tactile feedback technology with a piezoelectric actuator has been developed and integrated into the developed hand tracking algorithm, including the DTW (dynamic time warping gesture recognition algorithm for a complete solution of an immersive gesture control system. The quantitative and qualitative evaluations of the integrated system were conducted with human subjects, and the results demonstrate that our gesture control with tactile feedback is a promising technology compared to a vision-based gesture control system that has typically no feedback for the user’s gesture inputs. Our study provides researchers and designers with informative guidelines to develop more natural gesture control systems or immersive user interfaces with haptic feedback.

  19. The effect of static and dynamic visual gestures on stuttering inhibition.

    Science.gov (United States)

    Guntupalli, Vijaya K; Nanjundeswaran, Chayadevie; Kalinowski, Joseph; Dayalu, Vikram N

    2011-03-29

    The aim of the study was to evaluate the role of steady-state and dynamic visual gestures of vowels in stuttering inhibition. Eight adults who stuttered recited sentences from memory while watching video presentations of the following visual speech gestures: (a) a steady-state /u/, (b) dynamic production of /a-i-u/, (c) steady-state /u/ with an accompanying audible 1 kHz pure tone, and (d) dynamic production of /a-i-u/ with an accompanying audible 1 kHz pure tone. A 1 kHz pure tone and a no-external signal condition served as control conditions. Results revealed a significant main effect of auditory condition on stuttering frequency. Relative to the no-external signal condition, the combined visual plus pure tone conditions resulted in a statistically significant reduction in stuttering frequency. In addition, a significant difference in stuttering frequency was also observed when the visual plus pure tone conditions were compared to the visual only conditions. However, no significant differences were observed between the no-external signal condition and visual only conditions, or the no-external signal condition and pure tone condition. These findings are in contrast to previous findings demonstrated by similar vowel gestures presented via the auditory modality that resulted in high levels of stuttering inhibition. The differential role of sensory modalities in speech perception and production as well as their individual capacities to transfer gestural information for the purposes of stuttering inhibition is discussed. Copyright © 2011 Elsevier Ireland Ltd. All rights reserved.

  20. Iconic Gestures for Robot Avatars, Recognition and Integration with Speech

    Science.gov (United States)

    Bremner, Paul; Leonards, Ute

    2016-01-01

    Co-verbal gestures are an important part of human communication, improving its efficiency and efficacy for information conveyance. One possible means by which such multi-modal communication might be realized remotely is through the use of a tele-operated humanoid robot avatar. Such avatars have been previously shown to enhance social presence and operator salience. We present a motion tracking based tele-operation system for the NAO robot platform that allows direct transmission of speech and gestures produced by the operator. To assess the capabilities of this system for transmitting multi-modal communication, we have conducted a user study that investigated if robot-produced iconic gestures are comprehensible, and are integrated with speech. Robot performed gesture outcomes were compared directly to those for gestures produced by a human actor, using a within participant experimental design. We show that iconic gestures produced by a tele-operated robot are understood by participants when presented alone, almost as well as when produced by a human. More importantly, we show that gestures are integrated with speech when presented as part of a multi-modal communication equally well for human and robot performances. PMID:26925010

  1. Iconic Gestures for Robot Avatars, Recognition and Integration with Speech

    Directory of Open Access Journals (Sweden)

    Paul Adam Bremner

    2016-02-01

    Full Text Available Co-verbal gestures are an important part of human communication, improving its efficiency and efficacy for information conveyance. One possible means by which such multi-modal communication might be realised remotely is through the use of a tele-operated humanoid robot avatar. Such avatars have been previously shown to enhance social presence and operator salience. We present a motion tracking based tele-operation system for the NAO robot platform that allows direct transmission of speech and gestures produced by the operator. To assess the capabilities of this system for transmitting multi-modal communication, we have conducted a user study that investigated if robot-produced iconic gestures are comprehensible, and are integrated with speech. Robot performed gesture outcomes were compared directly to those for gestures produced by a human actor, using a within participant experimental design. We show that iconic gestures produced by a tele-operated robot are understood by participants when presented alone, almost as well as when produced by a human. More importantly, we show that gestures are integrated with speech when presented as part of a multi-modal communication equally well for human and robot performances.

  2. Optical gesture sensing and depth mapping technologies for head-mounted displays: an overview

    Science.gov (United States)

    Kress, Bernard; Lee, Johnny

    2013-05-01

    Head Mounted Displays (HMDs), and especially see-through HMDs have gained renewed interest in recent time, and for the first time outside the traditional military and defense realm, due to several high profile consumer electronics companies presenting their products to hit market. Consumer electronics HMDs have quite different requirements and constrains as their military counterparts. Voice comments are the de-facto interface for such devices, but when the voice recognition does not work (not connection to the cloud for example), trackpad and gesture sensing technologies have to be used to communicate information to the device. We review in this paper the various technologies developed today integrating optical gesture sensing in a small footprint, as well as the various related 3d depth mapping sensors.

  3. From gesture to sign language: conventionalization of classifier constructions by adult hearing learners of British Sign Language.

    Science.gov (United States)

    Marshall, Chloë R; Morgan, Gary

    2015-01-01

    There has long been interest in why languages are shaped the way they are, and in the relationship between sign language and gesture. In sign languages, entity classifiers are handshapes that encode how objects move, how they are located relative to one another, and how multiple objects of the same type are distributed in space. Previous studies have shown that hearing adults who are asked to use only manual gestures to describe how objects move in space will use gestures that bear some similarities to classifiers. We investigated how accurately hearing adults, who had been learning British Sign Language (BSL) for 1-3 years, produce and comprehend classifiers in (static) locative and distributive constructions. In a production task, learners of BSL knew that they could use their hands to represent objects, but they had difficulty choosing the same, conventionalized, handshapes as native signers. They were, however, highly accurate at encoding location and orientation information. Learners therefore show the same pattern found in sign-naïve gesturers. In contrast, handshape, orientation, and location were comprehended with equal (high) accuracy, and testing a group of sign-naïve adults showed that they too were able to understand classifiers with higher than chance accuracy. We conclude that adult learners of BSL bring their visuo-spatial knowledge and gestural abilities to the tasks of understanding and producing constructions that contain entity classifiers. We speculate that investigating the time course of adult sign language acquisition might shed light on how gesture became (and, indeed, becomes) conventionalized during the genesis of sign languages. Copyright © 2014 Cognitive Science Society, Inc.

  4. User-Generated Free-Form Gestures for Authentication: Security and Memorability

    OpenAIRE

    Sherman, Michael; Clark, Gradeigh; Yang, Yulong; Sugrim, Shridatt; Modig, Arttu; Lindqvist, Janne; Oulasvirta, Antti; Roos, Teemu

    2014-01-01

    This paper studies the security and memorability of free-form multitouch gestures for mobile authentication. Towards this end, we collected a dataset with a generate-test-retest paradigm where participants (N=63) generated free-form gestures, repeated them, and were later retested for memory. Half of the participants decided to generate one-finger gestures, and the other half generated multi-finger gestures. Although there has been recent work on template-based gestures, there are yet no metr...

  5. Cross-cultural variation of speech-accompanying gesture : a review

    OpenAIRE

    Kita, Sotaro

    2009-01-01

    This article reviews the literature on cross-cultural variation of gestures. Four factors governing the variation were identified. The first factor is the culture-specific convention for form-meaning associations. This factor is involved in well-known cross-cultural differences in emblem gestures (e.g., the OK-sign), as well as pointing gestures. The second factor is culture-specific spatial cognition. Representational gestures (i.e., iconic and deictic gestures) that express spatial contents...

  6. Seeing iconic gestures while encoding events facilitates children's memory of these events

    OpenAIRE

    Aussems, Suzanne; Kita, Sotaro

    2017-01-01

    An experiment with 72 three-year-olds investigated whether encoding events while seeing iconic gestures boosts children's memory representation of these events. The events, shown in videos of actors moving in an unusual manner, were presented with either iconic gestures depicting how the actors performed these actions, interactive gestures, or no gesture. In a recognition memory task, children in the iconic gesture condition remembered actors and actions better than children in the control co...

  7. An investigation of the use of co-verbal gestures in oral discourse among Chinese speakers with fluent versus non-fluent aphasia and healthy adults

    Directory of Open Access Journals (Sweden)

    Anthony Pak Hin Kong

    2015-04-01

    Full Text Available Introduction Co-verbal gestures can facilitate word production among persons with aphasia (PWA (Rose, Douglas, & Matyas, 2002 and play a communicative role for PWA to convey ideas (Sekine & Rose, 2013. Kong, Law, Kwan, Lai, and Lam (2015 recently reported a systematic approach to independently analyze gesture forms and functions in spontaneous oral discourse produced. When this annotation framework was used to compare speech-accompanying gestures used by PWA and unimpaired speakers, Kong, Law, Wat, and Lai (2013 found a significantly higher gesture-to-word ratio among PWAs. Speakers who were more severe in aphasia or produced a lower percentage of complete sentences or simple sentences in their narratives tended to use more gestures. Moreover, verbal-semantic processing impairment, but not the degree of hemiplegia, was found to affect PWAs’ employment of gestures. The current study aims to (1 investigate whether the frequency of gestural employment varied across speakers with non-fluent aphasia, fluent aphasia, and their controls, (2 examine how the distribution of gesture forms and functions differed across the three speaker groups, and (3 determine how well factors of complexity of linguistic output, aphasia severity, semantic processing integrity, and hemiplegia would predict the frequency of gesture use among PWAs. Method The participants included 23 Cantonese-speaking individuals with fluent aphasia, 21 with non-fluent aphasia, and 23 age- and education-matched controls. Three sets of language samples and video files were collected through the narrative tasks of recounting a personally important event, sequential description, and story-telling, using the Cantonese AphasiaBank protocol (Kong, Law, & Lee, 2009. While the language samples were linguistically quantified to reflect word- and sentential-level performance as well as discourse-level characteristics, the videos were annotated on the form and function of each gesture. All PWAs were

  8. UsiGesture: Test and Evaluation of an Environment for Integrating Gestures in User Interfaces

    OpenAIRE

    Beuvens, François; Vanderdonckt, Jean

    2014-01-01

    User interfaces allowing gesture recognition and manipulation are becoming more and more popular these last years. It however remains a hard task for programmers to developer such interfaces : some knowledge of recognition systems is required, along with user experience and user interface management knowledge. It is often difficult for only one developer to handle all this knowledge by itself and it is why a team gathering different skills is most of the time needed. We previously presented a...

  9. A Modified Tactile Brush Algorithm for Complex Touch Gestures

    Energy Technology Data Exchange (ETDEWEB)

    Ragan, Eric [Texas A& M University

    2015-01-01

    Several researchers have investigated phantom tactile sensation (i.e., the perception of a nonexistent actuator between two real actuators) and apparent tactile motion (i.e., the perception of a moving actuator due to time delays between onsets of multiple actuations). Prior work has focused primarily on determining appropriate Durations of Stimulation (DOS) and Stimulus Onset Asynchronies (SOA) for simple touch gestures, such as a single finger stroke. To expand upon this knowledge, we investigated complex touch gestures involving multiple, simultaneous points of contact, such as a whole hand touching the arm. To implement complex touch gestures, we modified the Tactile Brush algorithm to support rectangular areas of tactile stimulation.

  10. RENDIMIENTO Y REACCIÓN A COLLETOTRICHUM LINDEMUATHIANUM EN CULTIVARES DE FRÍJOL VOLUBLE (PHASEOLUS VULGARIS L. YIELD AND REACTION TO COLLETOTRICHUM LINDEMUATHIANUM IN CULTIVARS OF CLIMBING BEANS (PHASEOLUS VULGARIS L.

    Directory of Open Access Journals (Sweden)

    Carolina Gallego G.

    2010-12-01

    Full Text Available Bajo condiciones de la sabana de Bogotá (Colombia, se evaluaron 32 cultivares de fríjol voluble por componentes del rendimiento y por su reacción a una mezcla de aislamientos de Colletotrichum lindemuthianum procedentes de Boyacá y Cundinamarca. Los genotipos que presentaron un buen comportamiento en rendimiento y una reacción en campo a la resistencia de la enfermedad fueron: D. Moreno y 3198. Los que expresaron una reacción de resistencia a la antracnosis fueron: 3180, 3182, 3177 y G-2333. Aquellos que mostraron un buen comportamiento en componentes de rendimiento fueron: 3164, 3159, 3176 y Radical. Estos genotipos podrían usarse como posibles candidatos parentales ó sobresalientes en el programa de mejoramiento de fríjol. También se realizó análisis de dos marcadores moleculares tipo SCAR ligados a los genes Co-4 y Co-5 que confieren resistencia a C. lindemuthianum, ninguno de los materiales de evaluación a excepción del testigo resistente G-2333, amplificó los marcadores SCAR, asociados a los genes de resistencia de interés.Under Bogotá plateau (Colombia conditions, 32 cultivars of climbing bean were evaluated by components of yield and by reaction with a mixture of isolations of Colletotrichum lindemuthianum coming from Boyacá and Cundinamarca. The cultivars that presented a good behavior in yield and resistance reaction to the disease were: D. Moreno and 3198. Those that expressed a resistant reaction to the anthracnose were: 3180, 3182, 3177 and G-2333. Finally those that showed a good behavior in yield components were 3164, 3159, 3176 and Radical. These genotypes could be used as excellent candidates in the breeding program of common bean. It was also carried out a test for each cultivar, by means of two markers molecular type SCAR tried to resistance genes to anthracnose Co-4 and Co-5. Any of the evaluation materials amplified for the couple of genes, except for the resistant control G-2333.

  11. What makes a movement a gesture?

    Science.gov (United States)

    Novack, Miriam A; Wakefield, Elizabeth M; Goldin-Meadow, Susan

    2016-01-01

    Theories of how adults interpret the actions of others have focused on the goals and intentions of actors engaged in object-directed actions. Recent research has challenged this assumption, and shown that movements are often interpreted as being for their own sake (Schachner & Carey, 2013). Here we postulate a third interpretation of movement-movement that represents action, but does not literally act on objects in the world. These movements are gestures. In this paper, we describe a framework for predicting when movements are likely to be seen as representations. In Study 1, adults described one of three scenes: (1) an actor moving objects, (2) an actor moving her hands in the presence of objects (but not touching them) or (3) an actor moving her hands in the absence of objects. Participants systematically described the movements as depicting an object-directed action when the actor moved objects, and favored describing the movements as depicting movement for its own sake when the actor produced the same movements in the absence of objects. However, participants favored describing the movements as representations when the actor produced the movements near, but not on, the objects. Study 2 explored two additional features-the form of an actor's hands and the presence of speech-like sounds-to test the effect of context on observers' classification of movement as representational. When movements are seen as representations, they have the power to influence communication, learning, and cognition in ways that movement for its own sake does not. By incorporating representational gesture into our framework for movement analysis, we take an important step towards developing a more cohesive understanding of action-interpretation. Copyright © 2015 Elsevier B.V. All rights reserved.

  12. Increased androgenic sensitivity in the hind limb muscular system marks the evolution of a derived gestural display.

    Science.gov (United States)

    Mangiamele, Lisa A; Fuxjager, Matthew J; Schuppe, Eric R; Taylor, Rebecca S; Hödl, Walter; Preininger, Doris

    2016-05-17

    Physical gestures are prominent features of many species' multimodal displays, yet how evolution incorporates body and leg movements into animal signaling repertoires is unclear. Androgenic hormones modulate the production of reproductive signals and sexual motor skills in many vertebrates; therefore, one possibility is that selection for physical signals drives the evolution of androgenic sensitivity in select neuromotor pathways. We examined this issue in the Bornean rock frog (Staurois parvus, family: Ranidae). Males court females and compete with rivals by performing both vocalizations and hind limb gestural signals, called "foot flags." Foot flagging is a derived display that emerged in the ranids after vocal signaling. Here, we show that administration of testosterone (T) increases foot flagging behavior under seminatural conditions. Moreover, using quantitative PCR, we also find that adult male S. parvus maintain a unique androgenic phenotype, in which androgen receptor (AR) in the hind limb musculature is expressed at levels ∼10× greater than in two other anuran species, which do not produce foot flags (Rana pipiens and Xenopus laevis). Finally, because males of all three of these species solicit mates with calls, we accordingly detect no differences in AR expression in the vocal apparatus (larynx) among taxa. The results show that foot flagging is an androgen-dependent gestural signal, and its emergence is associated with increased androgenic sensitivity within the hind limb musculature. Selection for this novel gestural signal may therefore drive the evolution of increased AR expression in key muscles that control signal production to support adaptive motor performance.

  13. Hands in space: gesture interaction with augmented-reality interfaces.

    Science.gov (United States)

    Billinghurst, Mark; Piumsomboon, Tham; Huidong Bai

    2014-01-01

    Researchers at the Human Interface Technology Laboratory New Zealand (HIT Lab NZ) are investigating free-hand gestures for natural interaction with augmented-reality interfaces. They've applied the results to systems for desktop computers and mobile devices.

  14. Holographic Raman Tweezers Controlled by Hand Gestures and Voice Commands

    Czech Academy of Sciences Publication Activity Database

    Tomori, Z.; Antalík, M.; Kesa, P.; Kaňka, Jan; Jákl, Petr; Šerý, Mojmír; Bernatová, Silvie; Zemánek, Pavel

    2013-01-01

    Roč. 3, 2B (2013), s. 331-336 ISSN 2160-8881 Institutional support: RVO:68081731 Keywords : Holographic Optical Tweezers * Raman Tweezers * Natural User Interface * Leap Motion * Gesture Camera Subject RIV: BH - Optics, Masers, Lasers

  15. An Interactive Astronaut-Robot System with Gesture Control

    Directory of Open Access Journals (Sweden)

    Jinguo Liu

    2016-01-01

    Full Text Available Human-robot interaction (HRI plays an important role in future planetary exploration mission, where astronauts with extravehicular activities (EVA have to communicate with robot assistants by speech-type or gesture-type user interfaces embedded in their space suits. This paper presents an interactive astronaut-robot system integrating a data-glove with a space suit for the astronaut to use hand gestures to control a snake-like robot. Support vector machine (SVM is employed to recognize hand gestures and particle swarm optimization (PSO algorithm is used to optimize the parameters of SVM to further improve its recognition accuracy. Various hand gestures from American Sign Language (ASL have been selected and used to test and validate the performance of the proposed system.

  16. Iconic gestures prime related concepts: an ERP study.

    Science.gov (United States)

    Wu, Ying Croon; Coulson, Seana

    2007-02-01

    To assess priming by iconic gestures, we recorded EEG (at 29 scalp sites) in two experiments while adults watched short, soundless videos of spontaneously produced, cospeech iconic gestures followed by related or unrelated probe words. In Experiment 1, participants classified the relatedness between gestures and words. In Experiment 2, they attended to stimuli, and performed an incidental recognition memory test on words presented during the EEG recording session. Event-related potentials (ERPs) time-locked to the onset of probe words were measured, along with response latencies and word recognition rates. Although word relatedness did not affect reaction times or recognition rates, contextually related probe words elicited less-negative ERPs than did unrelated ones between 300 and 500 msec after stimulus onset (N400) in both experiments. These findings demonstrate sensitivity to semantic relations between iconic gestures and words in brain activity engendered during word comprehension.

  17. The didactic gesture of internal regulation in the ‘genres didactic sequence’

    Directory of Open Access Journals (Sweden)

    Eliana Merlin Deganutti de Barros

    2015-10-01

    Full Text Available This article brings results of a doctoral research based in the studies of Sociodiscursive Interactionism (SDI, particularly, in its didactic aspect. Data was obtained in a field research carried out in a public school of Londrina, Paraná State, in a 6th grade class, focused in the didactic transposition of the genre ‘letter of complaint’, under the instrumentalization of the ‘genre didactic sequence’ (GDS procedure. The epistemological focus of the article concentrates on the analysis of a founder didactic gesture: ‘the internal regulation’ –implemented by the diagnostic evaluation of the students initial production. The objective is to show the accomplishment of this gesture in the development of this step of the SDG, highlighting both the result of the learning diagnostic process and the specific didactic gestures mobilized to set this stage of the learning project. It was found that the didactic approach of the diagnostic regulation mobilized after the writing of the first version of the genre, is essential for the GDS global development, since it shows the ‘errors’ that require the didactic intervention.

  18. Chaotic Music Generation System Using Music Conductor Gesture

    OpenAIRE

    Chen, Shuai; Maeda, Yoichiro; Takahashi, Yasutake

    2013-01-01

    In the research of interactive music generation, we propose a music generation method, that the computer generates the music, under the recognition of human music conductor's gestures.In this research, the generated music is tuned by the recognized gestures for the parameters of the network of chaotic elements in real time. The music conductor's hand motions are detected by Microsoft Kinect in this system. Music theories are embedded in the algorithm, as a result, the generated music will be ...

  19. Seeing Iconic Gestures While Encoding Events Facilitates Children's Memory of These Events.

    Science.gov (United States)

    Aussems, Suzanne; Kita, Sotaro

    2017-11-08

    An experiment with 72 three-year-olds investigated whether encoding events while seeing iconic gestures boosts children's memory representation of these events. The events, shown in videos of actors moving in an unusual manner, were presented with either iconic gestures depicting how the actors performed these actions, interactive gestures, or no gesture. In a recognition memory task, children in the iconic gesture condition remembered actors and actions better than children in the control conditions. Iconic gestures were categorized based on how much of the actors was represented by the hands (feet, legs, or body). Only iconic hand-as-body gestures boosted actor memory. Thus, seeing iconic gestures while encoding events facilitates children's memory of those aspects of events that are schematically highlighted by gesture. © 2017 The Authors. Child Development © 2017 Society for Research in Child Development, Inc.

  20. Hybrid gesture recognition system for short-range use

    Science.gov (United States)

    Minagawa, Akihiro; Fan, Wei; Katsuyama, Yutaka; Takebe, Hiroaki; Ozawa, Noriaki; Hotta, Yoshinobu; Sun, Jun

    2012-03-01

    In recent years, various gesture recognition systems have been studied for use in television and video games[1]. In such systems, motion areas ranging from 1 to 3 meters deep have been evaluated[2]. However, with the burgeoning popularity of small mobile displays, gesture recognition systems capable of operating at much shorter ranges have become necessary. The problems related to such systems are exacerbated by the fact that the camera's field of view is unknown to the user during operation, which imposes several restrictions on his/her actions. To overcome the restrictions generated from such mobile camera devices, and to create a more flexible gesture recognition interface, we propose a hybrid hand gesture system, in which two types of gesture recognition modules are prepared and with which the most appropriate recognition module is selected by a dedicated switching module. The two recognition modules of this system are shape analysis using a boosting approach (detection-based approach)[3] and motion analysis using image frame differences (motion-based approach)(for example, see[4]). We evaluated this system using sample users and classified the resulting errors into three categories: errors that depend on the recognition module, errors caused by incorrect module identification, and errors resulting from user actions. In this paper, we show the results of our investigations and explain the problems related to short-range gesture recognition systems.

  1. How iconic gestures enhance communication: an ERP study.

    Science.gov (United States)

    Wu, Ying Choon; Coulson, Seana

    2007-06-01

    EEG was recorded as adults watched short segments of spontaneous discourse in which the speaker's gestures and utterances contained complementary information. Videos were followed by one of four types of picture probes: cross-modal related probes were congruent with both speech and gestures; speech-only related probes were congruent with information in the speech, but not the gesture; and two sorts of unrelated probes were created by pairing each related probe with a different discourse prime. Event-related potentials (ERPs) elicited by picture probes were measured within the time windows of the N300 (250-350 ms post-stimulus) and N400 (350-550 ms post-stimulus). Cross-modal related probes elicited smaller N300 and N400 than speech-only related ones, indicating that pictures were easier to interpret when they corresponded with gestures. N300 and N400 effects were not due to differences in the visual complexity of each probe type, since the same cross-modal and speech-only picture probes elicited N300 and N400 with similar amplitudes when they appeared as unrelated items. These findings extend previous research on gesture comprehension by revealing how iconic co-speech gestures modulate conceptualization, enabling listeners to better represent visuo-spatial aspects of the speaker's meaning.

  2. Pointing and tracing gestures may enhance anatomy and physiology learning.

    Science.gov (United States)

    Macken, Lucy; Ginns, Paul

    2014-07-01

    Currently, instructional effects generated by Cognitive load theory (CLT) are limited to visual and auditory cognitive processing. In contrast, "embodied cognition" perspectives suggest a range of gestures, including pointing, may act to support communication and learning, but there is relatively little research showing benefits of such "embodied learning" in the health sciences. This study investigated whether explicit instructions to gesture enhance learning through its cognitive effects. Forty-two university-educated adults were randomly assigned to conditions in which they were instructed to gesture, or not gesture, as they learnt from novel, paper-based materials about the structure and function of the human heart. Subjective ratings were used to measure levels of intrinsic, extraneous and germane cognitive load. Participants who were instructed to gesture performed better on a knowledge test of terminology and a test of comprehension; however, instructions to gesture had no effect on subjective ratings of cognitive load. This very simple instructional re-design has the potential to markedly enhance student learning of typical topics and materials in the health sciences and medicine.

  3. Stationary Hand Gesture Authentication Using Edit Distance on Finger Pointing Direction Interval

    Directory of Open Access Journals (Sweden)

    Alex Ming Hui Wong

    2016-01-01

    Full Text Available One of the latest authentication methods is by discerning human gestures. Previous research has shown that different people can develop distinct gesture behaviours even when executing the same gesture. Hand gesture is one of the most commonly used gestures in both communication and authentication research since it requires less room to perform as compared to other bodily gestures. There are different types of hand gesture and they have been researched by many researchers, but stationary hand gesture has yet to be thoroughly explored. There are a number of disadvantages and flaws in general hand gesture authentication such as reliability, usability, and computational cost. Although stationary hand gesture is not able to solve all these problems, it still provides more benefits and advantages over other hand gesture authentication methods, such as making gesture into a motion flow instead of trivial image capturing, and requires less room to perform, less vision cue needed during performance, and so forth. In this paper, we introduce stationary hand gesture authentication by implementing edit distance on finger pointing direction interval (ED-FPDI from hand gesture to model behaviour-based authentication system. The accuracy rate of the proposed ED-FPDI shows promising results.

  4. Getting to the elephants: Gesture and preschoolers' comprehension of route direction information.

    Science.gov (United States)

    Austin, Elizabeth E; Sweller, Naomi

    2017-11-01

    During early childhood, children find spatial tasks such as following novel route directions challenging. Spatial tasks place demands on multiple cognitive processes, including language comprehension and memory, at a time in development when resources are limited. As such, gestures accompanying route directions may aid comprehension and facilitate task performance by scaffolding cognitive processes, including language and memory processing. This study examined the effect of presenting gesture during encoding on spatial task performance during early childhood. Three- to five-year-olds were presented with verbal route directions through a zoo-themed spatial array and, depending on assigned condition (no gesture, beat gesture, or iconic/deictic gesture), accompanying gestures. Children presented with verbal route directions accompanied by a combination of iconic (pantomime) and deictic (pointing) gestures verbally recalled more than children presented with beat gestures (rhythmic hand movements) or no gestures accompanying the route directions. The presence of gesture accompanying route directions similarly influenced physical route navigation, such that children presented with gesture (beat, pantomime, and pointing) navigated the route more accurately than children presented with no gestures. Across all gesture conditions, location information (e.g., the penguin pond) was recalled more than movement information (e.g., go around) and descriptive information (e.g., bright red). These findings suggest that speakers' gestures accompanying spatial task information influence listeners' recall and task performance. Copyright © 2017 Elsevier Inc. All rights reserved.

  5. GestuRe and ACtion Exemplar (GRACE) video database: stimuli for research on manners of human locomotion and iconic gestures.

    Science.gov (United States)

    Aussems, Suzanne; Kwok, Natasha; Kita, Sotaro

    2018-06-01

    Human locomotion is a fundamental class of events, and manners of locomotion (e.g., how the limbs are used to achieve a change of location) are commonly encoded in language and gesture. To our knowledge, there is no openly accessible database containing normed human locomotion stimuli. Therefore, we introduce the GestuRe and ACtion Exemplar (GRACE) video database, which contains 676 videos of actors performing novel manners of human locomotion (i.e., moving from one location to another in an unusual manner) and videos of a female actor producing iconic gestures that represent these actions. The usefulness of the database was demonstrated across four norming experiments. First, our database contains clear matches and mismatches between iconic gesture videos and action videos. Second, the male actors and female actors whose action videos matched the gestures in the best possible way, perform the same actions in very similar manners and different actions in highly distinct manners. Third, all the actions in the database are distinct from each other. Fourth, adult native English speakers were unable to describe the 26 different actions concisely, indicating that the actions are unusual. This normed stimuli set is useful for experimental psychologists working in the language, gesture, visual perception, categorization, memory, and other related domains.

  6. Great ape gestures: intentional communication with a rich set of innate signals.

    Science.gov (United States)

    Byrne, R W; Cartmill, E; Genty, E; Graham, K E; Hobaiter, C; Tanner, J

    2017-09-08

    Great apes give gestures deliberately and voluntarily, in order to influence particular target audiences, whose direction of attention they take into account when choosing which type of gesture to use. These facts make the study of ape gesture directly relevant to understanding the evolutionary precursors of human language; here we present an assessment of ape gesture from that perspective, focusing on the work of the "St Andrews Group" of researchers. Intended meanings of ape gestures are relatively few and simple. As with human words, ape gestures often have several distinct meanings, which are effectively disambiguated by behavioural context. Compared to the signalling of most other animals, great ape gestural repertoires are large. Because of this, and the relatively small number of intended meanings they achieve, ape gestures are redundant, with extensive overlaps in meaning. The great majority of gestures are innate, in the sense that the species' biological inheritance includes the potential to develop each gestural form and use it for a specific range of purposes. Moreover, the phylogenetic origin of many gestures is relatively old, since gestures are extensively shared between different genera in the great ape family. Acquisition of an adult repertoire is a process of first exploring the innate species potential for many gestures and then gradual restriction to a final (active) repertoire that is much smaller. No evidence of syntactic structure has yet been detected.

  7. THE CONTRIBUTION OF GESTURES TO PERSONAL BRANDING

    Directory of Open Access Journals (Sweden)

    Brînduşa-Mariana Amălăncei

    2015-07-01

    Full Text Available A form of (self-promotion but also an authentic strategic choice, the personal brand has become a topical preoccupation of marketing specialists. Personal branding or self-marketing represents an innovative concept that associates the efficiency of personal development with the effectiveness of communication and marketing techniques adapted to the individual and that comprises the entire collection of techniques allowing the identification and promotion of the self/individual. The main objective is a clear communication with regard to personal identity, no matter by means of which method, so that it gives uniqueness and offers a competitive advantage. Although online promotion is increasingly gaining ground for the creation of a personal brand, an individual’s verbal and nonverbal behaviour represent very important differentiating elements. Starting from the premise that gestures often complement, anticipate, substitute or contradict the verbal, we will endeavour to highlight a number of significations that can be attributed to the various body movements and that can successfully contribute to the creation of a powerful personal brand.

  8. Hegel’s Gesture Towards Radical Cosmopolitanism

    Directory of Open Access Journals (Sweden)

    Shannon Brincat

    2009-09-01

    Full Text Available This is a preliminary argument of a much larger research project inquiring into the relation betweenHegel’s philosophical system and the project of emancipation in Critical International Relations Theory. Specifically, the paper examines how Hegel’s theory of recognition gestures towards a form of radical cosmopolitanism in world politics to ensure the conditions of rational freedom for all humankind. Much of the paper is a ground-clearing exercise defining what is ‘living’ in Hegel’s thought for emancipatory approaches in world politics, to borrow from Croce’s now famous question. It focuses on Hegel’s unique concept of freedom which places recognition as central in the formation of self-consciousness and therefore as a key determinant in the conditions necessary forhuman freedom to emerge in political community. While further research is needed to ascertain the precise relationship between Hegel’s recognition theoretic, emancipation and cosmopolitanism, it is contended that the intersubjective basis of Hegel’s concept of freedom through recognition necessitates some form of radical cosmopolitanism that ensures successful processes of recognition between all peoples, the precise institutional form of which remains unspecified.

  9. Grammatical Aspect and Gesture in French: A kinesiological approach

    Directory of Open Access Journals (Sweden)

    Доминик Бутэ

    2016-12-01

    Full Text Available In this paper, we defend the idea that research on Gesture with Speech can provide ways of studying speakers’ conceptualization of grammatical notions as they are speaking. Expressing an idea involves a dynamic interplay between our construal, shaped by the sensori-motoric and interactive experiences linked to that idea, the plurisemiotic means at our disposal for expressing it, and the linguistic category available for its expression in our language. By analyzing the expression of aspect in Speech with Gesture (GeSp in semi-guided oral interactions, we would like to make a new contribution to the field of aspect by exploring how speakers’ construal of aspectual differences grammaticalized in their language, may be enacted and visible in gesture. More specifically we want to see the degree to which event structure differences expressed in different grammatical aspects (perfective and imperfective correlate with kinesiological features of the gestures. To this end, we will focus on the speed and flow of the movements as well as on the segments involved (fingers, hand, forearm, arm, shoulder. A kinesiological approach to gestures enables us to analyze the movements of human bodies according to a biomechanical point of view that includes physiological features. This study is the first contribution focused on the links between speech and gesture in French in the domain of grammatical aspect. Grammatical aspect was defined by Comrie (1976 [1989] as involving the internal unfurling of the process, «[...] tense is a deictic category, i.e. locates situations in time, usually with reference to the present moment [...]. Aspect is not concerned with relating time of the situation to any other time-point, but rather with the internal temporal constituency of the one situation; one could state the difference as one between situation-internal time (aspect and situation-external time (tense » (Comrie, 1976 [1989]: 5. Can kinesic features express and make

  10. Preserved Imitation of Known Gestures in Children with High-Functioning Autism

    Science.gov (United States)

    Carmo, Joana C.; Rumiati, Raffaella I.; Siugzdaite, Roma; Brambilla, Paolo

    2013-01-01

    It has been suggested that children with autism are particularly deficient at imitating novel gestures or gestures without goals. In the present study, we asked high-functioning autistic children and age-matched typically developing children to imitate several types of gestures that could be either already known or novel to them. Known gestures either conveyed a communicative meaning (i.e., intransitive) or involved the use of objects (i.e., transitive). We observed a significant interaction between gesture type and group of participants, with children with autism performing known gestures better than novel gestures. However, imitation of intransitive and transitive gestures did not differ across groups. These findings are discussed in light of a dual-route model for action imitation. PMID:24062956

  11. Web-based interactive drone control using hand gesture

    Science.gov (United States)

    Zhao, Zhenfei; Luo, Hao; Song, Guang-Hua; Chen, Zhou; Lu, Zhe-Ming; Wu, Xiaofeng

    2018-01-01

    This paper develops a drone control prototype based on web technology with the aid of hand gesture. The uplink control command and downlink data (e.g., video) are transmitted by WiFi communication, and all the information exchange is realized on web. The control command is translated from various predetermined hand gestures. Specifically, the hardware of this friendly interactive control system is composed by a quadrotor drone, a computer vision-based hand gesture sensor, and a cost-effective computer. The software is simplified as a web-based user interface program. Aided by natural hand gestures, this system significantly reduces the complexity of traditional human-computer interaction, making remote drone operation more intuitive. Meanwhile, a web-based automatic control mode is provided in addition to the hand gesture control mode. For both operation modes, no extra application program is needed to be installed on the computer. Experimental results demonstrate the effectiveness and efficiency of the proposed system, including control accuracy, operation latency, etc. This system can be used in many applications such as controlling a drone in global positioning system denied environment or by handlers without professional drone control knowledge since it is easy to get started.

  12. Autonomous learning in gesture recognition by using lobe component analysis

    Science.gov (United States)

    Lu, Jian; Weng, Juyang

    2007-02-01

    Gesture recognition is a new human-machine interface method implemented by pattern recognition(PR).In order to assure robot safety when gesture is used in robot control, it is required to implement the interface reliably and accurately. Similar with other PR applications, 1) feature selection (or model establishment) and 2) training from samples, affect the performance of gesture recognition largely. For 1), a simple model with 6 feature points at shoulders, elbows, and hands, is established. The gestures to be recognized are restricted to still arm gestures, and the movement of arms is not considered. These restrictions are to reduce the misrecognition, but are not so unreasonable. For 2), a new biological network method, called lobe component analysis(LCA), is used in unsupervised learning. Lobe components, corresponding to high-concentrations in probability of the neuronal input, are orientation selective cells follow Hebbian rule and lateral inhibition. Due to the advantage of LCA method for balanced learning between global and local features, large amount of samples can be used in learning efficiently.

  13. Web-based interactive drone control using hand gesture.

    Science.gov (United States)

    Zhao, Zhenfei; Luo, Hao; Song, Guang-Hua; Chen, Zhou; Lu, Zhe-Ming; Wu, Xiaofeng

    2018-01-01

    This paper develops a drone control prototype based on web technology with the aid of hand gesture. The uplink control command and downlink data (e.g., video) are transmitted by WiFi communication, and all the information exchange is realized on web. The control command is translated from various predetermined hand gestures. Specifically, the hardware of this friendly interactive control system is composed by a quadrotor drone, a computer vision-based hand gesture sensor, and a cost-effective computer. The software is simplified as a web-based user interface program. Aided by natural hand gestures, this system significantly reduces the complexity of traditional human-computer interaction, making remote drone operation more intuitive. Meanwhile, a web-based automatic control mode is provided in addition to the hand gesture control mode. For both operation modes, no extra application program is needed to be installed on the computer. Experimental results demonstrate the effectiveness and efficiency of the proposed system, including control accuracy, operation latency, etc. This system can be used in many applications such as controlling a drone in global positioning system denied environment or by handlers without professional drone control knowledge since it is easy to get started.

  14. Comparison of gesture and conventional interaction techniques for interventional neuroradiology.

    Science.gov (United States)

    Hettig, Julian; Saalfeld, Patrick; Luz, Maria; Becker, Mathias; Skalej, Martin; Hansen, Christian

    2017-09-01

    Interaction with radiological image data and volume renderings within a sterile environment is a challenging task. Clinically established methods such as joystick control and task delegation can be time-consuming and error-prone and interrupt the workflow. New touchless input modalities may have the potential to overcome these limitations, but their value compared to established methods is unclear. We present a comparative evaluation to analyze the value of two gesture input modalities (Myo Gesture Control Armband and Leap Motion Controller) versus two clinically established methods (task delegation and joystick control). A user study was conducted with ten experienced radiologists by simulating a diagnostic neuroradiological vascular treatment with two frequently used interaction tasks in an experimental operating room. The input modalities were assessed using task completion time, perceived task difficulty, and subjective workload. Overall, the clinically established method of task delegation performed best under the study conditions. In general, gesture control failed to exceed the clinical input approach. However, the Myo Gesture Control Armband showed a potential for simple image selection task. Novel input modalities have the potential to take over single tasks more efficiently than clinically established methods. The results of our user study show the relevance of task characteristics such as task complexity on performance with specific input modalities. Accordingly, future work should consider task characteristics to provide a useful gesture interface for a specific use case instead of an all-in-one solution.

  15. Lexical learning in mild aphasia: gesture benefit depends on patholinguistic profile and lesion pattern.

    Science.gov (United States)

    Kroenke, Klaus-Martin; Kraft, Indra; Regenbrecht, Frank; Obrig, Hellmuth

    2013-01-01

    Gestures accompany speech and enrich human communication. When aphasia interferes with verbal abilities, gestures become even more relevant, compensating for and/or facilitating verbal communication. However, small-scale clinical studies yielded diverging results with regard to a therapeutic gesture benefit for lexical retrieval. Based on recent functional neuroimaging results, delineating a speech-gesture integration network for lexical learning in healthy adults, we hypothesized that the commonly observed variability may stem from differential patholinguistic profiles in turn depending on lesion pattern. Therefore we used a controlled novel word learning paradigm to probe the impact of gestures on lexical learning, in the lesioned language network. Fourteen patients with chronic left hemispheric lesions and mild residual aphasia learned 30 novel words for manipulable objects over four days. Half of the words were trained with gestures while the other half were trained purely verbally. For the gesture condition, rootwords were visually presented (e.g., Klavier, [piano]), followed by videos of the corresponding gestures and the auditory presentation of the novel words (e.g., /krulo/). Participants had to repeat pseudowords and simultaneously reproduce gestures. In the verbal condition no gesture-video was shown and participants only repeated pseudowords orally. Correlational analyses confirmed that gesture benefit depends on the patholinguistic profile: lesser lexico-semantic impairment correlated with better gesture-enhanced learning. Conversely largely preserved segmental-phonological capabilities correlated with better purely verbal learning. Moreover, structural MRI-analysis disclosed differential lesion patterns, most interestingly suggesting that integrity of the left anterior temporal pole predicted gesture benefit. Thus largely preserved semantic capabilities and relative integrity of a semantic integration network are prerequisites for successful use of

  16. Touch-less interaction with medical images using hand & foot gestures

    DEFF Research Database (Denmark)

    Jalaliniya, Shahram; Smith, Jeremiah; Sousa, Miguel

    2013-01-01

    control. In this paper, we present a system for gesture-based interaction with medical images based on a single wristband sensor and capacitive floor sensors, allowing for hand and foot gesture input. The first limited evaluation of the system showed an acceptable level of accuracy for 12 different hand...... & foot gestures; also users found that our combined hand and foot based gestures are intuitive for providing input....

  17. The Effect of Iconic and Beat Gestures on Memory Recall in Greek's First and Second Language

    OpenAIRE

    Eleni Ioanna Levantinou

    2016-01-01

    Gestures play a major role in comprehension and memory recall due to the fact that aid the efficient channel of the meaning and support listeners’ comprehension and memory. In the present study, the assistance of two kinds of gestures (iconic and beat gestures) is tested in regards to memory and recall. The hypothesis investigated here is whether or not iconic and beat gestures provide assistance in memory and recall in Greek and in Greek speakers’ second language. Two gr...

  18. LOCALIZATION AND RECOGNITION OF DYNAMIC HAND GESTURES BASED ON HIERARCHY OF MANIFOLD CLASSIFIERS

    OpenAIRE

    M. Favorskaya; A. Nosov; A. Popov

    2015-01-01

    Generally, the dynamic hand gestures are captured in continuous video sequences, and a gesture recognition system ought to extract the robust features automatically. This task involves the highly challenging spatio-temporal variations of dynamic hand gestures. The proposed method is based on two-level manifold classifiers including the trajectory classifiers in any time instants and the posture classifiers of sub-gestures in selected time instants. The trajectory classifiers contain skin dete...

  19. Brave NUI World Designing Natural User Interfaces for Touch and Gesture

    CERN Document Server

    Wigdor, Daniel

    2011-01-01

    Touch and gestural devices have been hailed as next evolutionary step in human-computer interaction. As software companies struggle to catch up with one another in terms of developing the next great touch-based interface, designers are charged with the daunting task of keeping up with the advances in new technology and this new aspect to user experience design. Product and interaction designers, developers and managers are already well versed in UI design, but touch-based interfaces have added a new level of complexity.

  20. Gesture analysis of students' majoring mathematics education in micro teaching process

    Science.gov (United States)

    Maldini, Agnesya; Usodo, Budi; Subanti, Sri

    2017-08-01

    In the process of learning, especially math learning, process of interaction between teachers and students is certainly a noteworthy thing. In these interactions appear gestures or other body spontaneously. Gesture is an important source of information, because it supports oral communication and reduce the ambiguity of understanding the concept/meaning of the material and improve posture. This research which is particularly suitable for an exploratory research design to provide an initial illustration of the phenomenon. The goal of the research in this article is to describe the gesture of S1 and S2 students of mathematics education at the micro teaching process. To analyze gesture subjects, researchers used McNeil clarification. The result is two subjects using 238 gesture in the process of micro teaching as a means of conveying ideas and concepts in mathematics learning. During the process of micro teaching, subjects using the four types of gesture that is iconic gestures, deictic gesture, regulator gesturesand adapter gesture as a means to facilitate the delivery of the intent of the material being taught and communication to the listener. Variance gesture that appear on the subject due to the subject using a different gesture patterns to communicate mathematical ideas of their own so that the intensity of gesture that appeared too different.

  1. Methodological Reflections on Gesture Analysis in Second Language Acquisition and Bilingualism Research

    Science.gov (United States)

    Gullberg, Marianne

    2010-01-01

    Gestures, i.e. the symbolic movements that speakers perform while they speak, form a closely interconnected system with speech, where gestures serve both addressee-directed ("communicative") and speaker-directed ("internal") functions. This article aims (1) to show that a combined analysis of gesture and speech offers new ways to address…

  2. Parent-Child Gesture Use during Problem Solving in Autistic Spectrum Disorder

    Science.gov (United States)

    Medeiros, Kristen; Winsler, Adam

    2014-01-01

    This study examined the relationship between child language skills and parent and child gestures of 58 youths with and without an autism spectrum disorder (ASD) diagnosis. Frequencies and rates of total gesture use as well as five categories of gestures (deictic, conventional, beat, iconic, and metaphoric) were reliably coded during the…

  3. Traveller: An Interactive Cultural Training System Controlled by User-Defined Body Gestures

    NARCIS (Netherlands)

    Kistler, F.; André, E.; Mascarenhas, S.; Silva, A.; Paiva, A.; Degens, D.M.; Hofstede, G.J.; Krumhuber, E.; Kappas, A.; Aylett, R.

    2013-01-01

    In this paper, we describe a cultural training system based on an interactive storytelling approach and a culturally-adaptive agent architecture, for which a user-defined gesture set was created. 251 full body gestures by 22 users were analyzed to find intuitive gestures for the in-game actions in

  4. Effects of eye contact and iconic gestures on message retention in human-robot interaction

    NARCIS (Netherlands)

    Dijk, van E.T.; Torta, E.; Cuijpers, R.H.

    2013-01-01

    The effects of iconic gestures and eye contact on message retention in human-robot interaction were investigated in a series of experiments. A humanoid robot gave short verbal messages to participants, accompanied either by iconic gestures or no gestures while making eye contact with the participant

  5. What is the best strategy for retaining gestures in working memory?

    Science.gov (United States)

    Gimenes, Guillaume; Pennequin, Valérie; Mercer, Tom

    2016-07-01

    This study aimed to determine whether the recall of gestures in working memory could be enhanced by verbal or gestural strategies. We also attempted to examine whether these strategies could help resist verbal or gestural interference. Fifty-four participants were divided into three groups according to the content of the training session. This included a control group, a verbal strategy group (where gestures were associated with labels) and a gestural strategy group (where participants repeated gestures and were told to imagine reproducing the movements). During the experiment, the participants had to reproduce a series of gestures under three conditions: "no interference", gestural interference (gestural suppression) and verbal interference (articulatory suppression). The results showed that task performance was enhanced in the verbal strategy group, but there was no significant difference between the gestural strategy and control groups. Moreover, compared to the "no interference" condition, performance decreased in the presence of gestural interference, except within the verbal strategy group. Finally, verbal interference hindered performance in all groups. The discussion focuses on the use of labels to recall gestures and differentiates the induced strategies from self-initiated strategies.

  6. Domestic Dogs Use Contextual Information and Tone of Voice when following a Human Pointing Gesture

    NARCIS (Netherlands)

    Scheider, Linda; Grassmann, Susanne; Kaminski, Juliane; Tomasello, Michael

    2011-01-01

    Domestic dogs are skillful at using the human pointing gesture. In this study we investigated whether dogs take contextual information into account when following pointing gestures, specifically, whether they follow human pointing gestures more readily in the context in which food has been found

  7. An Efficient Solution for Hand Gesture Recognition from Video Sequence

    Directory of Open Access Journals (Sweden)

    PRODAN, R.-C.

    2012-08-01

    Full Text Available The paper describes a system of hand gesture recognition by image processing for human robot interaction. The recognition and interpretation of the hand postures acquired through a video camera allow the control of the robotic arm activity: motion - translation and rotation in 3D - and tightening/releasing the clamp. A gesture dictionary was defined and heuristic algorithms for recognition were developed and tested. The system can be used for academic and industrial purposes, especially for those activities where the movements of the robotic arm were not previously scheduled, for training the robot easier than using a remote control. Besides the gesture dictionary, the novelty of the paper consists in a new technique for detecting the relative positions of the fingers in order to recognize the various hand postures, and in the achievement of a robust system for controlling robots by postures of the hands.

  8. Hand Gesture Recognition Using Modified 1$ and Background Subtraction Algorithms

    Directory of Open Access Journals (Sweden)

    Hazem Khaled

    2015-01-01

    Full Text Available Computers and computerized machines have tremendously penetrated all aspects of our lives. This raises the importance of Human-Computer Interface (HCI. The common HCI techniques still rely on simple devices such as keyboard, mice, and joysticks, which are not enough to convoy the latest technology. Hand gesture has become one of the most important attractive alternatives to existing traditional HCI techniques. This paper proposes a new hand gesture detection system for Human-Computer Interaction using real-time video streaming. This is achieved by removing the background using average background algorithm and the 1$ algorithm for hand’s template matching. Then every hand gesture is translated to commands that can be used to control robot movements. The simulation results show that the proposed algorithm can achieve high detection rate and small recognition time under different light changes, scales, rotation, and background.

  9. Scientific Visualization of Radio Astronomy Data using Gesture Interaction

    Science.gov (United States)

    Mulumba, P.; Gain, J.; Marais, P.; Woudt, P.

    2015-09-01

    MeerKAT in South Africa (Meer = More Karoo Array Telescope) will require software to help visualize, interpret and interact with multidimensional data. While visualization of multi-dimensional data is a well explored topic, little work has been published on the design of intuitive interfaces to such systems. More specifically, the use of non-traditional interfaces (such as motion tracking and multi-touch) has not been widely investigated within the context of visualizing astronomy data. We hypothesize that a natural user interface would allow for easier data exploration which would in turn lead to certain kinds of visualizations (volumetric, multidimensional). To this end, we have developed a multi-platform scientific visualization system for FITS spectral data cubes using VTK (Visualization Toolkit) and a natural user interface to explore the interaction between a gesture input device and multidimensional data space. Our system supports visual transformations (translation, rotation and scaling) as well as sub-volume extraction and arbitrary slicing of 3D volumetric data. These tasks were implemented across three prototypes aimed at exploring different interaction strategies: standard (mouse/keyboard) interaction, volumetric gesture tracking (Leap Motion controller) and multi-touch interaction (multi-touch monitor). A Heuristic Evaluation revealed that the volumetric gesture tracking prototype shows great promise for interfacing with the depth component (z-axis) of 3D volumetric space across multiple transformations. However, this is limited by users needing to remember the required gestures. In comparison, the touch-based gesture navigation is typically more familiar to users as these gestures were engineered from standard multi-touch actions. Future work will address a complete usability test to evaluate and compare the different interaction modalities against the different visualization tasks.

  10. A Versatile Embedded Platform for EMG Acquisition and Gesture Recognition.

    Science.gov (United States)

    Benatti, Simone; Casamassima, Filippo; Milosevic, Bojan; Farella, Elisabetta; Schönle, Philipp; Fateh, Schekeb; Burger, Thomas; Huang, Qiuting; Benini, Luca

    2015-10-01

    Wearable devices offer interesting features, such as low cost and user friendliness, but their use for medical applications is an open research topic, given the limited hardware resources they provide. In this paper, we present an embedded solution for real-time EMG-based hand gesture recognition. The work focuses on the multi-level design of the system, integrating the hardware and software components to develop a wearable device capable of acquiring and processing EMG signals for real-time gesture recognition. The system combines the accuracy of a custom analog front end with the flexibility of a low power and high performance microcontroller for on-board processing. Our system achieves the same accuracy of high-end and more expensive active EMG sensors used in applications with strict requirements on signal quality. At the same time, due to its flexible configuration, it can be compared to the few wearable platforms designed for EMG gesture recognition available on market. We demonstrate that we reach similar or better performance while embedding the gesture recognition on board, with the benefit of cost reduction. To validate this approach, we collected a dataset of 7 gestures from 4 users, which were used to evaluate the impact of the number of EMG channels, the number of recognized gestures and the data rate on the recognition accuracy and on the computational demand of the classifier. As a result, we implemented a SVM recognition algorithm capable of real-time performance on the proposed wearable platform, achieving a classification rate of 90%, which is aligned with the state-of-the-art off-line results and a 29.7 mW power consumption, guaranteeing 44 hours of continuous operation with a 400 mAh battery.

  11. Smart Remote for the Setup Box Using Gesture Control

    OpenAIRE

    Surepally Uday Kumar; K. Shamini

    2016-01-01

    The basic purpose of this project is to provide a means to control a set top box (capable of infrared communication), in this case Hathway using hand gestures. Thus, this system will act like a remote control for operating set top box, but this will be achieved through hand gestures instead of pushing buttons. To send and receive remote control signals, this project uses an infrared LED as Transmitter. Using an infrared receiver, an Arduino can detect the bits being sent by a remo...

  12. Gesture Recognition for Educational Games: Magic Touch Math

    Science.gov (United States)

    Kye, Neo Wen; Mustapha, Aida; Azah Samsudin, Noor

    2017-08-01

    Children nowadays are having problem learning and understanding basic mathematical operations because they are not interested in studying or learning mathematics. This project proposes an educational game called Magic Touch Math that focuses on basic mathematical operations targeted to children between the age of three to five years old using gesture recognition to interact with the game. Magic Touch Math was developed in accordance to the Game Development Life Cycle (GDLC) methodology. The prototype developed has helped children to learn basic mathematical operations via intuitive gestures. It is hoped that the application is able to get the children motivated and interested in mathematics.

  13. Asymmetric Dynamic Attunement of Speech and Gestures in the Construction of Children's Understanding.

    Science.gov (United States)

    De Jonge-Hoekstra, Lisette; Van der Steen, Steffie; Van Geert, Paul; Cox, Ralf F A

    2016-01-01

    As children learn they use their speech to express words and their hands to gesture. This study investigates the interplay between real-time gestures and speech as children construct cognitive understanding during a hands-on science task. 12 children (M = 6, F = 6) from Kindergarten (n = 5) and first grade (n = 7) participated in this study. Each verbal utterance and gesture during the task were coded, on a complexity scale derived from dynamic skill theory. To explore the interplay between speech and gestures, we applied a cross recurrence quantification analysis (CRQA) to the two coupled time series of the skill levels of verbalizations and gestures. The analysis focused on (1) the temporal relation between gestures and speech, (2) the relative strength and direction of the interaction between gestures and speech, (3) the relative strength and direction between gestures and speech for different levels of understanding, and (4) relations between CRQA measures and other child characteristics. The results show that older and younger children differ in the (temporal) asymmetry in the gestures-speech interaction. For younger children, the balance leans more toward gestures leading speech in time, while the balance leans more toward speech leading gestures for older children. Secondly, at the group level, speech attracts gestures in a more dynamically stable fashion than vice versa, and this asymmetry in gestures and speech extends to lower and higher understanding levels. Yet, for older children, the mutual coupling between gestures and speech is more dynamically stable regarding the higher understanding levels. Gestures and speech are more synchronized in time as children are older. A higher score on schools' language tests is related to speech attracting gestures more rigidly and more asymmetry between gestures and speech, only for the less difficult understanding levels. A higher score on math or past science tasks is related to less asymmetry between gestures and

  14. Asymmetric dynamic attunement of speech and gestures in the construction of children’s understanding

    Directory of Open Access Journals (Sweden)

    Lisette eDe Jonge-Hoekstra

    2016-03-01

    Full Text Available As children learn they use their speech to express words and their hands to gesture. This study investigates the interplay between real-time gestures and speech as children construct cognitive understanding during a hands-on science task. 12 children (M = 6, F = 6 from Kindergarten (n = 5 and first grade (n = 7 participated in this study. Each verbal utterance and gesture during the task were coded, on a complexity scale derived from dynamic skill theory. To explore the interplay between speech and gestures, we applied a cross recurrence quantification analysis (CRQA to the two coupled time series of the skill levels of verbalizations and gestures. The analysis focused on 1 the temporal relation between gestures and speech, 2 the relative strength and direction of the interaction between gestures and speech, 3 the relative strength and direction between gestures and speech for different levels of understanding, and 4 relations between CRQA measures and other child characteristics. The results show that older and younger children differ in the (temporal asymmetry in the gestures-speech interaction. For younger children, the balance leans more towards gestures leading speech in time, while the balance leans more towards speech leading gestures for older children. Secondly, at the group level, speech attracts gestures in a more dynamically stable fashion than vice versa, and this asymmetry in gestures and speech extends to lower and higher understanding levels. Yet, for older children, the mutual coupling between gestures and speech is more dynamically stable regarding the higher understanding levels. Gestures and speech are more synchronized in time as children are older. A higher score on schools’ language tests is related to speech attracting gestures more rigidly and more asymmetry between gestures and speech, only for the less difficult understanding levels. A higher score on math or past science tasks is related to less asymmetry between

  15. Early Gesture Provides a Helping Hand to Spoken Vocabulary Development for Children with Autism, Down Syndrome, and Typical Development

    Science.gov (United States)

    Özçaliskan, Seyda; Adamson, Lauren B.; Dimitrova, Nevena; Baumann, Stephanie

    2017-01-01

    Typically developing (TD) children refer to objects uniquely in gesture (e.g., point at a cat) before they produce verbal labels for these objects ("cat"). The onset of such gestures predicts the onset of similar spoken words, showing a strong positive relation between early gestures and early words. We asked whether gesture plays the…

  16. Coupling dynamics in speech gestures: amplitude and rate influences.

    Science.gov (United States)

    van Lieshout, Pascal H H M

    2017-08-01

    Speech is a complex oral motor function that involves multiple articulators that need to be coordinated in space and time at relatively high movement speeds. How this is accomplished remains an important and largely unresolved empirical question. From a coordination dynamics perspective, coordination involves the assembly of coordinative units that are characterized by inherently stable coupling patterns that act as attractor states for task-specific actions. In the motor control literature, one particular model formulated by Haken et al. (Biol Cybern 51(5):347-356, 1985) or HKB has received considerable attention in the way it can account for changes in the nature and stability of specific coordination patterns between limbs or between limbs and external stimuli. In this model (and related versions), movement amplitude is considered a critical factor in the formation of these patterns. Several studies have demonstrated its role for bimanual coordination and similar types of tasks, but for speech motor control such studies are lacking. The current study describes a systematic approach to evaluate the impact of movement amplitude and movement duration on coordination stability in the production of bilabial and tongue body gestures for specific vowel-consonant-vowel strings. The vowel combinations that were used induced a natural contrast in movement amplitude at three speaking rate conditions (slow, habitual, fast). Data were collected on ten young adults using electromagnetic articulography, recording movement data from lips and tongue with high temporal and spatial precision. The results showed that with small movement amplitudes there is a decrease in coordination stability, independent from movement duration. These findings were found to be robust across all individuals and are interpreted as further evidence that principles of coupling dynamics operate in the oral motor control system similar to other motor systems and can be explained in terms of coupling

  17. Language, gesture, and handedness: Evidence for independent lateralized networks.

    Science.gov (United States)

    Häberling, Isabelle S; Corballis, Paul M; Corballis, Michael C

    2016-09-01

    Language, gesture, and handedness are in most people represented in the left cerebral hemisphere. To explore the relations among these attributes, we collected fMRI images in a large sample of left- and right-handers while they performed language tasks and watched action sequences. Regions of interest included the frontal and parietal areas previously identified as comprising an action-observation network, and the frontal and temporal areas comprising the primary areas for language production and comprehension. All of the language areas and most of the action-observation areas showed an overall left-hemispheric bias, despite the participation of equal numbers of left- and right-handers. A factor analysis of the laterality indices derived from the different areas during the tasks indicated three independent networks, one associated with language, one associated with handedness, and one representing action observation independent of handedness. Areas 44 and 45, which together make up Broca's area, were part of the language and action-observation networks, but were not included in the part of the action observation network that was related to handedness, which in turn was strongly linked to areas in the parietal lobe. These results suggest an evolutionary scenario in which the primate mirror neuron system (MNS) became increasingly lateralized, and later fissioned onto subsystems with one mediating language and the other mediating the execution and observation of manual actions. The second network is further subdivided into one dependent on hand preference and one that is not, providing new insight into the tripartite system of language, handedness, and praxis. Copyright © 2016 Elsevier Ltd. All rights reserved.

  18. Frontal and temporal contributions to understanding the iconic co-speech gestures that accompany speech.

    Science.gov (United States)

    Dick, Anthony Steven; Mok, Eva H; Raja Beharelle, Anjali; Goldin-Meadow, Susan; Small, Steven L

    2014-03-01

    In everyday conversation, listeners often rely on a speaker's gestures to clarify any ambiguities in the verbal message. Using fMRI during naturalistic story comprehension, we examined which brain regions in the listener are sensitive to speakers' iconic gestures. We focused on iconic gestures that contribute information not found in the speaker's talk, compared with those that convey information redundant with the speaker's talk. We found that three regions-left inferior frontal gyrus triangular (IFGTr) and opercular (IFGOp) portions, and left posterior middle temporal gyrus (MTGp)--responded more strongly when gestures added information to nonspecific language, compared with when they conveyed the same information in more specific language; in other words, when gesture disambiguated speech as opposed to reinforced it. An increased BOLD response was not found in these regions when the nonspecific language was produced without gesture, suggesting that IFGTr, IFGOp, and MTGp are involved in integrating semantic information across gesture and speech. In addition, we found that activity in the posterior superior temporal sulcus (STSp), previously thought to be involved in gesture-speech integration, was not sensitive to the gesture-speech relation. Together, these findings clarify the neurobiology of gesture-speech integration and contribute to an emerging picture of how listeners glean meaning from gestures that accompany speech. Copyright © 2012 Wiley Periodicals, Inc.

  19. Localization and Recognition of Dynamic Hand Gestures Based on Hierarchy of Manifold Classifiers

    Science.gov (United States)

    Favorskaya, M.; Nosov, A.; Popov, A.

    2015-05-01

    Generally, the dynamic hand gestures are captured in continuous video sequences, and a gesture recognition system ought to extract the robust features automatically. This task involves the highly challenging spatio-temporal variations of dynamic hand gestures. The proposed method is based on two-level manifold classifiers including the trajectory classifiers in any time instants and the posture classifiers of sub-gestures in selected time instants. The trajectory classifiers contain skin detector, normalized skeleton representation of one or two hands, and motion history representing by motion vectors normalized through predetermined directions (8 and 16 in our case). Each dynamic gesture is separated into a set of sub-gestures in order to predict a trajectory and remove those samples of gestures, which do not satisfy to current trajectory. The posture classifiers involve the normalized skeleton representation of palm and fingers and relative finger positions using fingertips. The min-max criterion is used for trajectory recognition, and the decision tree technique was applied for posture recognition of sub-gestures. For experiments, a dataset "Multi-modal Gesture Recognition Challenge 2013: Dataset and Results" including 393 dynamic hand-gestures was chosen. The proposed method yielded 84-91% recognition accuracy, in average, for restricted set of dynamic gestures.

  20. LOCALIZATION AND RECOGNITION OF DYNAMIC HAND GESTURES BASED ON HIERARCHY OF MANIFOLD CLASSIFIERS

    Directory of Open Access Journals (Sweden)

    M. Favorskaya

    2015-05-01

    Full Text Available Generally, the dynamic hand gestures are captured in continuous video sequences, and a gesture recognition system ought to extract the robust features automatically. This task involves the highly challenging spatio-temporal variations of dynamic hand gestures. The proposed method is based on two-level manifold classifiers including the trajectory classifiers in any time instants and the posture classifiers of sub-gestures in selected time instants. The trajectory classifiers contain skin detector, normalized skeleton representation of one or two hands, and motion history representing by motion vectors normalized through predetermined directions (8 and 16 in our case. Each dynamic gesture is separated into a set of sub-gestures in order to predict a trajectory and remove those samples of gestures, which do not satisfy to current trajectory. The posture classifiers involve the normalized skeleton representation of palm and fingers and relative finger positions using fingertips. The min-max criterion is used for trajectory recognition, and the decision tree technique was applied for posture recognition of sub-gestures. For experiments, a dataset “Multi-modal Gesture Recognition Challenge 2013: Dataset and Results” including 393 dynamic hand-gestures was chosen. The proposed method yielded 84–91% recognition accuracy, in average, for restricted set of dynamic gestures.

  1. Giving cognition a helping hand: the effect of congruent gestures on object name retrieval.

    Science.gov (United States)

    Pine, Karen J; Reeves, Lindsey; Howlett, Neil; Fletcher, Ben C

    2013-02-01

    The gestures that accompany speech are more than just arbitrary hand movements or communicative devices. They are simulated actions that can both prime and facilitate speech and cognition. This study measured participants' reaction times for naming degraded images of objects when simultaneously adopting a gesture that was either congruent with the target object, incongruent with it, and when not making any hand gesture. A within-subjects design was used, with participants (N= 122) naming 10 objects under each condition. Participants named the objects significantly faster when adopting a congruent gesture than when not gesturing at all. Adopting an incongruent gesture resulted in significantly slower naming times. The findings are discussed in the context of the intrapersonal cognitive and facilitatory effects of gestures and underline the relatedness between language, action, and cognition. © 2012 The British Psychological Society.

  2. Generating Culture-Specific Gestures for Virtual Agent Dialogs

    DEFF Research Database (Denmark)

    Endrass, Birgit; Damian, Ionut; Huber, Peter

    2010-01-01

    Integrating culture into the behavioral model of virtual agents has come into focus lately. When investigating verbal aspects of behavior, nonverbal behaviors are desirably added automatically, driven by the speech-act. In this paper, we present a corpus driven approach of generating gestures...

  3. Effects of a robotic storyteller's moody gestures on storytelling perception

    NARCIS (Netherlands)

    Xu, J.; Broekens, J.; Hindriks, K.; Neerincx, M.A.

    2015-01-01

    A parameterized behavior model was developed for robots to show mood during task execution. In this study, we applied the model to the coverbal gestures of a robotic storyteller. This study investigated whether parameterized mood expression can 1) show mood that is changing over time; 2) reinforce

  4. Static gesture recognition using features extracted from skeletal data

    CSIR Research Space (South Africa)

    Mangera, R

    2013-12-01

    Full Text Available -optimal classification accuracy. Therefore to improve the classification accuracy, a new feature vector, combining joint angles and the relative position of the arm joints with respect to the head, is proposed. A k-means classifier is used to cluster each gesture. New...

  5. RehabGesture: An Alternative Tool for Measuring Human Movement.

    Science.gov (United States)

    Brandão, Alexandre F; Dias, Diego R C; Castellano, Gabriela; Parizotto, Nivaldo A; Trevelin, Luis Carlos

    2016-07-01

    Systems for range of motion (ROM) measurement such as OptoTrak, Motion Capture, Motion Analysis, Vicon, and Visual 3D are so expensive that they become impracticable in public health systems and even in private rehabilitation clinics. Telerehabilitation is a branch within telemedicine intended to offer ways to increase motor and/or cognitive stimuli, aimed at faster and more effective recovery of given disabilities, and to measure kinematic data such as the improvement in ROM. In the development of the RehabGesture tool, we used the gesture recognition sensor Kinect(®) (Microsoft, Redmond, WA) and the concepts of Natural User Interface and Open Natural Interaction. RehabGesture can measure and record the ROM during rehabilitation sessions while the user interacts with the virtual reality environment. The software allows the measurement of the ROM (in the coronal plane) from 0° extension to 145° flexion of the elbow joint, as well as from 0° adduction to 180° abduction of the glenohumeral (shoulder) joint, leaving the standing position. The proposed tool has application in the fields of training and physical evaluation of professional and amateur athletes in clubs and gyms and may have application in rehabilitation and physiotherapy clinics for patients with compromised motor abilities. RehabGesture represents a low-cost solution to measure the movement of the upper limbs, as well as to stimulate the process of teaching and learning in disciplines related to the study of human movement, such as kinesiology.

  6. Recognition of sign language gestures using neural networks

    Directory of Open Access Journals (Sweden)

    Simon Vamplew

    2007-04-01

    Full Text Available This paper describes the structure and performance of the SLARTI sign language recognition system developed at the University of Tasmania. SLARTI uses a modular architecture consisting of multiple feature-recognition neural networks and a nearest-neighbour classifier to recognise Australian sign language (Auslan hand gestures.

  7. Recognition of sign language gestures using neural networks

    OpenAIRE

    Simon Vamplew

    2007-01-01

    This paper describes the structure and performance of the SLARTI sign language recognition system developed at the University of Tasmania. SLARTI uses a modular architecture consisting of multiple feature-recognition neural networks and a nearest-neighbour classifier to recognise Australian sign language (Auslan) hand gestures.

  8. Onomatopoeia, Gesture, and Synaesthesia in the Perception of Poetic Meaning.

    Science.gov (United States)

    Salper, Donald R.

    The author states that phonetic symbolism is not a generalizable phenomenon but maintains that those interested in the status of a poem as a speech event need not totally discount or discredit such perceptions. In his discussion of the theories which ascribe meaning to vocal utterance--the two imitative theories, the onomatopoeic and the gestural,…

  9. Gesture and Signing in Support of Expressive Language Development

    Science.gov (United States)

    Baker-Ramos, Leslie K.

    2017-01-01

    The purpose of this teacher inquiry is to explore the effects of signing and gesturing on the expressive language development of non-verbal children. The first phase of my inquiry begins with the observations of several non-verbal students with various etiologies in three different educational settings. The focus of these observations is to…

  10. Children's Use of Gesture to Resolve Lexical Ambiguity

    Science.gov (United States)

    Kidd, Evan; Holler, Judith

    2009-01-01

    We report on a study investigating 3-5-year-old children's use of gesture to resolve lexical ambiguity. Children were told three short stories that contained two homonym senses; for example, "bat" (flying mammal) and "bat" (sports equipment). They were then asked to re-tell these stories to a second experimenter. The data were coded for the means…

  11. Cascading neural networks for upper-body gesture recognition

    CSIR Research Space (South Africa)

    Mangera, R

    2014-01-01

    Full Text Available Gesture recognition has many applications ranging from health care to entertainment. However for it to be a feasible method of human-computer interaction it is essential that only intentional movements are interpreted and that the system can work...

  12. Gesturing on the Telephone: Independent Effects of Dialogue and Visibility

    Science.gov (United States)

    Bavelas, Janet; Gerwing, Jennifer; Sutton, Chantelle; Prevost, Danielle

    2008-01-01

    Speakers often gesture in telephone conversations, even though they are not visible to their addressees. To test whether this effect is due to being in a dialogue, we separated visibility and dialogue with three conditions: face-to-face dialogue (10 dyads), telephone dialogue (10 dyads), and monologue to a tape recorder (10 individuals). For the…

  13. Nearest neighbour classification of Indian sign language gestures ...

    Indian Academy of Sciences (India)

    In the ideal case, a gesture recognition ... Every geographical region has developed its own sys- ... et al [10] present a study on vision-based static hand shape .... tures, and neural networks for recognition. ..... We used the city-block dis-.

  14. Sound Synthesis Affected by Physical Gestures in Real-Time

    DEFF Research Database (Denmark)

    Graugaard, Lars

    2006-01-01

    Motivation and strategies for affecting electronic music through physical gestures are presented and discussed. Two implementations are presented and experience with their use in performance is reported. A concept of sound shaping and sound colouring that connects an instrumental performer......’s playing and gesturest to sound synthesis is used. The results and future possibilities are discussed....

  15. From facial expressions to bodily gestures: Passions, photography and movement in French 19th-century sciences.

    Science.gov (United States)

    Pichel, Beatriz

    2016-02-01

    This article aims to determine to what extent photographic practices in psychology, psychiatry and physiology contributed to the definition of the external bodily signs of passions and emotions in the second half of the 19 th century in France. Bridging the gap between recent research in the history of emotions and photographic history, the following analyses focus on the photographic production of scientists and photographers who made significant contributions to the study of expressions and gestures, namely Duchenne de Boulogne, Charles Darwin, Paul Richer and Albert Londe. This article argues that photography became a key technology in their works due to the adequateness of the exposure time of different cameras to the duration of the bodily manifestations to be recorded, and that these uses constituted facial expressions and bodily gestures as particular objects for the scientific study.

  16. Give me a hand: Differential effects of gesture type in guiding young children's problem-solving.

    Science.gov (United States)

    Vallotton, Claire; Fusaro, Maria; Hayden, Julia; Decker, Kalli; Gutowski, Elizabeth

    2015-11-01

    Adults' gestures support children's learning in problem-solving tasks, but gestures may be differentially useful to children of different ages, and different features of gestures may make them more or less useful to children. The current study investigated parents' use of gestures to support their young children (1.5 - 6 years) in a block puzzle task (N = 126 parent-child dyads), and identified patterns in parents' gesture use indicating different gestural strategies. Further, we examined the effect of child age on both the frequency and types of gestures parents used, and on their usefulness to support children's learning. Children attempted to solve the puzzle independently before and after receiving help from their parent; half of the parents were instructed to sit on their hands while they helped. Parents who could use their hands appear to use gestures in three strategies: orienting the child to the task, providing abstract information, and providing embodied information; further, they adapted their gesturing to their child's age and skill level. Younger children elicited more frequent and more proximal gestures from parents. Despite the greater use of gestures with younger children, it was the oldest group (4.5-6.0 years) who were most affected by parents' gestures. The oldest group was positively affected by the total frequency of parents' gestures, and in particular, parents' use of embodying gestures (indexes that touched their referents, representational demonstrations with object in hand, and physically guiding child's hands). Though parents rarely used the embodying strategy with older children, it was this strategy which most enhanced the problem-solving of children 4.5 - 6 years.

  17. Give me a hand: Differential effects of gesture type in guiding young children's problem-solving

    Science.gov (United States)

    Vallotton, Claire; Fusaro, Maria; Hayden, Julia; Decker, Kalli; Gutowski, Elizabeth

    2015-01-01

    Adults’ gestures support children's learning in problem-solving tasks, but gestures may be differentially useful to children of different ages, and different features of gestures may make them more or less useful to children. The current study investigated parents’ use of gestures to support their young children (1.5 – 6 years) in a block puzzle task (N = 126 parent-child dyads), and identified patterns in parents’ gesture use indicating different gestural strategies. Further, we examined the effect of child age on both the frequency and types of gestures parents used, and on their usefulness to support children's learning. Children attempted to solve the puzzle independently before and after receiving help from their parent; half of the parents were instructed to sit on their hands while they helped. Parents who could use their hands appear to use gestures in three strategies: orienting the child to the task, providing abstract information, and providing embodied information; further, they adapted their gesturing to their child's age and skill level. Younger children elicited more frequent and more proximal gestures from parents. Despite the greater use of gestures with younger children, it was the oldest group (4.5-6.0 years) who were most affected by parents’ gestures. The oldest group was positively affected by the total frequency of parents’ gestures, and in particular, parents’ use of embodying gestures (indexes that touched their referents, representational demonstrations with object in hand, and physically guiding child's hands). Though parents rarely used the embodying strategy with older children, it was this strategy which most enhanced the problem-solving of children 4.5 – 6 years. PMID:26848192

  18. Generating Control Commands From Gestures Sensed by EMG

    Science.gov (United States)

    Wheeler, Kevin R.; Jorgensen, Charles

    2006-01-01

    An effort is under way to develop noninvasive neuro-electric interfaces through which human operators could control systems as diverse as simple mechanical devices, computers, aircraft, and even spacecraft. The basic idea is to use electrodes on the surface of the skin to acquire electromyographic (EMG) signals associated with gestures, digitize and process the EMG signals to recognize the gestures, and generate digital commands to perform the actions signified by the gestures. In an experimental prototype of such an interface, the EMG signals associated with hand gestures are acquired by use of several pairs of electrodes mounted in sleeves on a subject s forearm (see figure). The EMG signals are sampled and digitized. The resulting time-series data are fed as input to pattern-recognition software that has been trained to distinguish gestures from a given gesture set. The software implements, among other things, hidden Markov models, which are used to recognize the gestures as they are being performed in real time. Thus far, two experiments have been performed on the prototype interface to demonstrate feasibility: an experiment in synthesizing the output of a joystick and an experiment in synthesizing the output of a computer or typewriter keyboard. In the joystick experiment, the EMG signals were processed into joystick commands for a realistic flight simulator for an airplane. The acting pilot reached out into the air, grabbed an imaginary joystick, and pretended to manipulate the joystick to achieve left and right banks and up and down pitches of the simulated airplane. In the keyboard experiment, the subject pretended to type on a numerical keypad, and the EMG signals were processed into keystrokes. The results of the experiments demonstrate the basic feasibility of this method while indicating the need for further research to reduce the incidence of errors (including confusion among gestures). Topics that must be addressed include the numbers and arrangements

  19. Car Gestures - Advisory warning using additional steering wheel angles.

    Science.gov (United States)

    Maag, Christian; Schneider, Norbert; Lübbeke, Thomas; Weisswange, Thomas H; Goerick, Christian

    2015-10-01

    Advisory warning systems (AWS) notify the driver about upcoming hazards. This is in contrast to the majority of currently deployed advanced driver assistance systems (ADAS) that manage emergency situations. The target of this study is to investigate the effectiveness, acceptance, and controllability of a specific kind of AWS that uses the haptic information channel for warning the driver. This could be beneficial, as alternatives for using the visual modality can help to reduce the risk of visual overload. The driving simulator study (N=24) compared an AWS based on additional steering wheel angle control (Car Gestures) with a visual warning presented in a simulated head-up display (HUD). Both types of warning were activated 3.5s before the hazard object was reached. An additional condition of unassisted driving completed the experimental design. The subjects encountered potential hazards in a variety of urban situations (e.g. a pedestrian standing on the curbs). For the investigated situations, subjective ratings show that a majority of drivers prefer visual warnings over haptic information via gestures. An analysis of driving behavior indicates that both warning approaches guide the vehicle away from the potential hazard. Whereas gestures lead to a faster lateral driving reaction (compared to HUD warnings), the visual warnings result in a greater safety benefit (measured by the minimum distance to the hazard object). A controllability study with gestures in the wrong direction (i.e. leading toward the hazard object) shows that drivers are able to cope with wrong haptic warnings and safety is not reduced compared to unassisted driving as well as compared to (correct) haptic gestures and visual warnings. Copyright © 2015 Elsevier Ltd. All rights reserved.

  20. Personalized gesture interactions for cyber-physical smart-home environments

    Institute of Scientific and Technical Information of China (English)

    Yihua LOU; Wenjun WU; Radu-Daniel VATAVU; Wei-Tek TSAI

    2017-01-01

    A gesture-based interaction system for smart homes is a part of a complex cyber-physical environment,for which researchers and developers need to address major challenges in providing personalized gesture interactions.However,current research efforts have not tackled the problem of personalized gesture recognition that often involves user identification.To address this problem,we propose in this work a new event-driven service-oriented framework called gesture services for cyber-physical environments (GS-CPE) that extends the architecture of our previous work gesture profile for web services (GPWS).To provide user identification functionality,GS-CPE introduces a two-phase cascading gesture password recognition algorithm for gesture-based user identification using a two-phase cascading classifier with the hidden Markov model and the Golden Section Search,which achieves an accuracy rate of 96.2% with a small training dataset.To support personalized gesture interaction,an enhanced version of the Dynamic Time Warping algorithm with multiple gestural input sources and dynamic template adaptation support is implemented.Our experimental results demonstrate the performance of the algorithm can achieve an average accuracy rate of 98.5% in practical scenarios.Comparison results reveal that GS-CPE has faster response time and higher accuracy rate than other gesture interaction systems designed for smart-home environments.

  1. Beating time: How ensemble musicians' cueing gestures communicate beat position and tempo.

    Science.gov (United States)

    Bishop, Laura; Goebl, Werner

    2018-01-01

    Ensemble musicians typically exchange visual cues to coordinate piece entrances. "Cueing-in" gestures indicate when to begin playing and at what tempo. This study investigated how timing information is encoded in musicians' cueing-in gestures. Gesture acceleration patterns were expected to indicate beat position, while gesture periodicity, duration, and peak gesture velocity were expected to indicate tempo. Same-instrument ensembles (e.g., piano-piano) were expected to synchronize more successfully than mixed-instrument ensembles (e.g., piano-violin). Duos performed short passages as their head and (for violinists) bowing hand movements were tracked with accelerometers and Kinect sensors. Performers alternated between leader/follower roles; leaders heard a tempo via headphones and cued their partner in nonverbally. Violin duos synchronized more successfully than either piano duos or piano-violin duos, possibly because violinists were more experienced in ensemble playing than pianists. Peak acceleration indicated beat position in leaders' head-nodding gestures. Gesture duration and periodicity in leaders' head and bowing hand gestures indicated tempo. The results show that the spatio-temporal characteristics of cueing-in gestures guide beat perception, enabling synchronization with visual gestures that follow a range of spatial trajectories.

  2. Multisensory integration: the case of a time window of gesture-speech integration.

    Science.gov (United States)

    Obermeier, Christian; Gunter, Thomas C

    2015-02-01

    This experiment investigates the integration of gesture and speech from a multisensory perspective. In a disambiguation paradigm, participants were presented with short videos of an actress uttering sentences like "She was impressed by the BALL, because the GAME/DANCE...." The ambiguous noun (BALL) was accompanied by an iconic gesture fragment containing information to disambiguate the noun toward its dominant or subordinate meaning. We used four different temporal alignments between noun and gesture fragment: the identification point (IP) of the noun was either prior to (+120 msec), synchronous with (0 msec), or lagging behind the end of the gesture fragment (-200 and -600 msec). ERPs triggered to the IP of the noun showed significant differences for the integration of dominant and subordinate gesture fragments in the -200, 0, and +120 msec conditions. The outcome of this integration was revealed at the target words. These data suggest a time window for direct semantic gesture-speech integration ranging from at least -200 up to +120 msec. Although the -600 msec condition did not show any signs of direct integration at the homonym, significant disambiguation was found at the target word. An explorative analysis suggested that gesture information was directly integrated at the verb, indicating that there are multiple positions in a sentence where direct gesture-speech integration takes place. Ultimately, this would implicate that in natural communication, where a gesture lasts for some time, several aspects of that gesture will have their specific and possibly distinct impact on different positions in an utterance.

  3. Selection of suitable hand gestures for reliable myoelectric human computer interface.

    Science.gov (United States)

    Castro, Maria Claudia F; Arjunan, Sridhar P; Kumar, Dinesh K

    2015-04-09

    Myoelectric controlled prosthetic hand requires machine based identification of hand gestures using surface electromyogram (sEMG) recorded from the forearm muscles. This study has observed that a sub-set of the hand gestures have to be selected for an accurate automated hand gesture recognition, and reports a method to select these gestures to maximize the sensitivity and specificity. Experiments were conducted where sEMG was recorded from the muscles of the forearm while subjects performed hand gestures and then was classified off-line. The performances of ten gestures were ranked using the proposed Positive-Negative Performance Measurement Index (PNM), generated by a series of confusion matrices. When using all the ten gestures, the sensitivity and specificity was 80.0% and 97.8%. After ranking the gestures using the PNM, six gestures were selected and these gave sensitivity and specificity greater than 95% (96.5% and 99.3%); Hand open, Hand close, Little finger flexion, Ring finger flexion, Middle finger flexion and Thumb flexion. This work has shown that reliable myoelectric based human computer interface systems require careful selection of the gestures that have to be recognized and without such selection, the reliability is poor.

  4. Coronary Heart Disease Preoperative Gesture Interactive Diagnostic System Based on Augmented Reality.

    Science.gov (United States)

    Zou, Yi-Bo; Chen, Yi-Min; Gao, Ming-Ke; Liu, Quan; Jiang, Si-Yu; Lu, Jia-Hui; Huang, Chen; Li, Ze-Yu; Zhang, Dian-Hua

    2017-08-01

    Coronary heart disease preoperative diagnosis plays an important role in the treatment of vascular interventional surgery. Actually, most doctors are used to diagnosing the position of the vascular stenosis and then empirically estimating vascular stenosis by selective coronary angiography images instead of using mouse, keyboard and computer during preoperative diagnosis. The invasive diagnostic modality is short of intuitive and natural interaction and the results are not accurate enough. Aiming at above problems, the coronary heart disease preoperative gesture interactive diagnostic system based on Augmented Reality is proposed. The system uses Leap Motion Controller to capture hand gesture video sequences and extract the features which that are the position and orientation vector of the gesture motion trajectory and the change of the hand shape. The training planet is determined by K-means algorithm and then the effect of gesture training is improved by multi-features and multi-observation sequences for gesture training. The reusability of gesture is improved by establishing the state transition model. The algorithm efficiency is improved by gesture prejudgment which is used by threshold discriminating before recognition. The integrity of the trajectory is preserved and the gesture motion space is extended by employing space rotation transformation of gesture manipulation plane. Ultimately, the gesture recognition based on SRT-HMM is realized. The diagnosis and measurement of the vascular stenosis are intuitively and naturally realized by operating and measuring the coronary artery model with augmented reality and gesture interaction techniques. All of the gesture recognition experiments show the distinguish ability and generalization ability of the algorithm and gesture interaction experiments prove the availability and reliability of the system.

  5. Distinguishing the processing of gestures from signs in deaf individuals: an fMRI study.

    Science.gov (United States)

    Husain, Fatima T; Patkin, Debra J; Thai-Van, Hung; Braun, Allen R; Horwitz, Barry

    2009-06-18

    Manual gestures occur on a continuum from co-speech gesticulations to conventionalized emblems to language signs. Our goal in the present study was to understand the neural bases of the processing of gestures along such a continuum. We studied four types of gestures, varying along linguistic and semantic dimensions: linguistic and meaningful American Sign Language (ASL), non-meaningful pseudo-ASL, meaningful emblematic, and nonlinguistic, non-meaningful made-up gestures. Pre-lingually deaf, native signers of ASL participated in the fMRI study and performed two tasks while viewing videos of the gestures: a visuo-spatial (identity) discrimination task and a category discrimination task. We found that the categorization task activated left ventral middle and inferior frontal gyrus, among other regions, to a greater extent compared to the visual discrimination task, supporting the idea of semantic-level processing of the gestures. The reverse contrast resulted in enhanced activity of bilateral intraparietal sulcus, supporting the idea of featural-level processing (analogous to phonological-level processing of speech sounds) of the gestures. Regardless of the task, we found that brain activation patterns for the nonlinguistic, non-meaningful gestures were the most different compared to the ASL gestures. The activation patterns for the emblems were most similar to those of the ASL gestures and those of the pseudo-ASL were most similar to the nonlinguistic, non-meaningful gestures. The fMRI results provide partial support for the conceptualization of different gestures as belonging to a continuum and the variance in the fMRI results was best explained by differences in the processing of gestures along the semantic dimension.

  6. [George Herbert Mead. Thought as the conversation of interior gestures].

    Science.gov (United States)

    Quéré, Louis

    2010-01-01

    For George Herbert Mead, thinking amounts to holding an "inner conversation of gestures ". Such a conception does not seem especially original at first glance. What makes it truly original is the "social-behavioral" approach of which it is a part, and, particularly, two ideas. The first is that the conversation in question is a conversation of gestures or attitudes, and the second, that thought and reflexive intelligence arise from the internalization of an external process supported by the social mechanism of communication: that of conduct organization. It imports then to understand what distinguishes such ideas from those of the founder of behavioral psychology, John B. Watson, for whom thinking amounts to nothing other than subvocal speech.

  7. Gesture recognition for smart home applications using portable radar sensors.

    Science.gov (United States)

    Wan, Qian; Li, Yiran; Li, Changzhi; Pal, Ranadip

    2014-01-01

    In this article, we consider the design of a human gesture recognition system based on pattern recognition of signatures from a portable smart radar sensor. Powered by AAA batteries, the smart radar sensor operates in the 2.4 GHz industrial, scientific and medical (ISM) band. We analyzed the feature space using principle components and application-specific time and frequency domain features extracted from radar signals for two different sets of gestures. We illustrate that a nearest neighbor based classifier can achieve greater than 95% accuracy for multi class classification using 10 fold cross validation when features are extracted based on magnitude differences and Doppler shifts as compared to features extracted through orthogonal transformations. The reported results illustrate the potential of intelligent radars integrated with a pattern recognition system for high accuracy smart home and health monitoring purposes.

  8. Historical-critical objects and gestures: Hebreia by Fabio Mauri

    Directory of Open Access Journals (Sweden)

    Maria Augusta Vilalba Nunes

    2015-09-01

    Full Text Available In 1971, Italian artist Fabio Mauri produces the performance / installation artwork Ebrea, which emerges a memory of the Shoah through the performatic gestures and sculpture-objects that make up the scene. In Ebrea the time unfolds and bring back to present the memory of a catastrophe. So, over twenty years after the end of the war, why would Mauri up this memory that was getting distant and was beginning to have an unrealistic form? The anachronism of Mauri’s gesture is intrinsically connected to the need of render account to the hidden memories that don’t leave him and whose tracks, in his opinion, should not be erased from history.

  9. Finger tips detection for two handed gesture recognition

    Science.gov (United States)

    Bhuyan, M. K.; Kar, Mithun Kumar; Neog, Debanga Raj

    2011-10-01

    In this paper, a novel algorithm is proposed for fingertips detection in view of two-handed static hand pose recognition. In our method, finger tips of both hands are detected after detecting hand regions by skin color-based segmentation. At first, the face is removed in the image by using Haar classifier and subsequently, the regions corresponding to the gesturing hands are isolated by a region labeling technique. Next, the key geometric features characterizing gesturing hands are extracted for two hands. Finally, for all possible/allowable finger movements, a probabilistic model is developed for pose recognition. Proposed method can be employed in a variety of applications like sign language recognition and human-robot-interactions etc.

  10. Feasibility of interactive gesture control of a robotic microscope

    Directory of Open Access Journals (Sweden)

    Antoni Sven-Thomas

    2015-09-01

    Full Text Available Robotic devices become increasingly available in the clinics. One example are motorized surgical microscopes. While there are different scenarios on how to use the devices for autonomous tasks, simple and reliable interaction with the device is a key for acceptance by surgeons. We study, how gesture tracking can be integrated within the setup of a robotic microscope. In our setup, a Leap Motion Controller is used to track hand motion and adjust the field of view accordingly. We demonstrate with a survey that moving the field of view over a specified course is possible even for untrained subjects. Our results indicate that touch-less interaction with robots carrying small, near field gesture sensors is feasible and can be of use in clinical scenarios, where robotic devices are used in direct proximity of patient and physicians.

  11. Gesture Interaction Browser-Based 3D Molecular Viewer.

    Science.gov (United States)

    Virag, Ioan; Stoicu-Tivadar, Lăcrămioara; Crişan-Vida, Mihaela

    2016-01-01

    The paper presents an open source system that allows the user to interact with a 3D molecular viewer using associated hand gestures for rotating, scaling and panning the rendered model. The novelty of this approach is that the entire application is browser-based and doesn't require installation of third party plug-ins or additional software components in order to visualize the supported chemical file formats. This kind of solution is suitable for instruction of users in less IT oriented environments, like medicine or chemistry. For rendering various molecular geometries our team used GLmol (a molecular viewer written in JavaScript). The interaction with the 3D models is made with Leap Motion controller that allows real-time tracking of the user's hand gestures. The first results confirmed that the resulting application leads to a better way of understanding various types of translational bioinformatics related problems in both biomedical research and education.

  12. Perspectives on gesture from music informatics, performance and aesthetics

    DEFF Research Database (Denmark)

    Jensen, Kristoffer; Frimodt-Møller, Søren; Grund, Cynthia

    2014-01-01

    This article chronicles the research of the Nordic Network of Music Informatics, Performance and Aesthetics (NNIMIPA), and shows how the milieux bridge the gap between the disciplines involved. As examples, three projects within NNIMIPA involving performance interaction examine the role of audio...... and gestures in emotional musical expression using motion capture, the visual and auditive cues musicians provide each other in an ensemble when rehearsing, and the decision processes involved when a musician coordinates with other musicians. These projects seek to combine and compare intuitions derived from...... low-tech instructional music workshops that rely heavily on the use of whole-body gestures with the insights provided by high-tech studies and formal logic models of the performing musician, not only with respect to the sound, but also with regard to the movements of the performer and the mechanisms...

  13. Intelligent RF-Based Gesture Input Devices Implemented Using e-Textiles

    Directory of Open Access Journals (Sweden)

    Dana Hughes

    2017-01-01

    Full Text Available We present an radio-frequency (RF-based approach to gesture detection and recognition, using e-textile versions of common transmission lines used in microwave circuits. This approach allows for easy fabrication of input swatches that can detect a continuum of finger positions and similarly basic gestures, using a single measurement line. We demonstrate that the swatches can perform gesture detection when under thin layers of cloth or when weatherproofed, providing a high level of versatility not present with other types of approaches. Additionally, using small convolutional neural networks, low-level gestures can be identified with a high level of accuracy using a small, inexpensive microcontroller, allowing for an intelligent fabric that reports only gestures of interest, rather than a simple sensor requiring constant surveillance from an external computing device. The resulting e-textile smart composite has applications in controlling wearable devices by providing a simple, eyes-free mechanism to input simple gestures.

  14. Music Conductor Gesture Recognized Interactive Music Generation System

    OpenAIRE

    CHEN, Shuai; MAEDA, Yoichiro; TAKAHASHI, Yasutake

    2012-01-01

    In the research of interactive music generation, we propose a music generation method, that the computer generates the music automatically, and then the music will be arranged under the human music conductor's gestures, before it outputs to us. In this research, the generated music is processed from chaotic sound, which is generated from the network of chaotic elements in realtime. The music conductor's hand motions are detected by Microsoft Kinect in this system. Music theories are embedded ...

  15. Basic Hand Gestures Classification Based on Surface Electromyography

    Directory of Open Access Journals (Sweden)

    Aleksander Palkowski

    2016-01-01

    Full Text Available This paper presents an innovative classification system for hand gestures using 2-channel surface electromyography analysis. The system developed uses the Support Vector Machine classifier, for which the kernel function and parameter optimisation are conducted additionally by the Cuckoo Search swarm algorithm. The system developed is compared with standard Support Vector Machine classifiers with various kernel functions. The average classification rate of 98.12% has been achieved for the proposed method.

  16. Gestural representation of event structure in dyadic interaction

    OpenAIRE

    Christensen, Peer; Tylén, Kristian

    2013-01-01

    What are the underlying motivations for the conceptualization of events? Recent studies show that when people are asked to use nonverbal gestures to describe transitive events they prefer the semantic order Agent-Patient-Act, analogous to SOV in grammatical terms. The original explanation has been that this pattern reflects a cognitively “natural order” for the conceptualization of events. However, other types of transitive events have not been investigated in earlier studies. We report exper...

  17. Surgical gesture classification from video and kinematic data.

    Science.gov (United States)

    Zappella, Luca; Béjar, Benjamín; Hager, Gregory; Vidal, René

    2013-10-01

    Much of the existing work on automatic classification of gestures and skill in robotic surgery is based on dynamic cues (e.g., time to completion, speed, forces, torque) or kinematic data (e.g., robot trajectories and velocities). While videos could be equally or more discriminative (e.g., videos contain semantic information not present in kinematic data), they are typically not used because of the difficulties associated with automatic video interpretation. In this paper, we propose several methods for automatic surgical gesture classification from video data. We assume that the video of a surgical task (e.g., suturing) has been segmented into video clips corresponding to a single gesture (e.g., grabbing the needle, passing the needle) and propose three methods to classify the gesture of each video clip. In the first one, we model each video clip as the output of a linear dynamical system (LDS) and use metrics in the space of LDSs to classify new video clips. In the second one, we use spatio-temporal features extracted from each video clip to learn a dictionary of spatio-temporal words, and use a bag-of-features (BoF) approach to classify new video clips. In the third one, we use multiple kernel learning (MKL) to combine the LDS and BoF approaches. Since the LDS approach is also applicable to kinematic data, we also use MKL to combine both types of data in order to exploit their complementarity. Our experiments on a typical surgical training setup show that methods based on video data perform equally well, if not better, than state-of-the-art approaches based on kinematic data. In turn, the combination of both kinematic and video data outperforms any other algorithm based on one type of data alone. Copyright © 2013 Elsevier B.V. All rights reserved.

  18. Designing natural gesture interaction for archaeological data in immersive environments

    Directory of Open Access Journals (Sweden)

    Niccolò Albertini

    2017-05-01

    Full Text Available Archaeological data are heterogeneous, making it difficult to correlate and combine different types.  Datasheets  and pictures,  stratigraphic  data  and  3D  models,  time and  space  mixed  together: these are  only a few  of  the  categories  a researcher has to deal with. New technologies may be able to help in this process and trying to solve research related problems needs innovative solutions. In this paper, we describe the whole process for the design and development of a prototype application that uses an Immersive Virtual Reality system to acces archaeological excavation3Ddata through the Gesture Variation Follower (GVF algorithm. This makes it possible to recognise which gesture is being performed and how it is performed. Archaeologist shave participated actively in the design of the interface and the set of gestures used for triggering the different tasks. Interactive machine learning techniques have been used for the real time detection of the gestures. As a case  study  the  agora  of  Segesta  (Sicily,  Italy  has  been  selected.  Indeed,  due  to  the  complex architectural  features  and  the  still  ongoing  fieldwork  activities,  Segesta  represents  an  ideal  context  where  to  test  and develop a research approach integrating both traditional and more innovative tools and methods.

  19. ConductHome: Gesture Interface Control of Home Automation Boxes

    OpenAIRE

    J. Branstett; V. Gagneux; A. Leleu; B. Levadoux; J. Pascale

    2015-01-01

    This paper presents the interface ConductHome which controls home automation systems with a Leap Motion using "invariant gesture protocols". This interface is meant to simplify the interaction of the user with its environment. A hardware part allows the Leap Motion to be carried around the house. A software part interacts with the home automation box and displays the useful information for the user. An objective of this work is the development of a natural/invariant/simpl...

  20. Voice and gesture-based 3D multimedia presentation tool

    Science.gov (United States)

    Fukutake, Hiromichi; Akazawa, Yoshiaki; Okada, Yoshihiro

    2007-09-01

    This paper proposes a 3D multimedia presentation tool that allows the user to manipulate intuitively only through the voice input and the gesture input without using a standard keyboard or a mouse device. The authors developed this system as a presentation tool to be used in a presentation room equipped a large screen like an exhibition room in a museum because, in such a presentation environment, it is better to use voice commands and the gesture pointing input rather than using a keyboard or a mouse device. This system was developed using IntelligentBox, which is a component-based 3D graphics software development system. IntelligentBox has already provided various types of 3D visible, reactive functional components called boxes, e.g., a voice input component and various multimedia handling components. IntelligentBox also provides a dynamic data linkage mechanism called slot-connection that allows the user to develop 3D graphics applications by combining already existing boxes through direct manipulations on a computer screen. Using IntelligentBox, the 3D multimedia presentation tool proposed in this paper was also developed as combined components only through direct manipulations on a computer screen. The authors have already proposed a 3D multimedia presentation tool using a stage metaphor and its voice input interface. This time, we extended the system to make it accept the user gesture input besides voice commands. This paper explains details of the proposed 3D multimedia presentation tool and especially describes its component-based voice and gesture input interfaces.

  1. Developing A Physical Gesture Acquisition System for Guqin Performance

    OpenAIRE

    He, Jingyin; Kapur, Ajay; Carnegie, Dale

    2015-01-01

    Motion- based musical interfaces are ubiquitous. With the plethora of sensing solutions and the possibility of developing custom designs, it is important that the new musical interface has the capability to perform any number of tasks. This paper presents the theoretical framework for defining, designing, and evaluation process of a physical gesture acquisition for Guqin performance. The framework is based on an iterative design process, and draws upon the knowledge in Guqin performance to de...

  2. Gestures of grieving and mourning: a transhistoric dance-scheme

    OpenAIRE

    Briand , Michel

    2013-01-01

    International audience; This short analysis refers to cultural anthropology and aesthetics of dance, and intends to present a few remarkable steps in the long history of a special kind of danced gestures: expressions of feelings and representations of activities related to grieving and mourning, like lifting up hands in the air or upon one’s head and dramatically waving long hair. The focus is set on some universals and similarities as well as on contextualized variations and differences, in ...

  3. The importance of gestural communication: a study of human-dog communication using incongruent information.

    Science.gov (United States)

    D'Aniello, Biagio; Scandurra, Anna; Alterisio, Alessandra; Valsecchi, Paola; Prato-Previde, Emanuela

    2016-11-01

    We assessed how water rescue dogs, which were equally accustomed to respond to gestural and verbal requests, weighted gestural versus verbal information when asked by their owner to perform an action. Dogs were asked to perform four different actions ("sit", "lie down", "stay", "come") providing them with a single source of information (in Phase 1, gestural, and in Phase 2, verbal) or with incongruent information (in Phase 3, gestural and verbal commands referred to two different actions). In Phases 1 and 2, we recorded the frequency of correct responses as 0 or 1, whereas in Phase 3, we computed a 'preference index' (percentage of gestural commands followed over the total commands responded). Results showed that dogs followed gestures significantly better than words when these two types of information were used separately. Females were more likely to respond to gestural than verbal commands and males responded to verbal commands significantly better than females. In the incongruent condition, when gestures and words simultaneously indicated two different actions, the dogs overall preferred to execute the action required by the gesture rather than that required verbally, except when the verbal command "come" was paired with the gestural command "stay" with the owner moving away from the dog. Our data suggest that in dogs accustomed to respond to both gestural and verbal requests, gestures are more salient than words. However, dogs' responses appeared to be dependent also on the contextual situation: dogs' motivation to maintain proximity with an owner who was moving away could have led them to make the more 'convenient' choices between the two incongruent instructions.

  4. Effects of observing and producing deictic gestures on memory and learning in different age groups

    OpenAIRE

    Ouwehand, Kim

    2016-01-01

    markdownabstractThe studies presented in this dissertation aimed to investigate whether observing or producing deictic gestures (i.e., pointing and tracing gestures to index a referent in space or a movement pathway), could facilitate memory and learning in children, young adults, and older adults. More specifically, regarding memory it was investigated whether the use of deictic gestures would improve performance on tasks targeting cognitive functions that are found to change with age (worki...

  5. A Natural Interaction Interface for UAVs Using Intuitive Gesture Recognition

    Science.gov (United States)

    Chandarana, Meghan; Trujillo, Anna; Shimada, Kenji; Allen, Danette

    2016-01-01

    The popularity of unmanned aerial vehicles (UAVs) is increasing as technological advancements boost their favorability for a broad range of applications. One application is science data collection. In fields like Earth and atmospheric science, researchers are seeking to use UAVs to augment their current portfolio of platforms and increase their accessibility to geographic areas of interest. By increasing the number of data collection platforms UAVs will significantly improve system robustness and allow for more sophisticated studies. Scientists would like be able to deploy an available fleet of UAVs to fly a desired flight path and collect sensor data without needing to understand the complex low-level controls required to describe and coordinate such a mission. A natural interaction interface for a Ground Control System (GCS) using gesture recognition is developed to allow non-expert users (e.g., scientists) to define a complex flight path for a UAV using intuitive hand gesture inputs from the constructed gesture library. The GCS calculates the combined trajectory on-line, verifies the trajectory with the user, and sends it to the UAV controller to be flown.

  6. Gestural Interaction for Virtual Reality Environments through Data Gloves

    Directory of Open Access Journals (Sweden)

    G. Rodriguez

    2017-05-01

    Full Text Available In virtual environments, virtual hand interactions play a key role in interactivity and realism allowing to perform fine motions. Data glove is widely used in Virtual Reality (VR and through simulating a human hands natural anatomy (Avatar’s hands in its appearance and motion is possible to interact with the environment and virtual objects. Recently, hand gestures are considered as one of the most meaningful and expressive signals. As consequence, this paper explores the use of hand gestures as a mean of Human-Computer Interaction (HCI for VR applications through data gloves. Using a hand gesture recognition and tracking method, accurate and real-time interactive performance can be obtained. To verify the effectiveness and usability of the system, an experiment of ease learning based on execution’s time was performed. The experimental results demonstrate that this interaction’s approach does not present problems for people more experienced in the use of computer applications. While people with basic knowledge has some problems the system becomes easy to use with practice.

  7. Learning Semantics of Gestural Instructions for Human-Robot Collaboration

    Science.gov (United States)

    Shukla, Dadhichi; Erkent, Özgür; Piater, Justus

    2018-01-01

    Designed to work safely alongside humans, collaborative robots need to be capable partners in human-robot teams. Besides having key capabilities like detecting gestures, recognizing objects, grasping them, and handing them over, these robots need to seamlessly adapt their behavior for efficient human-robot collaboration. In this context we present the fast, supervised Proactive Incremental Learning (PIL) framework for learning associations between human hand gestures and the intended robotic manipulation actions. With the proactive aspect, the robot is competent to predict the human's intent and perform an action without waiting for an instruction. The incremental aspect enables the robot to learn associations on the fly while performing a task. It is a probabilistic, statistically-driven approach. As a proof of concept, we focus on a table assembly task where the robot assists its human partner. We investigate how the accuracy of gesture detection affects the number of interactions required to complete the task. We also conducted a human-robot interaction study with non-roboticist users comparing a proactive with a reactive robot that waits for instructions. PMID:29615888

  8. Conciliatory gestures promote forgiveness and reduce anger in humans.

    Science.gov (United States)

    McCullough, Michael E; Pedersen, Eric J; Tabak, Benjamin A; Carter, Evan C

    2014-07-29

    Conflict is an inevitable component of social life, and natural selection has exerted strong effects on many organisms to facilitate victory in conflict and to deter conspecifics from imposing harms upon them. Like many species, humans likely possess cognitive systems whose function is to motivate revenge as a means of deterring individuals who have harmed them from harming them again in the future. However, many social relationships often retain value even after conflicts have occurred between interactants, so natural selection has very likely also endowed humans with cognitive systems whose function is to motivate reconciliation with transgressors whom they perceive as valuable and nonthreatening, notwithstanding their harmful prior actions. In a longitudinal study with 337 participants who had recently been harmed by a relationship partner, we found that conciliatory gestures (e.g., apologies, offers of compensation) were associated with increases in victims' perceptions of their transgressors' relationship value and reductions in perceptions of their transgressors' exploitation risk. In addition, conciliatory gestures appeared to accelerate forgiveness and reduce reactive anger via their intermediate effects on relationship value and exploitation risk. These results strongly suggest that conciliatory gestures facilitate forgiveness and reduce anger by modifying victims' perceptions of their transgressors' value as relationship partners and likelihood of recidivism.

  9. Learning Semantics of Gestural Instructions for Human-Robot Collaboration.

    Science.gov (United States)

    Shukla, Dadhichi; Erkent, Özgür; Piater, Justus

    2018-01-01

    Designed to work safely alongside humans, collaborative robots need to be capable partners in human-robot teams. Besides having key capabilities like detecting gestures, recognizing objects, grasping them, and handing them over, these robots need to seamlessly adapt their behavior for efficient human-robot collaboration. In this context we present the fast, supervised Proactive Incremental Learning (PIL) framework for learning associations between human hand gestures and the intended robotic manipulation actions. With the proactive aspect, the robot is competent to predict the human's intent and perform an action without waiting for an instruction. The incremental aspect enables the robot to learn associations on the fly while performing a task. It is a probabilistic, statistically-driven approach. As a proof of concept, we focus on a table assembly task where the robot assists its human partner. We investigate how the accuracy of gesture detection affects the number of interactions required to complete the task. We also conducted a human-robot interaction study with non-roboticist users comparing a proactive with a reactive robot that waits for instructions.

  10. A biometric authentication model using hand gesture images.

    Science.gov (United States)

    Fong, Simon; Zhuang, Yan; Fister, Iztok; Fister, Iztok

    2013-10-30

    A novel hand biometric authentication method based on measurements of the user's stationary hand gesture of hand sign language is proposed. The measurement of hand gestures could be sequentially acquired by a low-cost video camera. There could possibly be another level of contextual information, associated with these hand signs to be used in biometric authentication. As an analogue, instead of typing a password 'iloveu' in text which is relatively vulnerable over a communication network, a signer can encode a biometric password using a sequence of hand signs, 'i' , 'l' , 'o' , 'v' , 'e' , and 'u'. Subsequently the features from the hand gesture images are extracted which are integrally fuzzy in nature, to be recognized by a classification model for telling if this signer is who he claimed himself to be, by examining over his hand shape and the postures in doing those signs. It is believed that everybody has certain slight but unique behavioral characteristics in sign language, so are the different hand shape compositions. Simple and efficient image processing algorithms are used in hand sign recognition, including intensity profiling, color histogram and dimensionality analysis, coupled with several popular machine learning algorithms. Computer simulation is conducted for investigating the efficacy of this novel biometric authentication model which shows up to 93.75% recognition accuracy.

  11. Gesture-controlled interfaces for self-service machines and other applications

    Science.gov (United States)

    Cohen, Charles J. (Inventor); Beach, Glenn (Inventor); Cavell, Brook (Inventor); Foulk, Gene (Inventor); Jacobus, Charles J. (Inventor); Obermark, Jay (Inventor); Paul, George (Inventor)

    2004-01-01

    A gesture recognition interface for use in controlling self-service machines and other devices is disclosed. A gesture is defined as motions and kinematic poses generated by humans, animals, or machines. Specific body features are tracked, and static and motion gestures are interpreted. Motion gestures are defined as a family of parametrically delimited oscillatory motions, modeled as a linear-in-parameters dynamic system with added geometric constraints to allow for real-time recognition using a small amount of memory and processing time. A linear least squares method is preferably used to determine the parameters which represent each gesture. Feature position measure is used in conjunction with a bank of predictor bins seeded with the gesture parameters, and the system determines which bin best fits the observed motion. Recognizing static pose gestures is preferably performed by localizing the body/object from the rest of the image, describing that object, and identifying that description. The disclosure details methods for gesture recognition, as well as the overall architecture for using gesture recognition to control of devices, including self-service machines.

  12. Better together: Simultaneous presentation of speech and gesture in math instruction supports generalization and retention.

    Science.gov (United States)

    Congdon, Eliza L; Novack, Miriam A; Brooks, Neon; Hemani-Lopez, Naureen; O'Keefe, Lucy; Goldin-Meadow, Susan

    2017-08-01

    When teachers gesture during instruction, children retain and generalize what they are taught (Goldin-Meadow, 2014). But why does gesture have such a powerful effect on learning? Previous research shows that children learn most from a math lesson when teachers present one problem-solving strategy in speech while simultaneously presenting a different, but complementary, strategy in gesture (Singer & Goldin-Meadow, 2005). One possibility is that gesture is powerful in this context because it presents information simultaneously with speech. Alternatively, gesture may be effective simply because it involves the body, in which case the timing of information presented in speech and gesture may be less important for learning. Here we find evidence for the importance of simultaneity: 3 rd grade children retain and generalize what they learn from a math lesson better when given instruction containing simultaneous speech and gesture than when given instruction containing sequential speech and gesture. Interpreting these results in the context of theories of multimodal learning, we find that gesture capitalizes on its synchrony with speech to promote learning that lasts and can be generalized.

  13. Iconic Gestures Facilitate Discourse Comprehension in Individuals With Superior Immediate Memory for Body Configurations.

    Science.gov (United States)

    Wu, Ying Choon; Coulson, Seana

    2015-11-01

    To understand a speaker's gestures, people may draw on kinesthetic working memory (KWM)-a system for temporarily remembering body movements. The present study explored whether sensitivity to gesture meaning was related to differences in KWM capacity. KWM was evaluated through sequences of novel movements that participants viewed and reproduced with their own bodies. Gesture sensitivity was assessed through a priming paradigm. Participants judged whether multimodal utterances containing congruent, incongruent, or no gestures were related to subsequent picture probes depicting the referents of those utterances. Individuals with low KWM were primarily inhibited by incongruent speech-gesture primes, whereas those with high KWM showed facilitation-that is, they were able to identify picture probes more quickly when preceded by congruent speech and gestures than by speech alone. Group differences were most apparent for discourse with weakly congruent speech and gestures. Overall, speech-gesture congruency effects were positively correlated with KWM abilities, which may help listeners match spatial properties of gestures to concepts evoked by speech. © The Author(s) 2015.

  14. Remembering what was said and done: The activation and facilitation of memory for gesture as a consequence of retrieval.

    Science.gov (United States)

    Overoye, Acacia L; Storm, Benjamin C

    2018-04-26

    The gestures that occur alongside speech provide listeners with cues that both improve and alter memory for speech. The present research investigated the interplay of gesture and speech by examining the influence of retrieval on memory for gesture. In three experiments, participants watched video clips of an actor speaking a series of statements with or without gesture before being asked to retrieve the speech portions of half of those statements. Participants were then tested on their ability to recall whether the actor had gestured during each statement and, if so, to recall the nature of the gesture that was produced. Results indicated that attempting to retrieve the speech portion of the statements enhanced participants' ability to remember the gesture portion of the statements. This result was only observed, however, for representational gestures when the speech and gesture components were meaningfully related (Experiments 1 & 2). It was not observed for beat gestures or nonsense gestures (Experiments 2 & 3). These results are consistent with the idea that gestures can be coactivated during the retrieval of speech and that such coactivation is due to the integrated representation of speech and gesture in memory. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

  15. The benefit of gestures during communication: evidence from hearing and hearing-impaired individuals.

    Science.gov (United States)

    Obermeier, Christian; Dolk, Thomas; Gunter, Thomas C

    2012-07-01

    There is no doubt that gestures are communicative and can be integrated online with speech. Little is known, however, about the nature of this process, for example, its automaticity and how our own communicative abilities and also our environment influence the integration of gesture and speech. In two Event Related Potential (ERP) experiments, the effects of gestures during speech comprehension were explored. In both experiments, participants performed a shallow task thereby avoiding explicit gesture-speech integration. In the first experiment, participants with normal hearing viewed videos in which a gesturing actress uttered sentences which were either embedded in multi-speaker babble noise or not. The sentences contained a homonym which was disambiguated by the information in a gesture, which was presented asynchronous to speech (1000 msec earlier). Downstream, the sentence contained a target word that was either related to the dominant or subordinate meaning of the homonym and was used to indicate the success of the disambiguation. Both the homonym and the target word position showed clear ERP evidence of gesture-speech integration and disambiguation only under babble noise. Thus, during noise, gestures were taken into account as an important communicative cue. In Experiment 2, the same asynchronous stimuli were presented to a group of hearing-impaired students and age-matched controls. Only the hearing-impaired individuals showed significant speech-gesture integration and successful disambiguation at the target word. The age-matched controls did not show any effect. Thus, individuals who chronically experience suboptimal communicative situations in daily life automatically take gestures into account. The data from both experiments indicate that gestures are beneficial in countering difficult communication conditions independent of whether the difficulties are due to external (babble noise) or internal (hearing impairment) factors. Copyright © 2011 Elsevier

  16. DEL GESTO ARTÍSTICO A LA PRODUCCIÓN DE ESPACIO PÚBLICO: CREACIÓN Y ACCIÓN EN LA CIUDAD VIVIDA / From the artistic gesture to the production of public space: creation and action in the living city

    Directory of Open Access Journals (Sweden)

    Marta Serra Permanyer

    2010-05-01

    Full Text Available RESUMEN Del gesto artístico a la producción de espacio público: creación y acción en la ciudad vivida explora las formas de intervención en el espacio público que relacionan el arte con la transformación de las ciudades contemporáneas, y que tiene como principal objetivo dar a conocer un nuevo género de prácticas espaciales creativas que, de forma crítica y colaborativa, son generadas desde el campo del arte y destinadas a la construcción social del paisaje urbano. Enmarcándose en la investigación de un proyecto de tesis, se pretende traspasar las fronteras de la disciplina del urbanismo y la arquitectura más ortodoxas para abrirlas a otras miradas y modos de hacer que descubren en la práctica artística en medio urbano su potencial transformador dentro del ámbito político y social. A través del trabajo de tres artistas europeas se deconstruye la idea tradicional de arte publico, un binomio contaminado y en cierto modo decadente. Loraine Leeson, Marjetica Potrč y Lara Almárcegui, entre muchas otras, se presentan aquí como herederas de unos años setenta que dejaron paso a un nuevo género artístico que insinuaba la disolución del arte en la experiencia más cercana y popular. Ejemplos pasados y recientes fijarán las particularidades y condiciones del espacio público contemporáneo desvelando, a su vez, carencias y oportunidades sobre las formas de sociabilizarnos y hacer ciudad, construyendo en su conjunto, una crítica a nuestro entorno cotidiano. SUMMARY From the artistic gesture to the production of public space: creation and action in the living city explores the forms of intervention in the public space that relate art to the transformation of contemporary cities. The articles main objective is that of presenting a new genre of creative space practices which, in a critical and collaborative manner, are generated from the field of art and destined for the social construction of the urban landscape. Framed within

  17. From facial expressions to bodily gestures

    OpenAIRE

    Pichel, Beatriz

    2016-01-01

    Open Access article This article aims to determine to what extent photographic practices in psychology, psychiatry and physiology contributed to the definition of the external bodily signs of passions and emotions in the second half of the 19th century in France. Bridging the gap between recent research in the history of emotions and photographic history, the following analyses focus on the photographic production of scientists and photographers who made significant contributio...

  18. Gesture, Meaning-Making, and Embodiment: Second Language Learning in an Elementary Classroom

    Science.gov (United States)

    Rosborough, Alessandro

    2014-01-01

    The purpose of the present study was to investigate the mediational role of gesture and body movement/positioning between a teacher and an English language learner in a second-grade classroom. Responding to Thibault's (2011) call for understanding language through whole-body sense making, aspects of gesture and body positioning were analyzed for…

  19. Cross-Cultural Transfer in Gesture Frequency in Chinese-English Bilinguals

    Science.gov (United States)

    So, Wing Chee

    2010-01-01

    The purpose of this paper is to examine cross-cultural differences in gesture frequency and the extent to which exposure to two cultures would affect the gesture frequency of bilinguals when speaking in both languages. The Chinese-speaking monolinguals from China, English-speaking monolinguals from America, and Chinese-English bilinguals from…

  20. The Role of Gesture in Supporting Mental Representations: The Case of Mental Abacus Arithmetic

    Science.gov (United States)

    Brooks, Neon B.; Barner, David; Frank, Michael; Goldin-Meadow, Susan

    2018-01-01

    People frequently gesture when problem-solving, particularly on tasks that require spatial transformation. Gesture often facilitates task performance by interacting with internal mental representations, but how this process works is not well understood. We investigated this question by exploring the case of mental abacus (MA), a technique in which…

  1. Effects of gestures on older adults' learning from video-based models

    NARCIS (Netherlands)

    Ouwehand, Kim; van Gog, Tamara|info:eu-repo/dai/nl/294304975; Paas, Fred

    2015-01-01

    This study investigated whether the positive effects of gestures on learning by decreasing working memory load, found in children and young adults, also apply to older adults, who might especially benefit from gestures given memory deficits associated with aging. Participants learned a

  2. A Comparison of Intention and Pantomime Gesture Treatment for Noun Retrieval in People with Aphasia

    Science.gov (United States)

    Ferguson, Neina F.; Evans, Kelli; Raymer, Anastasia M.

    2012-01-01

    Purpose: The effects of intention gesture treatment (IGT) and pantomime gesture treatment (PGT) on word retrieval were compared in people with aphasia. Method: Four individuals with aphasia and word retrieval impairments subsequent to left-hemisphere stroke participated in a single-participant crossover treatment design. Each participant viewed…

  3. Coverbal Gestures in the Recovery from Severe Fluent Aphasia: A Pilot Study

    Science.gov (United States)

    Carlomagno, Sergio; Zulian, Nicola; Razzano, Carmelina; De Mercurio, Ilaria; Marini, Andrea

    2013-01-01

    This post hoc study investigated coverbal gesture patterns in two persons with chronic Wernicke's aphasia. They had both received therapy focusing on multimodal communication therapy, and their pre- and post-therapy verbal and gestural skills in face-to-face conversational interaction with their speech therapist were analysed by administering a…

  4. Gesture and Speech Integration: An Exploratory Study of a Man with Aphasia

    Science.gov (United States)

    Cocks, Naomi; Sautin, Laetitia; Kita, Sotaro; Morgan, Gary; Zlotowitz, Sally

    2009-01-01

    Background: In order to comprehend fully a speaker's intention in everyday communication, information is integrated from multiple sources, including gesture and speech. There are no published studies that have explored the impact of aphasia on iconic co-speech gesture and speech integration. Aims: To explore the impact of aphasia on co-speech…

  5. Exploring the Relationship between Gestural Recognition and Imitation: Evidence of Dyspraxia in Autism Spectrum Disorders

    Science.gov (United States)

    Ham, Heidi Stieglitz; Bartolo, Angela; Corley, Martin; Rajendran, Gnanathusharan; Szabo, Aniko; Swanson, Sara

    2011-01-01

    In this study, the relationship between gesture recognition and imitation was explored. Nineteen individuals with Autism Spectrum Disorder (ASD) were compared to a control group of 23 typically developing children on their ability to imitate and recognize three gesture types (transitive, intransitive, and pantomimes). The ASD group performed more…

  6. Maternal Gesture Use and Language Development in Infant Siblings of Children with Autism Spectrum Disorder

    Science.gov (United States)

    Talbott, Meagan R.; Nelson, Charles A.; Tager-Flusberg, Helen

    2015-01-01

    Impairments in language and communication are an early-appearing feature of autism spectrum disorders (ASD), with delays in language and gesture evident as early as the first year of life. Research with typically developing populations highlights the importance of both infant and maternal gesture use in infants' early language development.…

  7. RisQ: Recognizing Smoking Gestures with Inertial Sensors on a Wristband

    Science.gov (United States)

    Parate, Abhinav; Chiu, Meng-Chieh; Chadowitz, Chaniel; Ganesan, Deepak; Kalogerakis, Evangelos

    2015-01-01

    Smoking-induced diseases are known to be the leading cause of death in the United States. In this work, we design RisQ, a mobile solution that leverages a wristband containing a 9-axis inertial measurement unit to capture changes in the orientation of a person's arm, and a machine learning pipeline that processes this data to accurately detect smoking gestures and sessions in real-time. Our key innovations are fourfold: a) an arm trajectory-based method that extracts candidate hand-to-mouth gestures, b) a set of trajectory-based features to distinguish smoking gestures from confounding gestures including eating and drinking, c) a probabilistic model that analyzes sequences of hand-to-mouth gestures and infers which gestures are part of individual smoking sessions, and d) a method that leverages multiple IMUs placed on a person's body together with 3D animation of a person's arm to reduce burden of self-reports for labeled data collection. Our experiments show that our gesture recognition algorithm can detect smoking gestures with high accuracy (95.7%), precision (91%) and recall (81%). We also report a user study that demonstrates that we can accurately detect the number of smoking sessions with very few false positives over the period of a day, and that we can reliably extract the beginning and end of smoking session periods. PMID:26688835

  8. A neuropsychological approach to the study of gesture and pantomime in aphasa

    Directory of Open Access Journals (Sweden)

    Jocelyn Kadish

    1978-11-01

    Full Text Available The impairment of  gesture and pantomime in aphasia was examined from  a neuropsychological perspective. The Boston Diagnostic Test of  Aphasia, Luria's Neuro-psychological Investigation, Pickett's Tests for  gesture and pantomime and the Performance Scale of  the Wechsler Adult Intelligence Scale were administered to six aphasic subjects with varying etiology and severity. Results indicated that severity of  aphasia was positively related to severity of  gestural disturbance; gestural ability was associated with verbal and non-linguistic aspects of  ability, within receptive and expressive levels respectively; performance  on gestural tasks was superior to that on verbal tasks irrespective of  severity of aphasia; damage to Luria's second and third functional  brain units were positively related to deficits  in receptive and expressive gesture respectively; no relationship was found  between seventy of  general intellectual impairment and gestural deficit.  It was concluded that the gestural impairment may best be understood as a breakdown in complex sequential manual motor activity. Theoretical and therapeutic implications were discussed.

  9. The Role of Gestures in a Teacher-Student-Discourse about Atoms

    Science.gov (United States)

    Abels, Simone

    2016-01-01

    Recent educational research emphasises the importance of analysing talk and gestures to come to an understanding about students' conceptual learning. Gestures are perceived as complex hand movements being equivalent to other language modes. They can convey experienceable as well as abstract concepts. As well as technical language, gestures…

  10. Prototyping with your hands: the many roles of gesture in the communication of design concepts

    DEFF Research Database (Denmark)

    Cash, Philip; Maier, Anja

    2016-01-01

    There is an on-going focus exploring the use of gesture in design situations; however, there are still significant questions as to how this is related to the understanding and communication of design concepts. This work explores the use of gesture through observing and video-coding four teams of ...

  11. Balancing Direction and Independence in Second Language Vocabulary Learning: A Gesture Pilot Study

    Science.gov (United States)

    Mathison, Lake

    2017-01-01

    This pilot study looks at the effect of learning second language vocabulary with gesture. Specifically, this current study asks whether researcher-instructed or student-constructed gestures are more effective. Depth of processing theories (Craik and Lockhart 1972) as well as more recent educational frameworks like ICAP ("Interactive,"…

  12. A Functional Analysis of Gestural Behaviors Emitted by Young Children with Severe Developmental Disabilities

    Science.gov (United States)

    Ferreri, Summer J.; Plavnick, Joshua B.

    2011-01-01

    Many children with severe developmental disabilities emit idiosyncratic gestures that may function as verbal operants (Sigafoos et al., 2000). This study examined the effectiveness of a functional analysis methodology to identify the variables responsible for gestures emitted by 2 young children with severe developmental disabilities. Potential…

  13. Hospitable Gestures in the University Lecture: Analysing Derrida's Pedagogy

    Science.gov (United States)

    Ruitenberg, Claudia

    2014-01-01

    Based on archival research, this article analyses the pedagogical gestures in Derrida's (largely unpublished) lectures on hospitality (1995/96), with particular attention to the enactment of hospitality in these gestures. The motivation for this analysis is twofold. First, since the large-group university lecture has been widely critiqued as…

  14. Gesture as Input in Language Acquisition: Learning "Who She Is" from "Where She Is"

    Science.gov (United States)

    Goodrich, Whitney Sarah-Iverson

    2009-01-01

    This dissertation explores the role co-speech gesture plays as input in language learning, specifically with respect to the acquisition of anaphoric pronouns. Four studies investigate how both adults and children interpret ambiguous pronouns, and how the order-of-mention tendency develops in children. The results suggest that gesture is a useful…

  15. Long-Term Effects of Gestures on Memory for Foreign Language Words Trained in the Classroom

    Science.gov (United States)

    Macedonia, Manuela; Klimesch, Wolfgang

    2014-01-01

    Language and gesture are viewed as highly interdependent systems. Besides supporting communication, gestures also have an impact on memory for verbal information compared to pure verbal encoding in native but also in foreign language learning. This article presents a within-subject longitudinal study lasting 14 months that tested the use of…

  16. Bridging Gaps in Common Ground: Speakers Design Their Gestures for Their Listeners

    Science.gov (United States)

    Hilliard, Caitlin; Cook, Susan Wagner

    2016-01-01

    Communication is shaped both by what we are trying to say and by whom we are saying it to. We examined whether and how shared information influences the gestures speakers produce along with their speech. Unlike prior work examining effects of common ground on speech and gesture, we examined a situation in which some speakers have the same amount…

  17. Effects of observing and producing deictic gestures on memory and learning in different age groups

    NARCIS (Netherlands)

    K.H.R. Ouwehand (Kim)

    2016-01-01

    markdownabstractThe studies presented in this dissertation aimed to investigate whether observing or producing deictic gestures (i.e., pointing and tracing gestures to index a referent in space or a movement pathway), could facilitate memory and learning in children, young adults, and older adults.

  18. Gesture as a Resource for Intersubjectivity in Second-Language Learning Situations

    Science.gov (United States)

    Belhiah, Hassan

    2013-01-01

    This study documents the role of hand gestures in achieving mutual understanding in second-language learning situations. The study tracks the way gesture is coordinated with talk in tutorials between two Korean students and their American teachers. The study adopts an interactional approach to the study of participants' talk and gestural…

  19. Technological evaluation of gesture and speech interfaces for enabling dismounted soldier-robot dialogue

    Science.gov (United States)

    Kattoju, Ravi Kiran; Barber, Daniel J.; Abich, Julian; Harris, Jonathan

    2016-05-01

    With increasing necessity for intuitive Soldier-robot communication in military operations and advancements in interactive technologies, autonomous robots have transitioned from assistance tools to functional and operational teammates able to service an array of military operations. Despite improvements in gesture and speech recognition technologies, their effectiveness in supporting Soldier-robot communication is still uncertain. The purpose of the present study was to evaluate the performance of gesture and speech interface technologies to facilitate Soldier-robot communication during a spatial-navigation task with an autonomous robot. Gesture and speech semantically based spatial-navigation commands leveraged existing lexicons for visual and verbal communication from the U.S Army field manual for visual signaling and a previously established Squad Level Vocabulary (SLV). Speech commands were recorded by a Lapel microphone and Microsoft Kinect, and classified by commercial off-the-shelf automatic speech recognition (ASR) software. Visual signals were captured and classified using a custom wireless gesture glove and software. Participants in the experiment commanded a robot to complete a simulated ISR mission in a scaled down urban scenario by delivering a sequence of gesture and speech commands, both individually and simultaneously, to the robot. Performance and reliability of gesture and speech hardware interfaces and recognition tools were analyzed and reported. Analysis of experimental results demonstrated the employed gesture technology has significant potential for enabling bidirectional Soldier-robot team dialogue based on the high classification accuracy and minimal training required to perform gesture commands.

  20. Differences in the Ability of Apes and Children to Instruct Others Using Gestures

    Science.gov (United States)

    Grosse, Katja; Call, Josep; Carpenter, Malinda; Tomasello, Michael

    2015-01-01

    In all human cultures, people gesture iconically. However, the evolutionary basis of iconic gestures is unknown. In this study, chimpanzees and bonobos, and 2- and 3-year-old children, learned how to operate two apparatuses to get rewards. Then, at test, only a human adult had access to the apparatuses, and participants could instruct her about…

  1. A gesture-controlled projection display for CT-guided interventions.

    Science.gov (United States)

    Mewes, A; Saalfeld, P; Riabikin, O; Skalej, M; Hansen, C

    2016-01-01

    The interaction with interventional imaging systems within a sterile environment is a challenging task for physicians. Direct physician-machine interaction during an intervention is rather limited because of sterility and workspace restrictions. We present a gesture-controlled projection display that enables a direct and natural physician-machine interaction during computed tomography (CT)-based interventions. Therefore, a graphical user interface is projected on a radiation shield located in front of the physician. Hand gestures in front of this display are captured and classified using a leap motion controller. We propose a gesture set to control basic functions of intervention software such as gestures for 2D image exploration, 3D object manipulation and selection. Our methods were evaluated in a clinically oriented user study with 12 participants. The results of the performed user study confirm that the display and the underlying interaction concept are accepted by clinical users. The recognition of the gestures is robust, although there is potential for improvements. The gesture training times are less than 10 min, but vary heavily between the participants of the study. The developed gestures are connected logically to the intervention software and intuitive to use. The proposed gesture-controlled projection display counters current thinking, namely it gives the radiologist complete control of the intervention software. It opens new possibilities for direct physician-machine interaction during CT-based interventions and is well suited to become an integral part of future interventional suites.

  2. Integrating gesture recognition in airplane seats for in-flight entertainment

    NARCIS (Netherlands)

    van de Westelaken, H.F.M.; Hu, J.; Liu, H.; Rauterberg, G.W.M.; Pan, Z.; Zhang, X.; El Rhalibi, A.; Woo, W.; Li, Y.

    2008-01-01

    In order to reduce both the psychological and physical stress in air travel, sensors are integrated in airplane seats to detect the gestures as input for in-flight entertainment systems. The content provided by the entertainment systems helps to reduce the psychological stress, and the gesture

  3. Embedding gesture recognition into airplane seats for in-flight entertainment

    NARCIS (Netherlands)

    van de Westelaken, H.F.M.; Hu, J.; Liu, H.; Rauterberg, G.W.M.

    2011-01-01

    In order to reduce both psychological and physical stress in air travel, sensors are integrated into airplane seats to detect gestures as input for in-flight entertainment systems. The content provided by the entertainment systems helps to reduce psychological stress, and gesture recognition is used

  4. Symbiotic Gesture and the Sociocognitive Visibility of Grammar in Second Language Acquisition

    Science.gov (United States)

    Churchill, Eton; Okada, Hanako; Nishino, Takako; Atkinson, Dwight

    2010-01-01

    This article argues for the embodied and environmentally embedded nature of second language acquisition (SLA). Through fine-grained analysis of interaction using Goodwin's (2003a) concept of "symbiotic gesture"--gesture coupled with its rich environmental context to produce complex social action--we illustrate how a tutor, learner, and grammar…

  5. Research on gesture recognition of augmented reality maintenance guiding system based on improved SVM

    Science.gov (United States)

    Zhao, Shouwei; Zhang, Yong; Zhou, Bin; Ma, Dongxi

    2014-09-01

    Interaction is one of the key techniques of augmented reality (AR) maintenance guiding system. Because of the complexity of the maintenance guiding system's image background and the high dimensionality of gesture characteristics, the whole process of gesture recognition can be divided into three stages which are gesture segmentation, gesture characteristic feature modeling and trick recognition. In segmentation stage, for solving the misrecognition of skin-like region, a segmentation algorithm combing background mode and skin color to preclude some skin-like regions is adopted. In gesture characteristic feature modeling of image attributes stage, plenty of characteristic features are analyzed and acquired, such as structure characteristics, Hu invariant moments features and Fourier descriptor. In trick recognition stage, a classifier based on Support Vector Machine (SVM) is introduced into the augmented reality maintenance guiding process. SVM is a novel learning method based on statistical learning theory, processing academic foundation and excellent learning ability, having a lot of issues in machine learning area and special advantages in dealing with small samples, non-linear pattern recognition at high dimension. The gesture recognition of augmented reality maintenance guiding system is realized by SVM after the granulation of all the characteristic features. The experimental results of the simulation of number gesture recognition and its application in augmented reality maintenance guiding system show that the real-time performance and robustness of gesture recognition of AR maintenance guiding system can be greatly enhanced by improved SVM.

  6. The Organization of Words and Symbolic Gestures in 18-Month-Olds' Lexicons: Evidence from a Disambiguation Task

    Science.gov (United States)

    Suanda, Sumarga H.; Namy, Laura L.

    2013-01-01

    Infants' early communicative repertoires include both words and symbolic gestures. The current study examined the extent to which infants organize words and gestures in a single unified lexicon. As a window into lexical organization, eighteen-month-olds' ("N" = 32) avoidance of word-gesture overlap was examined and compared with…

  7. Age-Related Changes in Preschoolers' Ability to Communicate Using Iconic Gestures in the Absence of Speech

    Science.gov (United States)

    Vasc, Dermina; Miclea, Mircea

    2018-01-01

    Iconic gestures illustrate complex meanings and clarify and enrich the speech they accompany. Little is known, however, about how children use iconic gestures in the absence of speech. In this study, we used a cross-sectional design to investigate how 3-, 4- and 5-year-old children (N = 51) communicate using pantomime iconic gestures. Children…

  8. Hearing and seeing meaning in noise. Alpha, beta and gamma oscillations predict gestural enhancement of degraded speech comprehension

    NARCIS (Netherlands)

    Drijvers, L.; Özyürek, A.; Jensen, O.

    2018-01-01

    During face-to-face communication, listeners integrate speech with gestures. The semantic information conveyed by iconic gestures (e.g., a drinking gesture) can aid speech comprehension in adverse listening conditions. In this magnetoencephalography (MEG) study, we investigated the spatiotemporal

  9. Observation of Depictive Versus Tracing Gestures Selectively Aids Verbal Versus Visual-Spatial Learning in Primary School Children

    NARCIS (Netherlands)

    van Wermeskerken, Margot; Fijan, Nathalie; Eielts, Charly; Pouw, Wim T. J. L.

    2016-01-01

    Previous research has established that gesture observation aids learning in children. The current study examined whether observation of gestures (i.e. depictive and tracing gestures) differentially affected verbal and visual-spatial retention when learning a route and its street names. Specifically,

  10. Gesturing during mental problem solving reduces eye movements, especially for individuals with lower visual working memory capacity

    NARCIS (Netherlands)

    W.T.J.L. Pouw (Wim); M.-F. Mavilidi (Myrto-Foteini); T.A.J.M. van Gog (Tamara); G.W.C. Paas (Fred)

    2016-01-01

    textabstractNon-communicative hand gestures have been found to benefit problem-solving performance. These gestures seem to compensate for limited internal cognitive capacities, such as visual working memory capacity. Yet, it is not clear how gestures might perform this cognitive function. One

  11. Gesturing during mental problem solving reduces eye movements, especially for individuals with lower visual working memory capacity

    NARCIS (Netherlands)

    Pouw, Wim T J L; Mavilidi, Myrto Foteini; van Gog, Tamara; Paas, Fred

    2016-01-01

    Non-communicative hand gestures have been found to benefit problem-solving performance. These gestures seem to compensate for limited internal cognitive capacities, such as visual working memory capacity. Yet, it is not clear how gestures might perform this cognitive function. One hypothesis is that

  12. A word in the hand: action, gesture and mental representation in humans and non-human primates

    Science.gov (United States)

    Cartmill, Erica A.; Beilock, Sian; Goldin-Meadow, Susan

    2012-01-01

    The movements we make with our hands both reflect our mental processes and help to shape them. Our actions and gestures can affect our mental representations of actions and objects. In this paper, we explore the relationship between action, gesture and thought in both humans and non-human primates and discuss its role in the evolution of language. Human gesture (specifically representational gesture) may provide a unique link between action and mental representation. It is kinaesthetically close to action and is, at the same time, symbolic. Non-human primates use gesture frequently to communicate, and do so flexibly. However, their gestures mainly resemble incomplete actions and lack the representational elements that characterize much of human gesture. Differences in the mirror neuron system provide a potential explanation for non-human primates' lack of representational gestures; the monkey mirror system does not respond to representational gestures, while the human system does. In humans, gesture grounds mental representation in action, but there is no evidence for this link in other primates. We argue that gesture played an important role in the transition to symbolic thought and language in human evolution, following a cognitive leap that allowed gesture to incorporate representational elements. PMID:22106432

  13. Gesture and Body-Movement as Teaching and Learning Tools in the Classical Voice Lesson: A Survey into Current Practice

    Science.gov (United States)

    Nafisi, Julia

    2013-01-01

    This article discusses the use of gesture and body-movement in the teaching of singing and reports on a survey amongst professional singing teachers in Germany regarding their use of gesture and body movement as pedagogic tools in their teaching. The nomenclature of gestures and movements used in the survey is based on a previous study by the…

  14. Person and gesture tracking with smart stereo cameras

    Science.gov (United States)

    Gordon, Gaile; Chen, Xiangrong; Buck, Ron

    2008-02-01

    Physical security increasingly involves sophisticated, real-time visual tracking of a person's location inside a given environment, often in conjunction with biometrics and other security-related technologies. However, demanding real-world conditions like crowded rooms, changes in lighting and physical obstructions have proved incredibly challenging for 2D computer vision technology. In contrast, 3D imaging technology is not affected by constant changes in lighting and apparent color, and thus allows tracking accuracy to be maintained in dynamically lit environments. In addition, person tracking with a 3D stereo camera can provide the location and movement of each individual very precisely, even in a very crowded environment. 3D vision only requires that the subject be partially visible to a single stereo camera to be correctly tracked; multiple cameras are used to extend the system's operational footprint, and to contend with heavy occlusion. A successful person tracking system, must not only perform visual analysis robustly, but also be small, cheap and consume relatively little power. The TYZX Embedded 3D Vision systems are perfectly suited to provide the low power, small footprint, and low cost points required by these types of volume applications. Several security-focused organizations, including the U.S Government, have deployed TYZX 3D stereo vision systems in security applications. 3D image data is also advantageous in the related application area of gesture tracking. Visual (uninstrumented) tracking of natural hand gestures and movement provides new opportunities for interactive control including: video gaming, location based entertainment, and interactive displays. 2D images have been used to extract the location of hands within a plane, but 3D hand location enables a much broader range of interactive applications. In this paper, we provide some background on the TYZX smart stereo cameras platform, describe the person tracking and gesture tracking systems

  15. Differentiation of energy concepts through speech and gesture in interaction

    Science.gov (United States)

    Close, Hunter G.; Scherr, Rachel E.

    2012-02-01

    Through microanalysis of speech and gesture in one interaction between learners (in a course on energy for in-service teachers), we observe coherent states of conceptual differentiation of different learners. We observe that the interaction among learners across different states of differentiation is not in itself sufficient to accomplish differentiation; however, the real-time receptivity of the learners to conceptually relevant details in each other's actions suggests that future instruction that focuses explicitly on such actions and their meaning in context may assist differentiation.

  16. The Perception of Sound Movements as Expressive Gestures

    DEFF Research Database (Denmark)

    Götzen, Amalia De; Sikström, Erik; Korsgaard, Dannie

    2014-01-01

    This paper is a preliminary attempt to investigate the perception of sound movements as expressive gestures. The idea is that if sound movement is used as a musical parameter, a listener (or a subject) should be able to distinguish among dierent movements and she/he should be able to group them a...... by drawing it on a tablet. Preliminary results show that subjects could consistently group the stimuli, and that they primarily used paths and legato{staccato patterns to discriminate among dierent sound movements/expressive intention....

  17. NUI framework based on real-time head pose estimation and hand gesture recognition

    Directory of Open Access Journals (Sweden)

    Kim Hyunduk

    2016-01-01

    Full Text Available The natural user interface (NUI is used for the natural motion interface without using device or tool such as mice, keyboards, pens and markers. In this paper, we develop natural user interface framework based on two recognition module. First module is real-time head pose estimation module using random forests and second module is hand gesture recognition module, named Hand gesture Key Emulation Toolkit (HandGKET. Using the head pose estimation module, we can know where the user is looking and what the user’s focus of attention is. Moreover, using the hand gesture recognition module, we can also control the computer using the user’s hand gesture without mouse and keyboard. In proposed framework, the user’s head direction and hand gesture are mapped into mouse and keyboard event, respectively.

  18. The role of gestures in achieving understanding in Early English teaching in Denmark

    DEFF Research Database (Denmark)

    aus der Wieschen, Maria Vanessa; Eskildsen, Søren Wind

    school in Denmark. The use of multimodal resources employed by teachers in foreign language classrooms has been studied by e.g. Muramuto (1999), Lazaraton (2004), Taleghani-Nikazm (2008), Eskildsen & Wagner (2013), Sert (2015). This research has established gestures as a pervasive phenomenon in language...... brings this established agreement on the importance of gestures in classroom interaction to bear on early foreign language learning: Whereas prior work on gestures in L2 classrooms has predominantly dealt with adult L2 learners, this paper investigates the extent to which a teacher makes use of gestures...... in early child foreign language teaching. Using multimodal conversation analysis of three hours of classroom instruction in a Danish primary school, we uncover how a teacher uses gestures to enhance the comprehension of his L2 talk when teaching English in the 1st and 3rd grade, both of which are beginning...

  19. Magic Ring: A Finger-Worn Device for Multiple Appliances Control Using Static Finger Gestures

    Directory of Open Access Journals (Sweden)

    Tongjun Huang

    2012-05-01

    Full Text Available An ultimate goal for Ubiquitous Computing is to enable people to interact with the surrounding electrical devices using their habitual body gestures as they communicate with each other. The feasibility of such an idea is demonstrated through a wearable gestural device named Magic Ring (MR, which is an original compact wireless sensing mote in a ring shape that can recognize various finger gestures. A scenario of wireless multiple appliances control is selected as a case study to evaluate the usability of such a gestural interface. Experiments comparing the MR and a Remote Controller (RC were performed to evaluate the usability. From the results, only with 10 minutes practice, the proposed paradigm of gestural-based control can achieve a performance of completing about six tasks per minute, which is in the same level of the RC-based method.

  20. A Component-Based Vocabulary-Extensible Sign Language Gesture Recognition Framework.

    Science.gov (United States)

    Wei, Shengjing; Chen, Xiang; Yang, Xidong; Cao, Shuai; Zhang, Xu

    2016-04-19

    Sign language recognition (SLR) can provide a helpful tool for the communication between the deaf and the external world. This paper proposed a component-based vocabulary extensible SLR framework using data from surface electromyographic (sEMG) sensors, accelerometers (ACC), and gyroscopes (GYRO). In this framework, a sign word was considered to be a combination of five common sign components, including hand shape, axis, orientation, rotation, and trajectory, and sign classification was implemented based on the recognition of five components. Especially, the proposed SLR framework consisted of two major parts. The first part was to obtain the component-based form of sign gestures and establish the code table of target sign gesture set using data from a reference subject. In the second part, which was designed for new users, component classifiers were trained using a training set suggested by the reference subject and the classification of unknown gestures was performed with a code matching method. Five subjects participated in this study and recognition experiments under different size of training sets were implemented on a target gesture set consisting of 110 frequently-used Chinese Sign Language (CSL) sign words. The experimental results demonstrated that the proposed framework can realize large-scale gesture set recognition with a small-scale training set. With the smallest training sets (containing about one-third gestures of the target gesture set) suggested by two reference subjects, (82.6 ± 13.2)% and (79.7 ± 13.4)% average recognition accuracy were obtained for 110 words respectively, and the average recognition accuracy climbed up to (88 ± 13.7)% and (86.3 ± 13.7)% when the training set included 50~60 gestures (about half of the target gesture set). The proposed framework can significantly reduce the user's training burden in large-scale gesture recognition, which will facilitate the implementation of a practical SLR system.

  1. A Component-Based Vocabulary-Extensible Sign Language Gesture Recognition Framework

    Directory of Open Access Journals (Sweden)

    Shengjing Wei

    2016-04-01

    Full Text Available Sign language recognition (SLR can provide a helpful tool for the communication between the deaf and the external world. This paper proposed a component-based vocabulary extensible SLR framework using data from surface electromyographic (sEMG sensors, accelerometers (ACC, and gyroscopes (GYRO. In this framework, a sign word was considered to be a combination of five common sign components, including hand shape, axis, orientation, rotation, and trajectory, and sign classification was implemented based on the recognition of five components. Especially, the proposed SLR framework consisted of two major parts. The first part was to obtain the component-based form of sign gestures and establish the code table of target sign gesture set using data from a reference subject. In the second part, which was designed for new users, component classifiers were trained using a training set suggested by the reference subject and the classification of unknown gestures was performed with a code matching method. Five subjects participated in this study and recognition experiments under different size of training sets were implemented on a target gesture set consisting of 110 frequently-used Chinese Sign Language (CSL sign words. The experimental results demonstrated that the proposed framework can realize large-scale gesture set recognition with a small-scale training set. With the smallest training sets (containing about one-third gestures of the target gesture set suggested by two reference subjects, (82.6 ± 13.2% and (79.7 ± 13.4% average recognition accuracy were obtained for 110 words respectively, and the average recognition accuracy climbed up to (88 ± 13.7% and (86.3 ± 13.7% when the training set included 50~60 gestures (about half of the target gesture set. The proposed framework can significantly reduce the user’s training burden in large-scale gesture recognition, which will facilitate the implementation of a practical SLR system.

  2. Touch and You’re Trapp(cked: Quantifying the Uniqueness of Touch Gestures for Tracking

    Directory of Open Access Journals (Sweden)

    Masood Rahat

    2018-04-01

    Full Text Available We argue that touch-based gestures on touch-screen devices enable the threat of a form of persistent and ubiquitous tracking which we call touch-based tracking. Touch-based tracking goes beyond the tracking of virtual identities and has the potential for cross-device tracking as well as identifying multiple users using the same device. We demonstrate the likelihood of touch-based tracking by focusing on touch gestures widely used to interact with touch devices such as swipes and taps.. Our objective is to quantify and measure the information carried by touch-based gestures which may lead to tracking users. For this purpose, we develop an information theoretic method that measures the amount of information about users leaked by gestures when modelled as feature vectors. Our methodology allows us to evaluate the information leaked by individual features of gestures, samples of gestures, as well as samples of combinations of gestures. Through our purpose-built app, called TouchTrack, we gather gesture samples from 89 users, and demonstrate that touch gestures contain sufficient information to uniquely identify and track users. Our results show that writing samples (on a touch pad can reveal 73.7% of information (when measured in bits, and left swipes can reveal up to 68.6% of information. Combining different combinations of gestures results in higher uniqueness, with the combination of keystrokes, swipes and writing revealing up to 98.5% of information about users. We further show that, through our methodology, we can correctly re-identify returning users with a success rate of more than 90%.

  3. Relating Gestures and Speech: An analysis of students' conceptions about geological sedimentary processes

    Science.gov (United States)

    Herrera, Juan Sebastian; Riggs, Eric M.

    2013-08-01

    Advances in cognitive science and educational research indicate that a significant part of spatial cognition is facilitated by gesture (e.g. giving directions, or describing objects or landscape features). We aligned the analysis of gestures with conceptual metaphor theory to probe the use of mental image schemas as a source of concept representations for students' learning of sedimentary processes. A hermeneutical approach enabled us to access student meaning-making from students' verbal reports and gestures about four core geological ideas that involve sea-level change and sediment deposition. The study included 25 students from three US universities. Participants were enrolled in upper-level undergraduate courses on sedimentology and stratigraphy. We used semi-structured interviews for data collection. Our gesture coding focused on three types of gestures: deictic, iconic, and metaphoric. From analysis of video recorded interviews, we interpreted image schemas in gestures and verbal reports. Results suggested that students attempted to make more iconic and metaphoric gestures when dealing with abstract concepts, such as relative sea level, base level, and unconformities. Based on the analysis of gestures that recreated certain patterns including time, strata, and sea-level fluctuations, we reasoned that proper representational gestures may indicate completeness in conceptual understanding. We concluded that students rely on image schemas to develop ideas about complex sedimentary systems. Our research also supports the hypothesis that gestures provide an independent and non-linguistic indicator of image schemas that shape conceptual development, and also play a role in the construction and communication of complex spatial and temporal concepts in the geosciences.

  4. DNA Encoding Training Using 3D Gesture Interaction.

    Science.gov (United States)

    Nicola, Stelian; Handrea, Flavia-Laura; Crişan-Vida, Mihaela; Stoicu-Tivadar, Lăcrămioara

    2017-01-01

    The work described in this paper summarizes the development process and presents the results of a human genetics training application, studying the 20 amino acids formed by the combination of the 3 nucleotides of DNA targeting mainly medical and bioinformatics students. Currently, the domain applications using recognized human gestures of the Leap Motion sensor are used in molecules controlling and learning from Mendeleev table or in visualizing the animated reactions of specific molecules with water. The novelty in the current application consists in using the Leap Motion sensor creating new gestures for the application control and creating a tag based algorithm corresponding to each amino acid, depending on the position in the 3D virtual space of the 4 nucleotides of DNA and their type. The team proposes a 3D application based on Unity editor and on Leap Motion sensor where the user has the liberty of forming different combinations of the 20 amino acids. The results confirm that this new type of study of medicine/biochemistry using the Leap Motion sensor for handling amino acids is suitable for students. The application is original and interactive and the users can create their own amino acid structures in a 3D-like environment which they could not do otherwise using traditional pen-and-paper.

  5. Modelling gesture use and early language development in autism spectrum disorder.

    Science.gov (United States)

    Manwaring, Stacy S; Mead, Danielle L; Swineford, Lauren; Thurm, Audrey

    2017-09-01

    Nonverbal communication abilities, including gesture use, are impaired in autism spectrum disorder (ASD). However, little is known about how common gestures may influence or be influenced by other areas of development. To examine the relationships between gesture, fine motor and language in young children with ASD compared with a comparison group using multiple measures and methods in a structural equation modelling framework. Participants included 110 children with ASD and a non-ASD comparison group of 87 children (that included children with developmental delays (DD) or typical development (TD)), from 12 to 48 months of age. A construct of gesture use as measured by the Communication and Symbolic Behavior Scales-Developmental Profile Caregiver Questionnaire (CQ) and the Autism Diagnostic Observation Schedule (ADOS), as well as fine motor from the Mullen Scales of Early Learning and Vineland Adaptive Behavior Scales-II (VABS-II) was examined using second-order confirmatory factor analysis (CFA). A series of structural equation models then examined concurrent relationships between the aforementioned latent gesture construct and expressive and receptive language. A series of hierarchical regression analyses was run in a subsample of 36 children with ASD with longitudinal data to determine how gesture factor scores predicted later language outcomes. Across study groups, the gesture CFA model with indicators of gesture use from both the CQ (parent-reported) and ADOS (direct observation), and measures of fine motor provided good fit with all indicators significantly and strongly loading onto one gesture factor. This model of gesture use, controlling for age, was found to correlate strongly with concurrent expressive and receptive language. The correlations between gestures and concurrent language were similar in magnitude in both the ASD and non-ASD groups. In the longitudinal subsample of children with ASD, gestures at time 1 predicted later receptive (but not

  6. Gesturing during mental problem solving reduces eye movements, especially for individuals with lower visual working memory capacity.

    Science.gov (United States)

    Pouw, Wim T J L; Mavilidi, Myrto-Foteini; van Gog, Tamara; Paas, Fred

    2016-08-01

    Non-communicative hand gestures have been found to benefit problem-solving performance. These gestures seem to compensate for limited internal cognitive capacities, such as visual working memory capacity. Yet, it is not clear how gestures might perform this cognitive function. One hypothesis is that gesturing is a means to spatially index mental simulations, thereby reducing the need for visually projecting the mental simulation onto the visual presentation of the task. If that hypothesis is correct, less eye movements should be made when participants gesture during problem solving than when they do not gesture. We therefore used mobile eye tracking to investigate the effect of co-thought gesturing and visual working memory capacity on eye movements during mental solving of the Tower of Hanoi problem. Results revealed that gesturing indeed reduced the number of eye movements (lower saccade counts), especially for participants with a relatively lower visual working memory capacity. Subsequent problem-solving performance was not affected by having (not) gestured during the mental solving phase. The current findings suggest that our understanding of gestures in problem solving could be improved by taking into account eye movements during gesturing.

  7. "Slight" of hand: the processing of visually degraded gestures with speech.

    Science.gov (United States)

    Kelly, Spencer D; Hansen, Bruce C; Clark, David T

    2012-01-01

    Co-speech hand gestures influence language comprehension. The present experiment explored what part of the visual processing system is optimized for processing these gestures. Participants viewed short video clips of speech and gestures (e.g., a person saying "chop" or "twist" while making a chopping gesture) and had to determine whether the two modalities were congruent or incongruent. Gesture videos were designed to stimulate the parvocellular or magnocellular visual pathways by filtering out low or high spatial frequencies (HSF versus LSF) at two levels of degradation severity (moderate and severe). Participants were less accurate and slower at processing gesture and speech at severe versus moderate levels of degradation. In addition, they were slower for LSF versus HSF stimuli, and this difference was most pronounced in the severely degraded condition. However, exploratory item analyses showed that the HSF advantage was modulated by the range of motion and amount of motion energy in each video. The results suggest that hand gestures exploit a wide range of spatial frequencies, and depending on what frequencies carry the most motion energy, parvocellular or magnocellular visual pathways are maximized to quickly and optimally extract meaning.

  8. The Different Patterns of Gesture between Genders in Mathematical Problem Solving of Geometry

    Science.gov (United States)

    Harisman, Y.; Noto, M. S.; Bakar, M. T.; Amam, A.

    2017-02-01

    This article discusses about students’ gesture between genders in answering problems of geometry. Gesture aims to check students’ understanding which is undefined from their writings. This study is a qualitative research, there were seven questions given to two students of eight grade Junior High School who had the equal ability. The data of this study were collected from mathematical problem solving test, videoing students’ presentation, and interviewing students by asking questions to check their understandings in geometry problems, in this case the researchers would observe the students’ gesture. The result of this study revealed that there were patterns of gesture through students’ conversation and prosodic cues, such as tones, intonation, speech rate and pause. Female students tended to give indecisive gestures, for instance bowing, hesitating, embarrassing, nodding many times in shifting cognitive comprehension, forwarding their body and asking questions to the interviewer when they found tough questions. However, male students acted some gestures such as playing their fingers, focusing on questions, taking longer time to answer hard questions, staying calm in shifting cognitive comprehension. We suggest to observe more sample and focus on students’ gesture consistency in showing their understanding to solve the given problems.

  9. Appearance-based human gesture recognition using multimodal features for human computer interaction

    Science.gov (United States)

    Luo, Dan; Gao, Hua; Ekenel, Hazim Kemal; Ohya, Jun

    2011-03-01

    The use of gesture as a natural interface plays an utmost important role for achieving intelligent Human Computer Interaction (HCI). Human gestures include different components of visual actions such as motion of hands, facial expression, and torso, to convey meaning. So far, in the field of gesture recognition, most previous works have focused on the manual component of gestures. In this paper, we present an appearance-based multimodal gesture recognition framework, which combines the different groups of features such as facial expression features and hand motion features which are extracted from image frames captured by a single web camera. We refer 12 classes of human gestures with facial expression including neutral, negative and positive meanings from American Sign Languages (ASL). We combine the features in two levels by employing two fusion strategies. At the feature level, an early feature combination can be performed by concatenating and weighting different feature groups, and LDA is used to choose the most discriminative elements by projecting the feature on a discriminative expression space. The second strategy is applied on decision level. Weighted decisions from single modalities are fused in a later stage. A condensation-based algorithm is adopted for classification. We collected a data set with three to seven recording sessions and conducted experiments with the combination techniques. Experimental results showed that facial analysis improve hand gesture recognition, decision level fusion performs better than feature level fusion.

  10. Real-Time Multiview Recognition of Human Gestures by Distributed Image Processing

    Directory of Open Access Journals (Sweden)

    Sato Kosuke

    2010-01-01

    Full Text Available Since a gesture involves a dynamic and complex motion, multiview observation and recognition are desirable. For the better representation of gestures, one needs to know, in the first place, from which views a gesture should be observed. Furthermore, it becomes increasingly important how the recognition results are integrated when larger numbers of camera views are considered. To investigate these problems, we propose a framework under which multiview recognition is carried out, and an integration scheme by which the recognition results are integrated online and in realtime. For performance evaluation, we use the ViHASi (Virtual Human Action Silhouette public image database as a benchmark and our Japanese sign language (JSL image database that contains 18 kinds of hand signs. By examining the recognition rates of each gesture for each view, we found gestures that exhibit view dependency and the gestures that do not. Also, we found that the view dependency itself could vary depending on the target gesture sets. By integrating the recognition results of different views, our swarm-based integration provides more robust and better recognition performance than individual fixed-view recognition agents.

  11. Good and bad in the hands of politicians: spontaneous gestures during positive and negative speech.

    Directory of Open Access Journals (Sweden)

    Daniel Casasanto

    2010-07-01

    Full Text Available According to the body-specificity hypothesis, people with different bodily characteristics should form correspondingly different mental representations, even in highly abstract conceptual domains. In a previous test of this proposal, right- and left-handers were found to associate positive ideas like intelligence, attractiveness, and honesty with their dominant side and negative ideas with their non-dominant side. The goal of the present study was to determine whether 'body-specific' associations of space and valence can be observed beyond the laboratory in spontaneous behavior, and whether these implicit associations have visible consequences.We analyzed speech and gesture (3012 spoken clauses, 1747 gestures from the final debates of the 2004 and 2008 US presidential elections, which involved two right-handers (Kerry, Bush and two left-handers (Obama, McCain. Blind, independent coding of speech and gesture allowed objective hypothesis testing. Right- and left-handed candidates showed contrasting associations between gesture and speech. In both of the left-handed candidates, left-hand gestures were associated more strongly with positive-valence clauses and right-hand gestures with negative-valence clauses; the opposite pattern was found in both right-handed candidates.Speakers associate positive messages more strongly with dominant hand gestures and negative messages with non-dominant hand gestures, revealing a hidden link between action and emotion. This pattern cannot be explained by conventions in language or culture, which associate 'good' with 'right' but not with 'left'; rather, results support and extend the body-specificity hypothesis. Furthermore, results suggest that the hand speakers use to gesture may have unexpected (and probably unintended communicative value, providing the listener with a subtle index of how the speaker feels about the content of the co-occurring speech.

  12. Evaluation of the safety and usability of touch gestures in operating in-vehicle information systems with visual occlusion.

    Science.gov (United States)

    Kim, Huhn; Song, Haewon

    2014-05-01

    Nowadays, many automobile manufacturers are interested in applying the touch gestures that are used in smart phones to operate their in-vehicle information systems (IVISs). In this study, an experiment was performed to verify the applicability of touch gestures in the operation of IVISs from the viewpoints of both driving safety and usability. In the experiment, two devices were used: one was the Apple iPad, with which various touch gestures such as flicking, panning, and pinching were enabled; the other was the SK EnNavi, which only allowed tapping touch gestures. The participants performed the touch operations using the two devices under visually occluded situations, which is a well-known technique for estimating load of visual attention while driving. In scrolling through a list, the flicking gestures required more time than the tapping gestures. Interestingly, both the flicking and simple tapping gestures required slightly higher visual attention. In moving a map, the average time taken per operation and the visual attention load required for the panning gestures did not differ from those of the simple tapping gestures that are used in existing car navigation systems. In zooming in/out of a map, the average time taken per pinching gesture was similar to that of the tapping gesture but required higher visual attention. Moreover, pinching gestures at a display angle of 75° required that the participants severely bend their wrists. Because the display angles of many car navigation systems tends to be more than 75°, pinching gestures can cause severe fatigue on users' wrists. Furthermore, contrary to participants' evaluation of other gestures, several participants answered that the pinching gesture was not necessary when operating IVISs. It was found that the panning gesture is the only touch gesture that can be used without negative consequences when operating IVISs while driving. The flicking gesture is likely to be used if the screen moving speed is slower or

  13. Data Fusion Research of Triaxial Human Body Motion Gesture based on Decision Tree

    Directory of Open Access Journals (Sweden)

    Feihong Zhou

    2014-05-01

    Full Text Available The development status of human body motion gesture data fusion domestic and overseas has been analyzed. A triaxial accelerometer is adopted to develop a wearable human body motion gesture monitoring system aimed at old people healthcare. On the basis of a brief introduction of decision tree algorithm, the WEKA workbench is adopted to generate a human body motion gesture decision tree. At last, the classification quality of the decision tree has been validated through experiments. The experimental results show that the decision tree algorithm could reach an average predicting accuracy of 97.5 % with lower time cost.

  14. Gesturing Entangled Journeys - Mobilities Design in Aalborg East, Denmark

    DEFF Research Database (Denmark)

    Lanng, Ditte Bendix

    2015-01-01

    situations are used in re-design experiments, which explore potential architectures of the suburban functionalist transit spaces to not only invite safe and effective transport but also gesture towards a richness of wayfaring ways of life. The empirical and experimental work was developed in a reciprocal...... and for being desensitised, placeless environments with little cultural and social value. This critique is acknowledged in the thesis, but it is also challenged by asking whether these transit Spaces could, in fact, be some of our cherished public spaces. The thesis explores how social and sensorial mobile...... on-the-move in transit spaces, the thesis explores what happens between point A and point B during daily life journeys to and from school, work, shopping, and other destinations. Along these journeys life is lived in transit spaces. While wayfarers are on the move they are also dwelling...

  15. Play-solicitation gestures in chimpanzees in the wild: flexible adjustment to social circumstances and individual matrices.

    Science.gov (United States)

    Fröhlich, Marlen; Wittig, Roman M; Pika, Simone

    2016-08-01

    Social play is a frequent behaviour in great apes and involves sophisticated forms of communicative exchange. While it is well established that great apes test and practise the majority of their gestural signals during play interactions, the influence of demographic factors and kin relationships between the interactants on the form and variability of gestures are relatively little understood. We thus carried out the first systematic study on the exchange of play-soliciting gestures in two chimpanzee ( Pan troglodytes ) communities of different subspecies. We examined the influence of age, sex and kin relationships of the play partners on gestural play solicitations, including object-associated and self-handicapping gestures. Our results demonstrated that the usage of (i) audible and visual gestures increased significantly with infant age, (ii) tactile gestures differed between the sexes, and (iii) audible and visual gestures were higher in interactions with conspecifics than with mothers. Object-associated and self-handicapping gestures were frequently used to initiate play with same-aged and younger play partners, respectively. Our study thus strengthens the view that gestures are mutually constructed communicative means, which are flexibly adjusted to social circumstances and individual matrices of interactants.

  16. Body language: The interplay between positional behavior and gestural signaling in the genus Pan and its implications for language evolution.

    Science.gov (United States)

    Smith, Lindsey W; Delgado, Roberto A

    2015-08-01

    The gestural repertoires of bonobos and chimpanzees are well documented, but the relationship between gestural signaling and positional behavior (i.e., body postures and locomotion) has yet to be explored. Given that one theory for language evolution attributes the emergence of increased gestural communication to habitual bipedality, this relationship is important to investigate. In this study, we examined the interplay between gestures, body postures, and locomotion in four captive groups of bonobos and chimpanzees using ad libitum and focal video data. We recorded 43 distinct manual (involving upper limbs and/or hands) and bodily (involving postures, locomotion, head, lower limbs, or feet) gestures. In both species, actors used manual and bodily gestures significantly more when recipients were attentive to them, suggesting these movements are intentionally communicative. Adults of both species spent less than 1.0% of their observation time in bipedal postures or locomotion, yet 14.0% of all bonobo gestures and 14.7% of all chimpanzee gestures were produced when subjects were engaged in bipedal postures or locomotion. Among both bonobo groups and one chimpanzee group, these were mainly manual gestures produced by infants and juvenile females. Among the other chimpanzee group, however, these were mainly bodily gestures produced by adult males in which bipedal posture and locomotion were incorporated into communicative displays. Overall, our findings reveal that bipedality did not prompt an increase in manual gesturing in these study groups. Rather, body postures and locomotion are intimately tied to many gestures and certain modes of locomotion can be used as gestures themselves. © 2015 Wiley Periodicals, Inc.

  17. Bringing back the body into the mind: gestures enhance word learning in foreign language.

    Science.gov (United States)

    Macedonia, Manuela

    2014-01-01

    Foreign language education in the twenty-first century still teaches vocabulary mainly through reading and listening activities. This is due to the link between teaching practice and traditional philosophy of language, where language is considered to be an abstract phenomenon of the mind. However, a number of studies have shown that accompanying words or phrases of a foreign language with gestures leads to better memory results. In this paper, I review behavioral research on the positive effects of gestures on memory. Then I move to the factors that have been addressed as contributing to the effect, and I embed the reviewed evidence in the theoretical framework of embodiment. Finally, I argue that gestures accompanying foreign language vocabulary learning create embodied representations of those words. I conclude by advocating the use of gestures in future language education as a learning tool that enhances the mind.

  18. Angle-of-arrival-based gesture recognition using ultrasonic multi-frequency signals

    KAUST Repository

    Chen, Hui; Ballal, Tarig; Saad, Mohamed; Al-Naffouri, Tareq Y.

    2017-01-01

    transducer. The 2-D angles of the moving hand are estimated using multi-frequency signals captured by a fixed receiver array. A simple redundant dictionary matching classifier is designed to recognize gestures representing the numbers from `0' to `9

  19. Interactive and Stereoscopic Hybrid 3D Viewer of Radar Data with Gesture Recognition

    Science.gov (United States)

    Goenetxea, Jon; Moreno, Aitor; Unzueta, Luis; Galdós, Andoni; Segura, Álvaro

    This work presents an interactive and stereoscopic 3D viewer of weather information coming from a Doppler radar. The hybrid system shows a GIS model of the regional zone where the radar is located and the corresponding reconstructed 3D volume weather data. To enhance the immersiveness of the navigation, stereoscopic visualization has been added to the viewer, using a polarized glasses based system. The user can interact with the 3D virtual world using a Nintendo Wiimote for navigating through it and a Nintendo Wii Nunchuk for giving commands by means of hand gestures. We also present a dynamic gesture recognition procedure that measures the temporal advance of the performed gesture postures. Experimental results show how dynamic gestures are effectively recognized so that a more natural interaction and immersive navigation in the virtual world is achieved.

  20. Online integration of information from speech and gesture: Insights from event related potentials

    NARCIS (Netherlands)

    Özyürek, A.; Willems, R.M.; Kita, S.; Hagoort, P.

    2007-01-01

    During language comprehension, listeners use the global semantic representation from previous sentence or discourse context to immediately integrate the meaning of each upcoming word into the unfolding message-level representation. Here we investigate whether communicative gestures that often

  1. Interface Everywhere: Further Development of a Gesture and Voice Commanding Interface

    Data.gov (United States)

    National Aeronautics and Space Administration — To capitalize on developments in voice and gesture control, we must identify a framework by which reliable control interfaces can be developed before these...

  2. Bringing back the body into the mind: Gestures enhance word learning in foreign language

    Directory of Open Access Journals (Sweden)

    Manuela eMacedonia

    2014-12-01

    Full Text Available Foreign language education in the 21st century still teaches vocabulary mainly through reading and listening activities. This is due to the link between teaching practice and traditional philosophy of language, where language is considered to be an abstract phenomenon of the mind. However, a number of studies have shown that accompanying words or phrases of a foreign language with gestures leads to better memory results. In this paper, I review behavioral research on the positive effects of gestures on memory. Then I move to the factors that have been addressed as contributing to the effect, and I embed the reviewed evidence in the theoretical framework of embodiment. Finally, I argue that gestures accompanying foreign language vocabulary learning create embodied representations of those words. I conclude by advocating the use of gestures in future language education as a learning tool that enhances learning the mind.

  3. Imitation in Newborn Infants: Exploring the Range of Gestures Imitated and the Underlying Mechanisms.

    Science.gov (United States)

    Meltzoff, Andrew N.; Moore, M. Keith

    1989-01-01

    Evaluated psychological mechanisms underlying imitation of facial actions in 40 newborn infants. Results showed imitation of head movement and a tongue-protrusion gesture. Subjects imitated from memory after displays had stopped. (RJC)

  4. User-independent accelerometer-based gesture recognition for mobile devices

    Directory of Open Access Journals (Sweden)

    Eduardo METOLA

    2013-07-01

    Full Text Available Many mobile devices embed nowadays inertial sensors. This enables new forms of human-computer interaction through the use of gestures (movements performed with the mobile device as a way of communication. This paper presents an accelerometer-based gesture recognition system for mobile devices which is able to recognize a collection of 10 different hand gestures. The system was conceived to be light and to operate in a user-independent manner in real time. The recognition system was implemented in a smart phone and evaluated through a collection of user tests, which showed a recognition accuracy similar to other state-of-the art techniques and a lower computational complexity. The system was also used to build a human-robot interface that enables controlling a wheeled robot with the gestures made with the mobile phone

  5. Understanding Human Hand Gestures for Learning Robot Pick-and-Place Tasks

    Directory of Open Access Journals (Sweden)

    Hsien-I Lin

    2015-05-01

    Full Text Available Programming robots by human demonstration is an intuitive approach, especially by gestures. Because robot pick-and-place tasks are widely used in industrial factories, this paper proposes a framework to learn robot pick-and-place tasks by understanding human hand gestures. The proposed framework is composed of the module of gesture recognition and the module of robot behaviour control. For the module of gesture recognition, transport empty (TE, transport loaded (TL, grasp (G, and release (RL from Gilbreth's therbligs are the hand gestures to be recognized. A convolution neural network (CNN is adopted to recognize these gestures from a camera image. To achieve the robust performance, the skin model by a Gaussian mixture model (GMM is used to filter out non-skin colours of an image, and the calibration of position and orientation is applied to obtain the neutral hand pose before the training and testing of the CNN. For the module of robot behaviour control, the corresponding robot motion primitives to TE, TL, G, and RL, respectively, are implemented in the robot. To manage the primitives in the robot system, a behaviour-based programming platform based on the Extensible Agent Behavior Specification Language (XABSL is adopted. Because the XABSL provides the flexibility and re-usability of the robot primitives, the hand motion sequence from the module of gesture recognition can be easily used in the XABSL programming platform to implement the robot pick-and-place tasks. The experimental evaluation of seven subjects performing seven hand gestures showed that the average recognition rate was 95.96%. Moreover, by the XABSL programming platform, the experiment showed the cube-stacking task was easily programmed by human demonstration.

  6. Intraspecific gestural laterality in chimpanzees and gorillas and the impact of social propensities.

    Science.gov (United States)

    Prieur, Jacques; Pika, Simone; Barbu, Stéphanie; Blois-Heulin, Catherine

    2017-09-01

    A relevant approach to address the mechanisms underlying the emergence of the right-handedness/left-hemisphere language specialization of humans is to investigate both proximal and distal causes of language lateralization through the study of non-human primates' gestural laterality. We carried out the first systematic, quantitative comparison of within-subjects' and between-species' laterality by focusing on the laterality of intraspecific gestures of chimpanzees (Pan troglodytes) and gorillas (Gorilla gorilla) living in six different captive groups. We addressed the following two questions: (1) Do chimpanzees and gorillas exhibit stable direction of laterality when producing different types of gestures at the individual level? If yes, is it related to the strength of laterality? (2) Is there a species difference in gestural laterality at the population level? If yes, which factors could explain this difference? During 1356 observation hours, we recorded 42335 cases of dyadic gesture use in the six groups totalling 39 chimpanzees and 35 gorillas. Results showed that both species could exhibit either stability or flexibility in their direction of gestural laterality. These results suggest that both stability and flexibility may have differently modulated the strength of laterality depending on the species social structure and dynamics. Furthermore, a multifactorial analysis indicates that these particular social components may have specifically impacted gestural laterality through the influence of gesture sensory modality and the position of the recipient in the signaller's visual field during interaction. Our findings provide further support to the social theory of laterality origins proposing that social pressures may have shaped laterality through natural selection. Copyright © 2017 Elsevier B.V. All rights reserved.

  7. Music as a mnemonic to learn gesture sequences in normal aging and Alzheimer’s disease

    OpenAIRE

    Aline eMoussard; Emmanuel eBigand; Emmanuel eBigand; Isabelle ePeretz; Isabelle ePeretz; Isabelle ePeretz; Sylvie eBelleville; Sylvie eBelleville

    2014-01-01

    Strong links between music and motor functions suggest that music could represent an interesting aid for motor learning. The present study aims for the first time to test the potential of music to assist in the learning of sequences of gestures in normal and pathological aging. Participants with mild Alzheimer's disease (AD) and healthy older adults (Controls) learned sequences of meaningless gestures that were either accompanied by music or a metronome. We also manipulated the learning proce...

  8. Music as a Mnemonic to Learn Gesture Sequences in Normal Aging and Alzheimer’s Disease

    OpenAIRE

    Moussard, Aline; Bigand, Emmanuel; Belleville, Sylvie; Peretz, Isabelle

    2014-01-01

    Strong links between music and motor functions suggest that music could represent an interesting aid for motor learning. The present study aims for the first time to test the potential of music to assist in the learning of sequences of gestures in normal and pathological aging. Participants with mild Alzheimer’s disease (AD) and healthy older adults (controls) learned sequences of meaningless gestures that were either accompanied by music or a metronome. We also manipulated the learning proce...

  9. Two-Stage Hidden Markov Model in Gesture Recognition for Human Robot Interaction

    Directory of Open Access Journals (Sweden)

    Nhan Nguyen-Duc-Thanh

    2012-07-01

    Full Text Available Hidden Markov Model (HMM is very rich in mathematical structure and hence can form the theoretical basis for use in a wide range of applications including gesture representation. Most research in this field, however, uses only HMM for recognizing simple gestures, while HMM can definitely be applied for whole gesture meaning recognition. This is very effectively applicable in Human-Robot Interaction (HRI. In this paper, we introduce an approach for HRI in which not only the human can naturally control the robot by hand gesture, but also the robot can recognize what kind of task it is executing. The main idea behind this method is the 2-stages Hidden Markov Model. The 1st HMM is to recognize the prime command-like gestures. Based on the sequence of prime gestures that are recognized from the 1st stage and which represent the whole action, the 2nd HMM plays a role in task recognition. Another contribution of this paper is that we use the output Mixed Gaussian distribution in HMM to improve the recognition rate. In the experiment, we also complete a comparison of the different number of hidden states and mixture components to obtain the optimal one, and compare to other methods to evaluate this performance.

  10. Motion-sensor fusion-based gesture recognition and its VLSI architecture design for mobile devices

    Science.gov (United States)

    Zhu, Wenping; Liu, Leibo; Yin, Shouyi; Hu, Siqi; Tang, Eugene Y.; Wei, Shaojun

    2014-05-01

    With the rapid proliferation of smartphones and tablets, various embedded sensors are incorporated into these platforms to enable multimodal human-computer interfaces. Gesture recognition, as an intuitive interaction approach, has been extensively explored in the mobile computing community. However, most gesture recognition implementations by now are all user-dependent and only rely on accelerometer. In order to achieve competitive accuracy, users are required to hold the devices in predefined manner during the operation. In this paper, a high-accuracy human gesture recognition system is proposed based on multiple motion sensor fusion. Furthermore, to reduce the energy overhead resulted from frequent sensor sampling and data processing, a high energy-efficient VLSI architecture implemented on a Xilinx Virtex-5 FPGA board is also proposed. Compared with the pure software implementation, approximately 45 times speed-up is achieved while operating at 20 MHz. The experiments show that the average accuracy for 10 gestures achieves 93.98% for user-independent case and 96.14% for user-dependent case when subjects hold the device randomly during completing the specified gestures. Although a few percent lower than the conventional best result, it still provides competitive accuracy acceptable for practical usage. Most importantly, the proposed system allows users to hold the device randomly during operating the predefined gestures, which substantially enhances the user experience.

  11. RGBD Video Based Human Hand Trajectory Tracking and Gesture Recognition System

    Directory of Open Access Journals (Sweden)

    Weihua Liu

    2015-01-01

    Full Text Available The task of human hand trajectory tracking and gesture trajectory recognition based on synchronized color and depth video is considered. Toward this end, in the facet of hand tracking, a joint observation model with the hand cues of skin saliency, motion and depth is integrated into particle filter in order to move particles to local peak in the likelihood. The proposed hand tracking method, namely, salient skin, motion, and depth based particle filter (SSMD-PF, is capable of improving the tracking accuracy considerably, in the context of the signer performing the gesture toward the camera device and in front of moving, cluttered backgrounds. In the facet of gesture recognition, a shape-order context descriptor on the basis of shape context is introduced, which can describe the gesture in spatiotemporal domain. The efficient shape-order context descriptor can reveal the shape relationship and embed gesture sequence order information into descriptor. Moreover, the shape-order context leads to a robust score for gesture invariant. Our approach is complemented with experimental results on the settings of the challenging hand-signed digits datasets and American sign language dataset, which corroborate the performance of the novel techniques.

  12. Imitation and matching of meaningless gestures: distinct involvement from motor and visual imagery.

    Science.gov (United States)

    Lesourd, Mathieu; Navarro, Jordan; Baumard, Josselin; Jarry, Christophe; Le Gall, Didier; Osiurak, François

    2017-05-01

    The aim of the present study was to understand the underlying cognitive processes of imitation and matching of meaningless gestures. Neuropsychological evidence obtained in brain damaged patients, has shown that distinct cognitive processes supported imitation and matching of meaningless gestures. Left-brain damaged (LBD) patients failed to imitate while right-brain damaged (RBD) patients failed to match meaningless gestures. Moreover, other studies with brain damaged patients showed that LBD patients were impaired in motor imagery while RBD patients were impaired in visual imagery. Thus, we hypothesize that imitation of meaningless gestures might rely on motor imagery, whereas matching of meaningless gestures might be based on visual imagery. In a first experiment, using a correlational design, we demonstrated that posture imitation relies on motor imagery but not on visual imagery (Experiment 1a) and that posture matching relies on visual imagery but not on motor imagery (Experiment 1b). In a second experiment, by manipulating directly the body posture of the participants, we demonstrated that such manipulation evokes a difference only in imitation task but not in matching task. In conclusion, the present study provides direct evidence that the way we imitate or we have to compare postures depends on motor imagery or visual imagery, respectively. Our results are discussed in the light of recent findings about underlying mechanisms of meaningful and meaningless gestures.

  13. Development of Pointing Gestures in Children With Typical and Delayed Language Acquisition.

    Science.gov (United States)

    Lüke, Carina; Ritterfeld, Ute; Grimminger, Angela; Liszkowski, Ulf; Rohlfing, Katharina J

    2017-11-09

    This longitudinal study compared the development of hand and index-finger pointing in children with typical language development (TD) and children with language delay (LD). First, we examined whether the number and the form of pointing gestures during the second year of life are potential indicators of later LD. Second, we analyzed the influence of caregivers' gestural and verbal input on children's communicative development. Thirty children with TD and 10 children with LD were observed together with their primary caregivers in a seminatural setting in 5 sessions between the ages of 12 and 21 months. Language skills were assessed at 24 months. Compared with children with TD, children with LD used fewer index-finger points at 12 and 14 months but more pointing gestures in total at 21 months. There were no significant differences in verbal or gestural input between caregivers of children with or without LD. Using more index-finger points at the beginning of the second year of life is associated with TD, whereas using more pointing gestures at the end of the second year of life is associated with delayed acquisition. Neither the verbal nor gestural input of caregivers accounted for differences in children's skills.

  14. Traffic Command Gesture Recognition for Virtual Urban Scenes Based on a Spatiotemporal Convolution Neural Network

    Directory of Open Access Journals (Sweden)

    Chunyong Ma

    2018-01-01

    Full Text Available Intelligent recognition of traffic police command gestures increases authenticity and interactivity in virtual urban scenes. To actualize real-time traffic gesture recognition, a novel spatiotemporal convolution neural network (ST-CNN model is presented. We utilized Kinect 2.0 to construct a traffic police command gesture skeleton (TPCGS dataset collected from 10 volunteers. Subsequently, convolution operations on the locational change of each skeletal point were performed to extract temporal features, analyze the relative positions of skeletal points, and extract spatial features. After temporal and spatial features based on the three-dimensional positional information of traffic police skeleton points were extracted, the ST-CNN model classified positional information into eight types of Chinese traffic police gestures. The test accuracy of the ST-CNN model was 96.67%. In addition, a virtual urban traffic scene in which real-time command tests were carried out was set up, and a real-time test accuracy rate of 93.0% was achieved. The proposed ST-CNN model ensured a high level of accuracy and robustness. The ST-CNN model recognized traffic command gestures, and such recognition was found to control vehicles in virtual traffic environments, which enriches the interactive mode of the virtual city scene. Traffic command gesture recognition contributes to smart city construction.

  15. The brain's dorsal route for speech represents word meaning: evidence from gesture.

    Science.gov (United States)

    Josse, Goulven; Joseph, Sabine; Bertasi, Eric; Giraud, Anne-Lise

    2012-01-01

    The dual-route model of speech processing includes a dorsal stream that maps auditory to motor features at the sublexical level rather than at the lexico-semantic level. However, the literature on gesture is an invitation to revise this model because it suggests that the premotor cortex of the dorsal route is a major site of lexico-semantic interaction. Here we investigated lexico-semantic mapping using word-gesture pairs that were either congruent or incongruent. Using fMRI-adaptation in 28 subjects, we found that temporo-parietal and premotor activity during auditory processing of single action words was modulated by the prior audiovisual context in which the words had been repeated. The BOLD signal was suppressed following repetition of the auditory word alone, and further suppressed following repetition of the word accompanied by a congruent gesture (e.g. ["grasp" + grasping gesture]). Conversely, repetition suppression was not observed when the same action word was accompanied by an incongruent gesture (e.g. ["grasp" + sprinkle]). We propose a simple model to explain these results: auditory and visual information converge onto premotor cortex where it is represented in a comparable format to determine (in)congruence between speech and gesture. This ability of the dorsal route to detect audiovisual semantic (in)congruence suggests that its function is not restricted to the sublexical level.

  16. Domestic dogs use contextual information and tone of voice when following a human pointing gesture.

    Directory of Open Access Journals (Sweden)

    Linda Scheider

    Full Text Available Domestic dogs are skillful at using the human pointing gesture. In this study we investigated whether dogs take contextual information into account when following pointing gestures, specifically, whether they follow human pointing gestures more readily in the context in which food has been found previously. Also varied was the human's tone of voice as either imperative or informative. Dogs were more sustained in their searching behavior in the 'context' condition as opposed to the 'no context' condition, suggesting that they do not simply follow a pointing gesture blindly but use previously acquired contextual information to inform their interpretation of that pointing gesture. Dogs also showed more sustained searching behavior when there was pointing than when there was not, suggesting that they expect to find a referent when they see a human point. Finally, dogs searched more in high-pitched informative trials as opposed to the low-pitched imperative trials, whereas in the latter dogs seemed more inclined to respond by sitting. These findings suggest that a dog's response to a pointing gesture is flexible and depends on the context as well as the human's tone of voice.

  17. Semantic brain areas are involved in gesture comprehension: An electrical neuroimaging study.

    Science.gov (United States)

    Proverbio, Alice Mado; Gabaro, Veronica; Orlandi, Andrea; Zani, Alberto

    2015-08-01

    While the mechanism of sign language comprehension in deaf people has been widely investigated, little is known about the neural underpinnings of spontaneous gesture comprehension in healthy speakers. Bioelectrical responses to 800 pictures of actors showing common Italian gestures (e.g., emblems, deictic or iconic gestures) were recorded in 14 persons. Stimuli were selected from a wider corpus of 1122 gestures. Half of the pictures were preceded by an incongruent description. ERPs were recorded from 128 sites while participants decided whether the stimulus was congruent. Congruent pictures elicited a posterior P300 followed by late positivity, while incongruent gestures elicited an anterior N400 response. N400 generators were investigated with swLORETA reconstruction. Processing of congruent gestures activated face- and body-related visual areas (e.g., BA19, BA37, BA22), the left angular gyrus, mirror fronto/parietal areas. The incongruent-congruent contrast particularly stimulated linguistic and semantic brain areas, such as the left medial and the superior temporal lobe. Copyright © 2015 Elsevier Inc. All rights reserved.

  18. The impact of iconic gestures on foreign language word learning and its neural substrate.

    Science.gov (United States)

    Macedonia, Manuela; Müller, Karsten; Friederici, Angela D

    2011-06-01

    Vocabulary acquisition represents a major challenge in foreign language learning. Research has demonstrated that gestures accompanying speech have an impact on memory for verbal information in the speakers' mother tongue and, as recently shown, also in foreign language learning. However, the neural basis of this effect remains unclear. In a within-subjects design, we compared learning of novel words coupled with iconic and meaningless gestures. Iconic gestures helped learners to significantly better retain the verbal material over time. After the training, participants' brain activity was registered by means of fMRI while performing a word recognition task. Brain activations to words learned with iconic and with meaningless gestures were contrasted. We found activity in the premotor cortices for words encoded with iconic gestures. In contrast, words encoded with meaningless gestures elicited a network associated with cognitive control. These findings suggest that memory performance for newly learned words is not driven by the motor component as such, but by the motor image that matches an underlying representation of the word's semantics. Copyright © 2010 Wiley-Liss, Inc.

  19. Co-speech iconic gestures and visuo-spatial working memory.

    Science.gov (United States)

    Wu, Ying Choon; Coulson, Seana

    2014-11-01

    Three experiments tested the role of verbal versus visuo-spatial working memory in the comprehension of co-speech iconic gestures. In Experiment 1, participants viewed congruent discourse primes in which the speaker's gestures matched the information conveyed by his speech, and incongruent ones in which the semantic content of the speaker's gestures diverged from that in his speech. Discourse primes were followed by picture probes that participants judged as being either related or unrelated to the preceding clip. Performance on this picture probe classification task was faster and more accurate after congruent than incongruent discourse primes. The effect of discourse congruency on response times was linearly related to measures of visuo-spatial, but not verbal, working memory capacity, as participants with greater visuo-spatial WM capacity benefited more from congruent gestures. In Experiments 2 and 3, participants performed the same picture probe classification task under conditions of high and low loads on concurrent visuo-spatial (Experiment 2) and verbal (Experiment 3) memory tasks. Effects of discourse congruency and verbal WM load were additive, while effects of discourse congruency and visuo-spatial WM load were interactive. Results suggest that congruent co-speech gestures facilitate multi-modal language comprehension, and indicate an important role for visuo-spatial WM in these speech-gesture integration processes. Copyright © 2014 Elsevier B.V. All rights reserved.

  20. Gesture recognition based on computer vision and glove sensor for remote working environments

    Energy Technology Data Exchange (ETDEWEB)

    Chien, Sung Il; Kim, In Chul; Baek, Yung Mok; Kim, Dong Su; Jeong, Jee Won; Shin, Kug [Kyungpook National University, Taegu (Korea)

    1998-04-01

    In this research, we defined a gesture set needed for remote monitoring and control of a manless system in atomic power station environments. Here, we define a command as the loci of a gesture. We aim at the development of an algorithm using a vision sensor and glove sensors in order to implement the gesture recognition system. The gesture recognition system based on computer vision tracks a hand by using cross correlation of PDOE image. To recognize the gesture word, the 8 direction code is employed as the input symbol for discrete HMM. Another gesture recognition based on sensor has introduced Pinch glove and Polhemus sensor as an input device. The extracted feature through preprocessing now acts as an input signal of the recognizer. For recognition 3D loci of Polhemus sensor, discrete HMM is also adopted. The alternative approach of two foregoing recognition systems uses the vision and and glove sensors together. The extracted mesh feature and 8 direction code from the locus tracking are introduced for further enhancing recognition performance. MLP trained by backpropagation is introduced here and its performance is compared to that of discrete HMM. (author). 32 refs., 44 figs., 21 tabs.

  1. Parents' Translations of Child Gesture Facilitate Word Learning in Children with Autism, Down Syndrome and Typical Development.

    Science.gov (United States)

    Dimitrova, Nevena; Özçalışkan, Şeyda; Adamson, Lauren B

    2016-01-01

    Typically-developing (TD) children frequently refer to objects uniquely in gesture. Parents translate these gestures into words, facilitating children's acquisition of these words (Goldin-Meadow et al. in Dev Sci 10(6):778-785, 2007). We ask whether this pattern holds for children with autism (AU) and with Down syndrome (DS) who show delayed vocabulary development. We observed 23 children with AU, 23 with DS, and 23 TD children with their parents over a year. Children used gestures to indicate objects before labeling them and parents translated their gestures into words. Importantly, children benefited from this input, acquiring more words for the translated gestures than the not translated ones. Results highlight the role contingent parental input to child gesture plays in language development of children with developmental disorders.

  2. How does linguistic framing of events influence co-speech gestures? Insights from crosslinguistic variations and similarities

    OpenAIRE

    Ozyurek, A.; Kita, S.; Allen, S.; Furman, R.; Brown, A.

    2005-01-01

    What are the relations between linguistic encoding and gestural representations of events during online speaking? The few studies that have been conducted on this topic have yielded somewhat incompatible results with regard to whether and how gestural representations of events change with differences in the preferred semantic and syntactic encoding possibilities of languages. Here we provide large scale semantic, syntactic and temporal analyses of speech- gesture pairs that depict 10 differen...

  3. What speakers do and what addressees look at: Visual attention to gestures in human interaction live and on video

    OpenAIRE

    Gullberg, Marianne; Holmqvist, Kenneth

    2006-01-01

    This study investigates whether addressees visually attend to speakers’ gestures in interaction and whether attention is modulated by changes in social setting and display size. We compare a live face-to-face setting to two video conditions. In all conditions, the face dominates as a fixation target and only a minority of gestures draw fixations. The social and size parameters affect gaze mainly when combined and in the opposite direction from the predicted with fewer gestures fixated on vide...

  4. The Cinematic Bergson: From Virtual Image to Actual Gesture

    Directory of Open Access Journals (Sweden)

    John Ó Maoilearca

    2016-12-01

    Full Text Available Deleuze’s film-philosophy makes much of the notion of virtual images in Bergson’s Matter and Memory, but in doing so he transforms a psycho-meta-physical thesis into a (very unBergsonian ontological one. In this essay, we will offer a corrective by exploring Bergson’s own explanation of the image as an “attitude of the body”—something that projects an actual, corporeal, and postural approach, not only to cinema, but also to philosophy. Indeed, just as Renoir famously said that “a director makes only one movie in his life. Then he breaks it into pieces and makes it again,” so Bergson wrote that each philosopher only makes one “single point” throughout his or her whole career. And this one point, he then declares, is like a “vanishing image,” only one best understood as an attitude of the body. It is embodied image that underlies an alternative Bergsonian cinema of the actual and the body—one that we will examine through what Bergson’s has to say about “attitude” as well as “gesture” and “mime.” We will also look at it through a gestural concept enacted by a film, to be precise, the five remakes that comprise Lars von Trier’s and Jørgen Leth’s The Five Obstructions (2003. This will bring us back to the idea of what it is that is being remade, both by directors and philosophers, in Renoir’s “one film” and Bergson’s singular “vanishing image” respectively.  Is the “one” being remade an image understood as a representation, or is it a gesture, understood as a bodily movement? It is the latter stance that provides a wholly new and alternative view of Bergson’s philosophy of cinema.

  5. TeMoto: Intuitive Multi-Range Telerobotic System with Natural Gestural and Verbal Instruction Interface

    Directory of Open Access Journals (Sweden)

    Robert Valner

    2018-02-01

    Full Text Available Teleoperated mobile robots, equipped with object manipulation capabilities, provide safe means for executing dangerous tasks in hazardous environments without putting humans at risk. However, mainly due to a communication delay, complex operator interfaces and insufficient Situational Awareness (SA, the task productivity of telerobots remains inferior to human workers. This paper addresses the shortcomings of telerobots by proposing a combined approach of (i a scalable and intuitive operator interface with gestural and verbal input, (ii improved Situational Awareness (SA through sensor fusion according to documented best practices, (iii integrated virtual fixtures for task simplification and minimizing the operator’s cognitive burden and (iv integrated semiautonomous behaviors that further reduce cognitive burden and negate the impact of communication delays, execution latency and/or failures. The proposed teleoperation system, TeMoto, is implemented using ROS (Robot Operating System to ensure hardware agnosticism, extensibility and community access. The operator’s command interface consists of a Leap Motion Controller for hand tracking, Griffin PowerMate USB as turn knob for scaling and a microphone for speech input. TeMoto is evaluated on multiple robots including two mobile manipulator platforms. In addition to standard, task-specific evaluation techniques (completion time, user studies, number of steps, etc.—which are platform and task dependent and thus difficult to scale—this paper presents additional metrics for evaluating the user interface including task-independent criteria for measuring generalized (i task completion efficiency and (ii operator context switching.

  6. A real-time vision-based hand gesture interaction system for virtual EAST

    Energy Technology Data Exchange (ETDEWEB)

    Wang, K.R., E-mail: wangkr@mail.ustc.edu.cn [Institute of Plasma Physics, Chinese Academy of Sciences, Hefei, Anhui (China); University of Science and Technology of China, Hefei, Anhui (China); Xiao, B.J.; Xia, J.Y. [Institute of Plasma Physics, Chinese Academy of Sciences, Hefei, Anhui (China); University of Science and Technology of China, Hefei, Anhui (China); Li, Dan [Institute of Plasma Physics, Chinese Academy of Sciences, Hefei, Anhui (China); Luo, W.L. [709th Research Institute, Shipbuilding Industry Corporation (China)

    2016-11-15

    Highlights: • Hand gesture interaction is first introduced to EAST model interaction. • We can interact with EAST model by a bared hand and a web camera. • We can interact with EAST model with a distance to screen. • Interaction is free, direct and effective. - Abstract: The virtual Experimental Advanced Superconducting Tokamak device (VEAST) is a very complicated 3D model, to interact with which, the traditional interaction devices are limited and inefficient. However, with the development of human-computer interaction (HCI), the hand gesture interaction has become a much popular choice in recent years. In this paper, we propose a real-time vision-based hand gesture interaction system for VEAST. By using one web camera, we can use our bare hand to interact with VEAST at a certain distance, which proves to be more efficient and direct than mouse. The system is composed of four modules: initialization, hand gesture recognition, interaction control and system settings. The hand gesture recognition method is based on codebook (CB) background modeling and open finger counting. Firstly, we build a background model with CB algorithm. Then, we segment the hand region by detecting skin color regions with “elliptical boundary model” in CbCr flat of YCbCr color space. Open finger which is used as a key feature of gesture can be tracked by an improved curvature-based method. Based on the method, we define nine gestures for interaction control of VEAST. Finally, we design a test to demonstrate effectiveness of our system.

  7. Tracking and Classification of In-Air Hand Gesture Based on Thermal Guided Joint Filter.

    Science.gov (United States)

    Kim, Seongwan; Ban, Yuseok; Lee, Sangyoun

    2017-01-17

    The research on hand gestures has attracted many image processing-related studies, as it intuitively conveys the intention of a human as it pertains to motional meaning. Various sensors have been used to exploit the advantages of different modalities for the extraction of important information conveyed by the hand gesture of a user. Although many works have focused on learning the benefits of thermal information from thermal cameras, most have focused on face recognition or human body detection, rather than hand gesture recognition. Additionally, the majority of the works that take advantage of multiple modalities (e.g., the combination of a thermal sensor and a visual sensor), usually adopting simple fusion approaches between the two modalities. As both thermal sensors and visual sensors have their own shortcomings and strengths, we propose a novel joint filter-based hand gesture recognition method to simultaneously exploit the strengths and compensate the shortcomings of each. Our study is motivated by the investigation of the mutual supplementation between thermal and visual information in low feature level for the consistent representation of a hand in the presence of varying lighting conditions. Accordingly, our proposed method leverages the thermal sensor's stability against luminance and the visual sensors textural detail, while complementing the low resolution and halo effect of thermal sensors and the weakness against illumination of visual sensors. A conventional region tracking method and a deep convolutional neural network have been leveraged to track the trajectory of a hand gesture and to recognize the hand gesture, respectively. Our experimental results show stability in recognizing a hand gesture against varying lighting conditions based on the contribution of the joint kernels of spatial adjacency and thermal range similarity.

  8. A real-time vision-based hand gesture interaction system for virtual EAST

    International Nuclear Information System (INIS)

    Wang, K.R.; Xiao, B.J.; Xia, J.Y.; Li, Dan; Luo, W.L.

    2016-01-01

    Highlights: • Hand gesture interaction is first introduced to EAST model interaction. • We can interact with EAST model by a bared hand and a web camera. • We can interact with EAST model with a distance to screen. • Interaction is free, direct and effective. - Abstract: The virtual Experimental Advanced Superconducting Tokamak device (VEAST) is a very complicated 3D model, to interact with which, the traditional interaction devices are limited and inefficient. However, with the development of human-computer interaction (HCI), the hand gesture interaction has become a much popular choice in recent years. In this paper, we propose a real-time vision-based hand gesture interaction system for VEAST. By using one web camera, we can use our bare hand to interact with VEAST at a certain distance, which proves to be more efficient and direct than mouse. The system is composed of four modules: initialization, hand gesture recognition, interaction control and system settings. The hand gesture recognition method is based on codebook (CB) background modeling and open finger counting. Firstly, we build a background model with CB algorithm. Then, we segment the hand region by detecting skin color regions with “elliptical boundary model” in CbCr flat of YCbCr color space. Open finger which is used as a key feature of gesture can be tracked by an improved curvature-based method. Based on the method, we define nine gestures for interaction control of VEAST. Finally, we design a test to demonstrate effectiveness of our system.

  9. Usability Evaluation Methods for Gesture-Based Games: A Systematic Review.

    Science.gov (United States)

    Simor, Fernando Winckler; Brum, Manoela Rogofski; Schmidt, Jaison Dairon Ebertz; Rieder, Rafael; De Marchi, Ana Carolina Bertoletti

    2016-10-04

    Gestural interaction systems are increasingly being used, mainly in games, expanding the idea of entertainment and providing experiences with the purpose of promoting better physical and/or mental health. Therefore, it is necessary to establish mechanisms for evaluating the usability of these interfaces, which make gestures the basis of interaction, to achieve a balance between functionality and ease of use. This study aims to present the results of a systematic review focused on usability evaluation methods for gesture-based games, considering devices with motion-sensing capability. We considered the usability methods used, the common interface issues, and the strategies adopted to build good gesture-based games. The research was centered on four electronic databases: IEEE, Association for Computing Machinery (ACM), Springer, and Science Direct from September 4 to 21, 2015. Within 1427 studies evaluated, 10 matched the eligibility criteria. As a requirement, we considered studies about gesture-based games, Kinect and/or Wii as devices, and the use of a usability method to evaluate the user interface. In the 10 studies found, there was no standardization in the methods because they considered diverse analysis variables. Heterogeneously, authors used different instruments to evaluate gesture-based interfaces and no default approach was proposed. Questionnaires were the most used instruments (70%, 7/10), followed by interviews (30%, 3/10), and observation and video recording (20%, 2/10). Moreover, 60% (6/10) of the studies used gesture-based serious games to evaluate the performance of elderly participants in rehabilitation tasks. This highlights the need for creating an evaluation protocol for older adults to provide a user-friendly interface according to the user's age and limitations. Through this study, we conclude this field is in need of a usability evaluation method for serious games, especially games for older adults, and that the definition of a methodology

  10. Real-time face and gesture analysis for human-robot interaction

    Science.gov (United States)

    Wallhoff, Frank; Rehrl, Tobias; Mayer, Christoph; Radig, Bernd

    2010-05-01

    Human communication relies on a large number of different communication mechanisms like spoken language, facial expressions, or gestures. Facial expressions and gestures are one of the main nonverbal communication mechanisms and pass large amounts of information between human dialog partners. Therefore, to allow for intuitive human-machine interaction, a real-time capable processing and recognition of facial expressions, hand and head gestures are of great importance. We present a system that is tackling these challenges. The input features for the dynamic head gestures and facial expressions are obtained from a sophisticated three-dimensional model, which is fitted to the user in a real-time capable manner. Applying this model different kinds of information are extracted from the image data and afterwards handed over to a real-time capable data-transferring framework, the so-called Real-Time DataBase (RTDB). In addition to the head and facial-related features, also low-level image features regarding the human hand - optical flow, Hu-moments are stored into the RTDB for the evaluation process of hand gestures. In general, the input of a single camera is sufficient for the parallel evaluation of the different gestures and facial expressions. The real-time capable recognition of the dynamic hand and head gestures are performed via different Hidden Markov Models, which have proven to be a quick and real-time capable classification method. On the other hand, for the facial expressions classical decision trees or more sophisticated support vector machines are used for the classification process. These obtained results of the classification processes are again handed over to the RTDB, where other processes (like a Dialog Management Unit) can easily access them without any blocking effects. In addition, an adjustable amount of history can be stored by the RTDB buffer unit.

  11. Tracking and Classification of In-Air Hand Gesture Based on Thermal Guided Joint Filter

    Directory of Open Access Journals (Sweden)

    Seongwan Kim

    2017-01-01

    Full Text Available The research on hand gestures has attracted many image processing-related studies, as it intuitively conveys the intention of a human as it pertains to motional meaning. Various sensors have been used to exploit the advantages of different modalities for the extraction of important information conveyed by the hand gesture of a user. Although many works have focused on learning the benefits of thermal information from thermal cameras, most have focused on face recognition or human body detection, rather than hand gesture recognition. Additionally, the majority of the works that take advantage of multiple modalities (e.g., the combination of a thermal sensor and a visual sensor, usually adopting simple fusion approaches between the two modalities. As both thermal sensors and visual sensors have their own shortcomings and strengths, we propose a novel joint filter-based hand gesture recognition method to simultaneously exploit the strengths and compensate the shortcomings of each. Our study is motivated by the investigation of the mutual supplementation between thermal and visual information in low feature level for the consistent representation of a hand in the presence of varying lighting conditions. Accordingly, our proposed method leverages the thermal sensor’s stability against luminance and the visual sensors textural detail, while complementing the low resolution and halo effect of thermal sensors and the weakness against illumination of visual sensors. A conventional region tracking method and a deep convolutional neural network have been leveraged to track the trajectory of a hand gesture and to recognize the hand gesture, respectively. Our experimental results show stability in recognizing a hand gesture against varying lighting conditions based on the contribution of the joint kernels of spatial adjacency and thermal range similarity.

  12. Range and variability in gesture-based interactions with medical images : do non-stereo versus stereo visualizations elicit different types of gestures?

    NARCIS (Netherlands)

    Beurden, van M.H.P.H.; IJsselsteijn, W.A.

    2010-01-01

    The current paper presents a study into the range and variability of natural gestures when interacting with medical images, using traditional non stereo and stereoscopic modes of presentation. The results have implications for the design of computer-vision algorithms developed to support natural

  13. Gesturing beyond the Frame: Transnational Trauma and US War Fiction

    Directory of Open Access Journals (Sweden)

    Ruth A. H. Lahti

    2012-12-01

    Full Text Available The convergent boundary between the fields of trauma theory and US war fiction has resulted in a narrow focus on the subjectivity of the American soldier in war fiction, which partly conditions American war fiction's privileging of the soldier-author. However, this focus on American soldiers does not adequately account for the essentially interactive nature of war trauma, and it elides the experiences of nurses and noncombatants on all sides of the battle while also obscuring women's distinctive war experiences, even when the fiction itself sometimes includes these dimensions. In this essay, Lahti argues that a transnational method can counter these imbalances in trauma theory and in studies of US war fiction. She engages Tim O'Brien's highly influential The Things They Carried from a transnational perspective by interrogating the text's figuring of the survivor author and focusing on critically neglected scenes of interaction between the American soldiers and Vietnamese civilians. In order to discern the way these scenes reveal the text's own struggle with its national US frame, she elaborates a methodology of close reading characters' bodily gestures to foreground the way that fiction offers a glimpse into war as a relational event, always involving two or more participants. In the case of The Things They Carried, this approach brings into view a heretofore unnoticed pattern of mimicry between the American characters and Vietnamese characters that reshapes our scholarly understanding of the text's representation of war trauma.

  14. The effect of musical practice on gesture/sound pairing

    Directory of Open Access Journals (Sweden)

    Alice Mado eProverbio

    2015-04-01

    Full Text Available Learning to play a musical instrument is a demanding process requiring years of intense practice. Dramatic changes in brain connectivity, volume and functionality have been shown in skilled musicians. It is thought that music learning involves the formation of novel audio visuomotor associations, but not much is known about the gradual acquisition of this ability. In the present study, we investigated whether formal music training enhances audiovisual multisensory processing. To this end, pupils at different stages of education were examined based on the hypothesis that the strength of audio/visuomotor associations would be augmented as a function of the number of years of conservatory study (expertise. The study participants were violin and clarinet students of pre-academic and academic levels and of different chronological ages, ages of acquisition and academic levels. A violinist and a clarinetist each played the same score, and each participant viewed the video corresponding to his or her instrument. Pitch, intensity, rhythm and sound duration were matched across instruments. In half of the trials, the soundtrack did not match (in pitch the corresponding musical gestures. Data analysis indicated a correlation between the number of years of formal training (expertise and the ability to detect an audiomotor incongruence in music performance (relative to the musical instrument practiced, thus suggesting a direct correlation between knowing how to play and perceptual sensitivity.

  15. The impact of autism spectrum disorder symptoms on gesture use in fragile X syndrome and Down syndrome

    Directory of Open Access Journals (Sweden)

    Emily Lorang

    2017-12-01

    Full Text Available Background & aims This study compared gesture rate and purpose in participants with Down syndrome and fragile X syndrome, and the impact of autism spectrum disorder symptoms on each syndrome. Methods Twenty individuals with fragile X syndrome and 20 individuals with Down syndrome between nine and 22 years of age participated in this study. We coded gesture rate and purpose from an autism spectrum disorder evaluation, the Autism Diagnostic Observation Schedule – Second Edition. Results We did not find between-group differences (Down syndrome compared to fragile X syndrome in gesture rate or purpose. Notably, as autism spectrum disorder symptoms increased, the group with Down syndrome produced a lower rate of gestures, but used gestures for the same purpose. Gesture rate did not change based on autism spectrum disorder symptoms in the participants with fragile X syndrome, but as autism spectrum disorder symptoms increased, the participants with fragile X syndrome produced a larger proportion of gestures to regulate behavior and a smaller proportion for joint attention/social interaction. Conclusions Overall, the amount or purpose of gestures did not differentiate individuals with Down syndrome and fragile X syndrome. However, the presence of autism spectrum disorder symptoms had a significant and unique impact on these genetic disorders. In individuals with Down syndrome, the presence of more autism spectrum disorder symptoms resulted in a reduction in the rate of gesturing, but did not change the purpose. However, in fragile X syndrome, the rate of gestures remained the same, but the purpose of those gestures changed based on autism spectrum disorder symptoms. Implications Autism spectrum disorder symptoms differentially impact gestures in Down syndrome and fragile X syndrome. Individuals with Down syndrome and more autism spectrum disorder symptoms are using gestures less frequently. Therefore, clinicians may need to consider children with

  16. Transcranial Magnetic Stimulation over Left Inferior Frontal and Posterior Temporal Cortex Disrupts Gesture-Speech Integration.

    Science.gov (United States)

    Zhao, Wanying; Riggs, Kevin; Schindler, Igor; Holle, Henning

    2018-02-21

    Language and action naturally occur together in the form of cospeech gestures, and there is now convincing evidence that listeners display a strong tendency to integrate semantic information from both domains during comprehension. A contentious question, however, has been which brain areas are causally involved in this integration process. In previous neuroimaging studies, left inferior frontal gyrus (IFG) and posterior middle temporal gyrus (pMTG) have emerged as candidate areas; however, it is currently not clear whether these areas are causally or merely epiphenomenally involved in gesture-speech integration. In the present series of experiments, we directly tested for a potential critical role of IFG and pMTG by observing the effect of disrupting activity in these areas using transcranial magnetic stimulation in a mixed gender sample of healthy human volunteers. The outcome measure was performance on a Stroop-like gesture task (Kelly et al., 2010a), which provides a behavioral index of gesture-speech integration. Our results provide clear evidence that disrupting activity in IFG and pMTG selectively impairs gesture-speech integration, suggesting that both areas are causally involved in the process. These findings are consistent with the idea that these areas play a joint role in gesture-speech integration, with IFG regulating strategic semantic access via top-down signals acting upon temporal storage areas. SIGNIFICANCE STATEMENT Previous neuroimaging studies suggest an involvement of inferior frontal gyrus and posterior middle temporal gyrus in gesture-speech integration, but findings have been mixed and due to methodological constraints did not allow inferences of causality. By adopting a virtual lesion approach involving transcranial magnetic stimulation, the present study provides clear evidence that both areas are causally involved in combining semantic information arising from gesture and speech. These findings support the view that, rather than being

  17. Working Memory for Linguistic and Non-linguistic Manual Gestures: Evidence, Theory, and Application.

    Science.gov (United States)

    Rudner, Mary

    2018-01-01

    Linguistic manual gestures are the basis of sign languages used by deaf individuals. Working memory and language processing are intimately connected and thus when language is gesture-based, it is important to understand related working memory mechanisms. This article reviews work on working memory for linguistic and non-linguistic manual gestures and discusses theoretical and applied implications. Empirical evidence shows that there are effects of load and stimulus degradation on working memory for manual gestures. These effects are similar to those found for working memory for speech-based language. Further, there are effects of pre-existing linguistic representation that are partially similar across language modalities. But above all, deaf signers score higher than hearing non-signers on an n-back task with sign-based stimuli, irrespective of their semantic and phonological content, but not with non-linguistic manual actions. This pattern may be partially explained by recent findings relating to cross-modal plasticity in deaf individuals. It suggests that in linguistic gesture-based working memory, semantic aspects may outweigh phonological aspects when processing takes place under challenging conditions. The close association between working memory and language development should be taken into account in understanding and alleviating the challenges faced by deaf children growing up with cochlear implants as well as other clinical populations.

  18. When to Take a Gesture Seriously: On How We Use and Prioritize Communicative Cues.

    Science.gov (United States)

    Gunter, Thomas C; Weinbrenner, J E Douglas

    2017-08-01

    When people talk, their speech is often accompanied by gestures. Although it is known that co-speech gestures can influence face-to-face communication, it is currently unclear to what extent they are actively used and under which premises they are prioritized to facilitate communication. We investigated these open questions in two experiments that varied how pointing gestures disambiguate the utterances of an interlocutor. Participants, whose event-related brain responses were measured, watched a video, where an actress was interviewed about, for instance, classical literature (e.g., Goethe and Shakespeare). While responding, the actress pointed systematically to the left side to refer to, for example, Goethe, or to the right to refer to Shakespeare. Her final statement was ambiguous and combined with a pointing gesture. The P600 pattern found in Experiment 1 revealed that, when pointing was unreliable, gestures were only monitored for their cue validity and not used for reference tracking related to the ambiguity. However, when pointing was a valid cue (Experiment 2), it was used for reference tracking, as indicated by a reduced N400 for pointing. In summary, these findings suggest that a general prioritization mechanism is in use that constantly monitors and evaluates the use of communicative cues against communicative priors on the basis of accumulated error information.

  19. Automatic imitation of pro- and antisocial gestures: Is implicit social behavior censored?

    Science.gov (United States)

    Cracco, Emiel; Genschow, Oliver; Radkova, Ina; Brass, Marcel

    2018-01-01

    According to social reward theories, automatic imitation can be understood as a means to obtain positive social consequences. In line with this view, it has been shown that automatic imitation is modulated by contextual variables that constrain the positive outcomes of imitation. However, this work has largely neglected that many gestures have an inherent pro- or antisocial meaning. As a result of their meaning, antisocial gestures are considered taboo and should not be used in public. In three experiments, we show that automatic imitation of symbolic gestures is modulated by the social intent of these gestures. Experiment 1 (N=37) revealed reduced automatic imitation of antisocial compared with prosocial gestures. Experiment 2 (N=118) and Experiment 3 (N=118) used a social priming procedure to show that this effect was stronger in a prosocial context than in an antisocial context. These findings were supported in a within-study meta-analysis using both frequentist and Bayesian statistics. Together, our results indicate that automatic imitation is regulated by internalized social norms that act as a stop signal when inappropriate actions are triggered. Copyright © 2017 Elsevier B.V. All rights reserved.

  20. Gesture language use in natural UI: pen-based sketching in conceptual design

    Science.gov (United States)

    Ma, Cuixia; Dai, Guozhong

    2003-04-01

    Natural User Interface is one of the important next generation interactions. Computers are not just the tools of many special people or areas but for most people. Ubiquitous computing makes the world magic and more comfortable. In the design domain, current systems, which need the detail information, cannot conveniently support the conceptual design of the early phrase. Pen and paper are the natural and simple tools to use in our daily life, especially in design domain. Gestures are the useful and natural mode in the interaction of pen-based. In natural UI, gestures can be introduced and used through the similar mode to the existing resources in interaction. But the gestures always are defined beforehand without the users' intention and recognized to represent something in certain applications without being transplanted to others. We provide the gesture description language (GDL) to try to cite the useful gestures to the applications conveniently. It can be used in terms of the independent control resource such as menus or icons in applications. So we give the idea from two perspectives: one from the application-dependent point of view and the other from the application-independent point of view.

  1. Toward a new conception of conceptions: Interplay of talk, gestures, and structures in the setting

    Science.gov (United States)

    Givry, Damien; Roth, Wolff-Michael

    2006-12-01

    Most studies about students' conceptions and conceptual change are based exclusively on the analysis language, which is treated as a tool to make private contents of the mind public to researchers. Following recent studies that focused on (a) language and discursive practice and (b) the pragmatics of communication that draws on talk, gestures, and semiotic resources in the setting, we propose a redefinition of the nature of conception. Conceptions are understood as the dialectical relation of simultaneously available speech, gestures, and contextual structures that cannot be reduced to verbal rendering because gestures and contextual structures constitute different modalities in the communication. Drawing on data collected during a physics unit about gas taught in French tenth grade classrooms, we show why an appropriate account of conceptions requires: (a) gestures simultaneously produced with talk; and (b) identification of the relevant structures in the setting used by the participants as meaning-making (semiotic) resources. We propose to: (a) reconceptualize the notion of conception as consisting of a dialectical unit of all relevant semiotic (meaning-making) resources publicly made available by a speaker (talk, gesture, context); and (b) consider conceptual change through the temporal evolution of the dialectical unit defined in this manner.

  2. A Kinect-Based Gesture Recognition Approach for a Natural Human Robot Interface

    Directory of Open Access Journals (Sweden)

    Grazia Cicirelli

    2015-03-01

    Full Text Available In this paper, we present a gesture recognition system for the development of a human-robot interaction (HRI interface. Kinect cameras and the OpenNI framework are used to obtain real-time tracking of a human skeleton. Ten different gestures, performed by different persons, are defined. Quaternions of joint angles are first used as robust and significant features. Next, neural network (NN classifiers are trained to recognize the different gestures. This work deals with different challenging tasks, such as the real-time implementation of a gesture recognition system and the temporal resolution of gestures. The HRI interface developed in this work includes three Kinect cameras placed at different locations in an indoor environment and an autonomous mobile robot that can be remotely controlled by one operator standing in front of one of the Kinects. Moreover, the system is supplied with a people re-identification module which guarantees that only one person at a time has control of the robot. The system's performance is first validated offline, and then online experiments are carried out, proving the real-time operation of the system as required by a HRI interface.

  3. White Lies in Hand: Are Other-Oriented Lies Modified by Hand Gestures? Possibly Not

    Directory of Open Access Journals (Sweden)

    Katarzyna Cantarero

    2017-06-01

    Full Text Available Previous studies have shown that the hand-over-heart gesture is related to being more honest as opposed to using self-centered dishonesty. We assumed that the hand-over-heart gesture would also relate to other-oriented dishonesty, though the latter differs highly from self-centered lying. In Study 1 (N = 79, we showed that performing a hand-over-heart gesture diminished the tendency to use other-oriented white lies and that the fingers crossed behind one’s back gesture was not related to higher dishonesty. We then pre-registered and conducted Study 2 (N = 88, which was designed following higher methodological standards than Study 1. Contrary, to the findings of Study 1, we found that using the hand-over-heart gesture did not result in refraining from using other-oriented white lies. We discuss the findings of this failed replication indicating the importance of strict methodological guidelines in conducting research and also reflect on relatively small effect sizes related to some findings in embodied cognition.

  4. White Lies in Hand: Are Other-Oriented Lies Modified by Hand Gestures? Possibly Not.

    Science.gov (United States)

    Cantarero, Katarzyna; Parzuchowski, Michal; Dukala, Karolina

    2017-01-01

    Previous studies have shown that the hand-over-heart gesture is related to being more honest as opposed to using self-centered dishonesty. We assumed that the hand-over-heart gesture would also relate to other-oriented dishonesty, though the latter differs highly from self-centered lying. In Study 1 ( N = 79), we showed that performing a hand-over-heart gesture diminished the tendency to use other-oriented white lies and that the fingers crossed behind one's back gesture was not related to higher dishonesty. We then pre-registered and conducted Study 2 ( N = 88), which was designed following higher methodological standards than Study 1. Contrary, to the findings of Study 1, we found that using the hand-over-heart gesture did not result in refraining from using other-oriented white lies. We discuss the findings of this failed replication indicating the importance of strict methodological guidelines in conducting research and also reflect on relatively small effect sizes related to some findings in embodied cognition.

  5. Working Memory for Linguistic and Non-linguistic Manual Gestures: Evidence, Theory, and Application

    Directory of Open Access Journals (Sweden)

    Mary Rudner

    2018-05-01

    Full Text Available Linguistic manual gestures are the basis of sign languages used by deaf individuals. Working memory and language processing are intimately connected and thus when language is gesture-based, it is important to understand related working memory mechanisms. This article reviews work on working memory for linguistic and non-linguistic manual gestures and discusses theoretical and applied implications. Empirical evidence shows that there are effects of load and stimulus degradation on working memory for manual gestures. These effects are similar to those found for working memory for speech-based language. Further, there are effects of pre-existing linguistic representation that are partially similar across language modalities. But above all, deaf signers score higher than hearing non-signers on an n-back task with sign-based stimuli, irrespective of their semantic and phonological content, but not with non-linguistic manual actions. This pattern may be partially explained by recent findings relating to cross-modal plasticity in deaf individuals. It suggests that in linguistic gesture-based working memory, semantic aspects may outweigh phonological aspects when processing takes place under challenging conditions. The close association between working memory and language development should be taken into account in understanding and alleviating the challenges faced by deaf children growing up with cochlear implants as well as other clinical populations.

  6. Sistem Gesture Accelerometer dengan Metode Fast Dynamic Time Warping (FastDTW

    Directory of Open Access Journals (Sweden)

    Sam Farisa Chaerul Haviana

    2016-01-01

    Full Text Available In the modern environment, the interaction between humans and computers require a more natural form of interaction. Therefore, it is important to be able to build a system that can meet these demands, such as by building a hand gesture recognition system or gesture to create a more natural form of interaction. This study aims to design a smartphone’s accelerometer gesture system as human computer interaction interfaces using FastDTW (Fast Dynamic Time Warping.The result of this study is form of gesture interaction which implemented in a system that can make the process of recognition of the human hand movements based on a smartphone accelerometer which generates a command to run the media player application functions as a case study. FastDTW as the development of Dynamic Time Warping method (DTW is able to compute faster than DTW and have an accuracy approaching DTW. From the test results, FastDTW show a fairly high degree of accuracy reached 86% and showed a better computing speed compared to DTW   Keywords: Human and Computer Interaction, Accelerometer-based gesture, FastDTW, Media player application function

  7. Using Arm and Hand Gestures to Command Robots during Stealth Operations

    Science.gov (United States)

    Stoica, Adrian; Assad, Chris; Wolf, Michael; You, Ki Sung; Pavone, Marco; Huntsberger, Terry; Iwashita, Yumi

    2012-01-01

    Command of support robots by the warfighter requires intuitive interfaces to quickly communicate high degree-of-freedom (DOF) information while leaving the hands unencumbered. Stealth operations rule out voice commands and vision-based gesture interpretation techniques, as they often entail silent operations at night or in other low visibility conditions. Targeted at using bio-signal inputs to set navigation and manipulation goals for the robot (say, simply by pointing), we developed a system based on an electromyography (EMG) "BioSleeve", a high density sensor array for robust, practical signal collection from forearm muscles. The EMG sensor array data is fused with inertial measurement unit (IMU) data. This paper describes the BioSleeve system and presents initial results of decoding robot commands from the EMG and IMU data using a BioSleeve prototype with up to sixteen bipolar surface EMG sensors. The BioSleeve is demonstrated on the recognition of static hand positions (e.g. palm facing front, fingers upwards) and on dynamic gestures (e.g. hand wave). In preliminary experiments, over 90% correct recognition was achieved on five static and nine dynamic gestures. We use the BioSleeve to control a team of five LANdroid robots in individual and group/squad behaviors. We define a gesture composition mechanism that allows the specification of complex robot behaviors with only a small vocabulary of gestures/commands, and we illustrate it with a set of complex orders.

  8. Exploring the role of hand gestures in learning novel phoneme contrasts and vocabulary in a second language

    Science.gov (United States)

    Kelly, Spencer D.; Hirata, Yukari; Manansala, Michael; Huang, Jessica

    2014-01-01

    Co-speech hand gestures are a type of multimodal input that has received relatively little attention in the context of second language learning. The present study explored the role that observing and producing different types of gestures plays in learning novel speech sounds and word meanings in an L2. Naïve English-speakers were taught two components of Japanese—novel phonemic vowel length contrasts and vocabulary items comprised of those contrasts—in one of four different gesture conditions: Syllable Observe, Syllable Produce, Mora Observe, and Mora Produce. Half of the gestures conveyed intuitive information about syllable structure, and the other half, unintuitive information about Japanese mora structure. Within each Syllable and Mora condition, half of the participants only observed the gestures that accompanied speech during training, and the other half also produced the gestures that they observed along with the speech. The main finding was that participants across all four conditions had similar outcomes in two different types of auditory identification tasks and a vocabulary test. The results suggest that hand gestures may not be well suited for learning novel phonetic distinctions at the syllable level within a word, and thus, gesture-speech integration may break down at the lowest levels of language processing and learning. PMID:25071646

  9. Tapping-In-Place: Increasing the Naturalness of Immersive Walking-In-Place Locomotion Through Novel Gestural Input

    DEFF Research Database (Denmark)

    Nilsson, Niels Christian; Serafin, Stefania; Laursen, Morten Havmøller

    2013-01-01

    lower leg backwards, and a tapping gesture where the user in turn lifts each heel without breaking contact with the ground. Visual feedback was delivered through a head-mounted display and auditory feedback was provided by means of a 24-channel surround sound system. The gestures were evaluated in terms...

  10. Point to a Referent, and Say, "What Is This?" Gesture as a Potential Cue to Identify Referents in a Discourse

    Science.gov (United States)

    So, Wing Chee; Lim, Jia Yi

    2012-01-01

    This study explored whether caregivers' gestures followed the discourse-pragmatic principle of information status of referents (given vs. new) and how their children responded to those gestures when identifying referents. Ten Chinese-speaking and eight English-speaking caregivers were videotaped while interacting spontaneously with their children.…

  11. Parents' Translations of Child Gesture Facilitate Word Learning in Children with Autism, Down Syndrome and Typical Development

    Science.gov (United States)

    Dimitrova, Nevena; Özçaliskan, Seyda; Adamson, Lauren B.

    2016-01-01

    Typically-developing (TD) children frequently refer to objects uniquely in gesture. Parents translate these gestures into words, facilitating children's acquisition of these words (Goldin-Meadow et al. in "Dev Sci" 10(6):778-785, 2007). We ask whether this pattern holds for children with autism (AU) and with Down syndrome (DS) who show…

  12. Hearing and seeing meaning in noise: Alpha, beta, and gamma oscillations predict gestural enhancement of degraded speech comprehension.

    Science.gov (United States)

    Drijvers, Linda; Özyürek, Asli; Jensen, Ole

    2018-05-01

    During face-to-face communication, listeners integrate speech with gestures. The semantic information conveyed by iconic gestures (e.g., a drinking gesture) can aid speech comprehension in adverse listening conditions. In this magnetoencephalography (MEG) study, we investigated the spatiotemporal neural oscillatory activity associated with gestural enhancement of degraded speech comprehension. Participants watched videos of an actress uttering clear or degraded speech, accompanied by a gesture or not and completed a cued-recall task after watching every video. When gestures semantically disambiguated degraded speech comprehension, an alpha and beta power suppression and a gamma power increase revealed engagement and active processing in the hand-area of the motor cortex, the extended language network (LIFG/pSTS/STG/MTG), medial temporal lobe, and occipital regions. These observed low- and high-frequency oscillatory modulations in these areas support general unification, integration and lexical access processes during online language comprehension, and simulation of and increased visual attention to manual gestures over time. All individual oscillatory power modulations associated with gestural enhancement of degraded speech comprehension predicted a listener's correct disambiguation of the degraded verb after watching the videos. Our results thus go beyond the previously proposed role of oscillatory dynamics in unimodal degraded speech comprehension and provide first evidence for the role of low- and high-frequency oscillations in predicting the integration of auditory and visual information at a semantic level. © 2018 The Authors Human Brain Mapping Published by Wiley Periodicals, Inc.

  13. Memory Effects of Speech and Gesture Binding: Cortical and Hippocampal Activation in Relation to Subsequent Memory Performance

    Science.gov (United States)

    Straube, Benjamin; Green, Antonia; Weis, Susanne; Chatterjee, Anjan; Tilo, Kircher

    2009-01-01

    In human face-to-face communication, the content of speech is often illustrated by coverbal gestures. Behavioral evidence suggests that gestures provide advantages in the comprehension and memory of speech. Yet, how the human brain integrates abstract auditory and visual information into a common representation is not known. Our study investigates…

  14. Divergence of Verbal Expression and Embodied Knowledge: Evidence from Speech and Gesture in Children with Specific Language Impairment.

    Science.gov (United States)

    Evans, Julia L.; Alibali, Martha W.; McNeil, Nicole M.

    2001-01-01

    Explores the extent to which children with specific language impairment (SLI) with severe phonological working memory deficits express knowledge uniquely in gesture as compared to speech. Using a paradigm in which gesture-speech relationships have been studied extensively, children with SLI and conversation judgment-matched, typically developing…

  15. Automated pattern analysis in gesture research : similarity measuring in 3D motion capture models of communicative action

    NARCIS (Netherlands)

    Schueller, D.; Beecks, C.; Hassani, M.; Hinnell, J.; Brenger, B.; Seidl, T.; Mittelberg, I.

    2017-01-01

    The question of how to model similarity between gestures plays an important role in current studies in the domain of human communication. Most research into recurrent patterns in co-verbal gestures – manual communicative movements emerging spontaneously during conversation – is driven by qualitative

  16. Infant Signs as Intervention? Promoting Symbolic Gestures for Preverbal Children in Low-Income Families Supports Responsive Parent-Child Relationships

    Science.gov (United States)

    Vallotton, Claire D.

    2012-01-01

    Gestures are a natural form of communication between preverbal children and parents which support children's social and language development; however, low-income parents gesture less frequently, disadvantaging their children. In addition to pointing and waving, children are capable of learning many symbolic gestures, known as "infant signs," if…

  17. Human Classification Based on Gestural Motions by Using Components of PCA

    International Nuclear Information System (INIS)

    Aziz, Azri A; Wan, Khairunizam; Za'aba, S K; Shahriman A B; Asyekin H; Zuradzman M R; Adnan, Nazrul H

    2013-01-01

    Lately, a study of human capabilities with the aim to be integrated into machine is the famous topic to be discussed. Moreover, human are bless with special abilities that they can hear, see, sense, speak, think and understand each other. Giving such abilities to machine for improvement of human life is researcher's aim for better quality of life in the future. This research was concentrating on human gesture, specifically arm motions for differencing the individuality which lead to the development of the hand gesture database. We try to differentiate the human physical characteristic based on hand gesture represented by arm trajectories. Subjects are selected from different type of the body sizes, and then acquired data undergo resampling process. The results discuss the classification of human based on arm trajectories by using Principle Component Analysis (PCA)

  18. The higher the farther: distance-specific referential gestures in chimpanzees (Pan troglodytes).

    Science.gov (United States)

    Gonseth, Chloe; Kawakami, Fumito; Ichino, Etsuko; Tomonaga, Masaki

    2017-11-01

    Referential signals, such as manual pointing or deictic words, allow individuals to efficiently locate a specific entity in the environment, using distance-specific linguistic and/or gestural units. To explore the evolutionary prerequisites of such deictic ability, the present study investigates the ability of chimpanzees to adjust their communicative signals to the distance of a referent. A food-request paradigm in which the chimpanzees had to request a close or distant piece of food on a table in the presence/absence of an experimenter was employed. Our main finding concerns the chimpanzees adjusting their requesting behaviours to the distance of the food such that higher manual gestures and larger mouth openings were used to request the distant piece of food. To the best of our knowledge, this is the first study to demonstrate that chimpanzees are able to use distance-specific gestures. © 2017 The Authors.

  19. Non-intrusive gesture recognition system combining with face detection based on Hidden Markov Model

    Science.gov (United States)

    Jin, Jing; Wang, Yuanqing; Xu, Liujing; Cao, Liqun; Han, Lei; Zhou, Biye; Li, Minggao

    2014-11-01

    A non-intrusive gesture recognition human-machine interaction system is proposed in this paper. In order to solve the hand positioning problem which is a difficulty in current algorithms, face detection is used for the pre-processing to narrow the search area and find user's hand quickly and accurately. Hidden Markov Model (HMM) is used for gesture recognition. A certain number of basic gesture units are trained as HMM models. At the same time, an improved 8-direction feature vector is proposed and used to quantify characteristics in order to improve the detection accuracy. The proposed system can be applied in interaction equipments without special training for users, such as household interactive television

  20. Adaptive Kalman Filter Applied to Vision Based Head Gesture Tracking for Playing Video Games

    Directory of Open Access Journals (Sweden)

    Mohammadreza Asghari Oskoei

    2017-11-01

    Full Text Available This paper proposes an adaptive Kalman filter (AKF to improve the performance of a vision-based human machine interface (HMI applied to a video game. The HMI identifies head gestures and decodes them into corresponding commands. Face detection and feature tracking algorithms are used to detect optical flow produced by head gestures. Such approaches often fail due to changes in head posture, occlusion and varying illumination. The adaptive Kalman filter is applied to estimate motion information and reduce the effect of missing frames in a real-time application. Failure in head gesture tracking eventually leads to malfunctioning game control, reducing the scores achieved, so the performance of the proposed vision-based HMI is examined using a game scoring mechanism. The experimental results show that the proposed interface has a good response time, and the adaptive Kalman filter improves the game scores by ten percent.

  1. The Role of Embodiment and Individual Empathy Levels in Gesture Comprehension.

    Science.gov (United States)

    Jospe, Karine; Flöel, Agnes; Lavidor, Michal

    2017-01-01

    Research suggests that the action-observation network is involved in both emotional-embodiment (empathy) and action-embodiment (imitation) mechanisms. Here we tested whether empathy modulates action-embodiment, hypothesizing that restricting imitation abilities will impair performance in a hand gesture comprehension task. Moreover, we hypothesized that empathy levels will modulate the imitation restriction effect. One hundred twenty participants with a range of empathy scores performed gesture comprehension under restricted and unrestricted hand conditions. Empathetic participants performed better under the unrestricted compared to the restricted condition, and compared to the low empathy participants. Remarkably however, the latter showed the exactly opposite pattern and performed better under the restricted condition. This pattern was not found in a facial expression recognition task. The selective interaction of embodiment restriction and empathy suggests that empathy modulates the way people employ embodiment in gesture comprehension. We discuss the potential of embodiment-induced therapy to improve empathetic abilities in individuals with low empathy.

  2. Geometry and Gesture-Based Features from Saccadic Eye-Movement as a Biometric in Radiology

    Energy Technology Data Exchange (ETDEWEB)

    Hammond, Tracy [Texas A& M University, College Station; Tourassi, Georgia [ORNL; Yoon, Hong-Jun [ORNL; Alamudun, Folami T. [ORNL

    2017-07-01

    In this study, we present a novel application of sketch gesture recognition on eye-movement for biometric identification and estimating task expertise. The study was performed for the task of mammographic screening with simultaneous viewing of four coordinated breast views as typically done in clinical practice. Eye-tracking data and diagnostic decisions collected for 100 mammographic cases (25 normal, 25 benign, 50 malignant) and 10 readers (three board certified radiologists and seven radiology residents), formed the corpus for this study. Sketch gesture recognition techniques were employed to extract geometric and gesture-based features from saccadic eye-movements. Our results show that saccadic eye-movement, characterized using sketch-based features, result in more accurate models for predicting individual identity and level of expertise than more traditional eye-tracking features.

  3. Gestures for Picture Archiving and Communication Systems (PACS) operation in the operating room: Is there any standard?

    Science.gov (United States)

    Madapana, Naveen; Gonzalez, Glebys; Rodgers, Richard; Zhang, Lingsong; Wachs, Juan P

    2018-01-01

    Gestural interfaces allow accessing and manipulating Electronic Medical Records (EMR) in hospitals while keeping a complete sterile environment. Particularly, in the Operating Room (OR), these interfaces enable surgeons to browse Picture Archiving and Communication System (PACS) without the need of delegating functions to the surgical staff. Existing gesture based medical interfaces rely on a suboptimal and an arbitrary small set of gestures that are mapped to a few commands available in PACS software. The objective of this work is to discuss a method to determine the most suitable set of gestures based on surgeon's acceptability. To achieve this goal, the paper introduces two key innovations: (a) a novel methodology to incorporate gestures' semantic properties into the agreement analysis, and (b) a new agreement metric to determine the most suitable gesture set for a PACS. Three neurosurgical diagnostic tasks were conducted by nine neurosurgeons. The set of commands and gesture lexicons were determined using a Wizard of Oz paradigm. The gestures were decomposed into a set of 55 semantic properties based on the motion trajectory, orientation and pose of the surgeons' hands and their ground truth values were manually annotated. Finally, a new agreement metric was developed, using the known Jaccard similarity to measure consensus between users over a gesture set. A set of 34 PACS commands were found to be a sufficient number of actions for PACS manipulation. In addition, it was found that there is a level of agreement of 0.29 among the surgeons over the gestures found. Two statistical tests including paired t-test and Mann Whitney Wilcoxon test were conducted between the proposed metric and the traditional agreement metric. It was found that the agreement values computed using the former metric are significantly higher (p operation is higher than the previously reported metric (0.29 vs 0.13). This observation is based on the fact that the agreement focuses on main

  4. A Case Study of MasterMind Chess: Comparing Mouse/Keyboard Interaction with Kinect-Based Gestural Interface

    Directory of Open Access Journals (Sweden)

    Gabriel Alves Mendes Vasiljevic

    2016-01-01

    Full Text Available As gestural interfaces emerged as a new type of user interface, their use has been vastly explored by the entertainment industry to better immerse the player in games. Despite being mainly used in dance and sports games, little use was made of gestural interaction in more slow-paced genres, such as board games. In this work, we present a Kinect-based gestural interface for an online and multiplayer chess game and describe a case study with users with different playing skill levels. Comparing the mouse/keyboard interaction with the gesture-based interaction, the results of the activity were synthesized into lessons learned regarding general usability and design of game control mechanisms. These results could be applied to slow-paced board games like chess. Our findings indicate that gestural interfaces may not be suitable for competitive chess matches, yet it can be fun to play while using them in casual matches.

  5. Neural systems supporting linguistic structure, linguistic experience, and symbolic communication in sign language and gesture.

    Science.gov (United States)

    Newman, Aaron J; Supalla, Ted; Fernandez, Nina; Newport, Elissa L; Bavelier, Daphne

    2015-09-15

    Sign languages used by deaf communities around the world possess the same structural and organizational properties as spoken languages: In particular, they are richly expressive and also tightly grammatically constrained. They therefore offer the opportunity to investigate the extent to which the neural organization for language is modality independent, as well as to identify ways in which modality influences this organization. The fact that sign languages share the visual-manual modality with a nonlinguistic symbolic communicative system-gesture-further allows us to investigate where the boundaries lie between language and symbolic communication more generally. In the present study, we had three goals: to investigate the neural processing of linguistic structure in American Sign Language (using verbs of motion classifier constructions, which may lie at the boundary between language and gesture); to determine whether we could dissociate the brain systems involved in deriving meaning from symbolic communication (including both language and gesture) from those specifically engaged by linguistically structured content (sign language); and to assess whether sign language experience influences the neural systems used for understanding nonlinguistic gesture. The results demonstrated that even sign language constructions that appear on the surface to be similar to gesture are processed within the left-lateralized frontal-temporal network used for spoken languages-supporting claims that these constructions are linguistically structured. Moreover, although nonsigners engage regions involved in human action perception to process communicative, symbolic gestures, signers instead engage parts of the language-processing network-demonstrating an influence of experience on the perception of nonlinguistic stimuli.

  6. Speech-associated gestures, Broca’s area, and the human mirror system

    Science.gov (United States)

    Skipper, Jeremy I.; Goldin-Meadow, Susan; Nusbaum, Howard C.; Small, Steven L

    2009-01-01

    Speech-associated gestures are hand and arm movements that not only convey semantic information to listeners but are themselves actions. Broca’s area has been assumed to play an important role both in semantic retrieval or selection (as part of a language comprehension system) and in action recognition (as part of a “mirror” or “observation–execution matching” system). We asked whether the role that Broca’s area plays in processing speech-associated gestures is consistent with the semantic retrieval/selection account (predicting relatively weak interactions between Broca’s area and other cortical areas because the meaningful information that speech-associated gestures convey reduces semantic ambiguity and thus reduces the need for semantic retrieval/selection) or the action recognition account (predicting strong interactions between Broca’s area and other cortical areas because speech-associated gestures are goal-direct actions that are “mirrored”). We compared the functional connectivity of Broca’s area with other cortical areas when participants listened to stories while watching meaningful speech-associated gestures, speech-irrelevant self-grooming hand movements, or no hand movements. A network analysis of neuroimaging data showed that interactions involving Broca’s area and other cortical areas were weakest when spoken language was accompanied by meaningful speech-associated gestures, and strongest when spoken language was accompanied by self-grooming hand movements or by no hand movements at all. Results are discussed with respect to the role that the human mirror system plays in processing speech-associated movements. PMID:17533001

  7. Hearing and seeing meaning in speech and gesture: insights from brain and behaviour.

    Science.gov (United States)

    Özyürek, Aslı

    2014-09-19

    As we speak, we use not only the arbitrary form-meaning mappings of the speech channel but also motivated form-meaning correspondences, i.e. iconic gestures that accompany speech (e.g. inverted V-shaped hand wiggling across gesture space to demonstrate walking). This article reviews what we know about processing of semantic information from speech and iconic gestures in spoken languages during comprehension of such composite utterances. Several studies have shown that comprehension of iconic gestures involves brain activations known to be involved in semantic processing of speech: i.e. modulation of the electrophysiological recording component N400, which is sensitive to the ease of semantic integration of a word to previous context, and recruitment of the left-lateralized frontal-posterior temporal network (left inferior frontal gyrus (IFG), medial temporal gyrus (MTG) and superior temporal gyrus/sulcus (STG/S)). Furthermore, we integrate the information coming from both channels recruiting brain areas such as left IFG, posterior superior temporal sulcus (STS)/MTG and even motor cortex. Finally, this integration is flexible: the temporal synchrony between the iconic gesture and the speech segment, as well as the perceived communicative intent of the speaker, modulate the integration process. Whether these findings are special to gestures or are shared with actions or other visual accompaniments to speech (e.g. lips) or other visual symbols such as pictures are discussed, as well as the implications for a multimodal view of language. © 2014 The Author(s) Published by the Royal Society. All rights reserved.

  8. The use of open and machine vision technologies for development of gesture recognition intelligent systems

    Science.gov (United States)

    Cherkasov, Kirill V.; Gavrilova, Irina V.; Chernova, Elena V.; Dokolin, Andrey S.

    2018-05-01

    The article is devoted to reflection of separate aspects of intellectual system gesture recognition development. The peculiarity of the system is its intellectual block which completely based on open technologies: OpenCV library and Microsoft Cognitive Toolkit (CNTK) platform. The article presents the rationale for the choice of such set of tools, as well as the functional scheme of the system and the hierarchy of its modules. Experiments have shown that the system correctly recognizes about 85% of images received from sensors. The authors assume that the improvement of the algorithmic block of the system will increase the accuracy of gesture recognition up to 95%.

  9. Enhancing in-Museum Informal Learning by Augmenting Artworks with Gesture Interactions and AIED Paradigms

    DEFF Research Database (Denmark)

    Blanchard, Emmanuel G.; Zanciu, Alin-Nicolae; Mahmoud, Haydar

    2014-01-01

    This paper presents a computer-supported approach for providing ‘enhanced’ discovery learning in informal settings like museums. It is grounded on a combination of gesture-based interactions and artwork-embedded AIED paradigms, and is implemented through a distributed architecture.......This paper presents a computer-supported approach for providing ‘enhanced’ discovery learning in informal settings like museums. It is grounded on a combination of gesture-based interactions and artwork-embedded AIED paradigms, and is implemented through a distributed architecture....

  10. Sequential Classification of Palm Gestures Based on A* Algorithm and MLP Neural Network for Quadrocopter Control

    Directory of Open Access Journals (Sweden)

    Wodziński Marek

    2017-06-01

    Full Text Available This paper presents an alternative approach to the sequential data classification, based on traditional machine learning algorithms (neural networks, principal component analysis, multivariate Gaussian anomaly detector and finding the shortest path in a directed acyclic graph, using A* algorithm with a regression-based heuristic. Palm gestures were used as an example of the sequential data and a quadrocopter was the controlled object. The study includes creation of a conceptual model and practical construction of a system using the GPU to ensure the realtime operation. The results present the classification accuracy of chosen gestures and comparison of the computation time between the CPU- and GPU-based solutions.

  11. Nonverbal Communication : Teachers' Use of Gestures in the Classrooms

    OpenAIRE

    Yogi, Minako; 與儀, 峰奈子

    2002-01-01

    ジェスチャー研究の歴史は長く、古くは17世紀にまで遡る。その多くは発話の代替物としてのジェスチャーがどのような意味を表すのかという研究に費やされてきたと言える。しかし1980年代に入ると研究者の関心はジェスチャーの表す意味体系の構築だけではなく、人間の思考と思考過程を探る手がかりとして、発話とジェスチャーの関係に注目するようになってきた。その代表的研究がMcNeill(1992)である。彼はジェスチャーを発話との関係において、手まね(pantomime)、表象(emblem)、映像的ジェスチャー(iconic gesmre)、暗喩的ジェスチャー(metaphoric gcsture)、指さし(pointing)、拍子(beat)、談話結束的ジェスチャー(cohesive gesture)の7つに分類し深い考察を加えている。本稿ではKendon(1980)とMcNeill(1992)で示された理論的枠組み及び7つの分類に焦点を絞り、データを分析・考察した。McNeillが実験心理学的手法を用いているの対し、本研究ではフィールドワークを伴う社会学・社会言語学的アプローチを用いた。分析...

  12. Eyes, Grip and Gesture as Objective Indicators of Intentions and Attention

    DEFF Research Database (Denmark)

    Mortensen, Ditte Hvas

    This poster abstract presents the first part of a study concerning the use of information about gaze, grip and gesture to create non-command interaction. The experiment reported here seeks to establish the occurrence of patterns in nonverbal communication,  which may be used in an activity aware...

  13. SeleCon: Scalable IoT Device Selection and Control Using Hand Gestures.

    Science.gov (United States)

    Alanwar, Amr; Alzantot, Moustafa; Ho, Bo-Jhang; Martin, Paul; Srivastava, Mani

    2017-04-01

    Although different interaction modalities have been proposed in the field of human-computer interface (HCI), only a few of these techniques could reach the end users because of scalability and usability issues. Given the popularity and the growing number of IoT devices, selecting one out of many devices becomes a hurdle in a typical smarthome environment. Therefore, an easy-to-learn, scalable, and non-intrusive interaction modality has to be explored. In this paper, we propose a pointing approach to interact with devices, as pointing is arguably a natural way for device selection. We introduce SeleCon for device selection and control which uses an ultra-wideband (UWB) equipped smartwatch. To interact with a device in our system, people can point to the device to select it then draw a hand gesture in the air to specify a control action. To this end, SeleCon employs inertial sensors for pointing gesture detection and a UWB transceiver for identifying the selected device from ranging measurements. Furthermore, SeleCon supports an alphabet of gestures that can be used for controlling the selected devices. We performed our experiment in a 9 m -by-10 m lab space with eight deployed devices. The results demonstrate that SeleCon can achieve 84.5% accuracy for device selection and 97% accuracy for hand gesture recognition. We also show that SeleCon is power efficient to sustain daily use by turning off the UWB transceiver, when a user's wrist is stationary.

  14. Releasing the Constraints on Aphasia Therapy: The Positive Impact of Gesture and Multimodality Treatments

    Science.gov (United States)

    Rose, Miranda L.

    2013-01-01

    Purpose: There is a 40-year history of interest in the use of arm and hand gestures in treatments that target the reduction of aphasic linguistic impairment and compensatory methods of communication (Rose, 2006). Arguments for constraining aphasia treatment to the verbal modality have arisen from proponents of constraint-induced aphasia therapy…

  15. Dogs account for body orientation but not visual barriers when responding to pointing gestures

    Science.gov (United States)

    MacLean, Evan L.; Krupenye, Christopher; Hare, Brian

    2014-01-01

    In a series of 4 experiments we investigated whether dogs use information about a human’s visual perspective when responding to pointing gestures. While there is evidence that dogs may know what humans can and cannot see, and that they flexibly use human communicative gestures, it is unknown if they can integrate these two skills. In Experiment 1 we first determined that dogs were capable of using basic information about a human’s body orientation (indicative of her visual perspective) in a point following context. Subjects were familiarized with experimenters who either faced the dog and accurately indicated the location of hidden food, or faced away from the dog and (falsely) indicated the un-baited container. In test trials these cues were pitted against one another and dogs tended to follow the gesture from the individual who faced them while pointing. In Experiments 2–4 the experimenter pointed ambiguously toward two possible locations where food could be hidden. On test trials a visual barrier occluded the pointer’s view of one container, while dogs could always see both containers. We predicted that if dogs could take the pointer’s visual perspective they should search in the only container visible to the pointer. This hypothesis was supported only in Experiment 2. We conclude that while dogs are skilled both at following human gestures, and exploiting information about others’ visual perspectives, they may not integrate these skills in the manner characteristic of human children. PMID:24611643

  16. Finger Angle-Based Hand Gesture Recognition for Smart Infrastructure Using Wearable Wrist-Worn Camera

    Directory of Open Access Journals (Sweden)

    Feiyu Chen

    2018-03-01

    Full Text Available The arising of domestic robots in smart infrastructure has raised demands for intuitive and natural interaction between humans and robots. To address this problem, a wearable wrist-worn camera (WwwCam is proposed in this paper. With the capability of recognizing human hand gestures in real-time, it enables services such as controlling mopping robots, mobile manipulators, or appliances in smart-home scenarios. The recognition is based on finger segmentation and template matching. Distance transformation algorithm is adopted and adapted to robustly segment fingers from the hand. Based on fingers’ angles relative to the wrist, a finger angle prediction algorithm and a template matching metric are proposed. All possible gesture types of the captured image are first predicted, and then evaluated and compared to the template image to achieve the classification. Unlike other template matching methods relying highly on large training set, this scheme possesses high flexibility since it requires only one image as the template, and can classify gestures formed by different combinations of fingers. In the experiment, it successfully recognized ten finger gestures from number zero to nine defined by American Sign Language with an accuracy up to 99.38%. Its performance was further demonstrated by manipulating a robot arm using the implemented algorithms and WwwCam to transport and pile up wooden building blocks.

  17. Fusion of Range and Intensity Information for View Invariant Gesture Recognition

    DEFF Research Database (Denmark)

    Holte, Michael Boelstoft; Moeslund, Thomas B.; Fihl, Preben

    2008-01-01

    This paper presents a system for view invariant gesture recognition. The approach is based on 3D data from a CSEM SwissRanger SR-2 camera. This camera produces both a depth map as well as an intensity image of a scene. Since the two information types are aligned, we can use the intensity image...

  18. An interactive VR system based on full-body tracking and gesture recognition

    Science.gov (United States)

    Zeng, Xia; Sang, Xinzhu; Chen, Duo; Wang, Peng; Guo, Nan; Yan, Binbin; Wang, Kuiru

    2016-10-01

    Most current virtual reality (VR) interactions are realized with the hand-held input device which leads to a low degree of presence. There is other solutions using sensors like Leap Motion to recognize the gestures of users in order to interact in a more natural way, but the navigation in these systems is still a problem, because they fail to map the actual walking to virtual walking only with a partial body of the user represented in the synthetic environment. Therefore, we propose a system in which users can walk around in the virtual environment as a humanoid model, selecting menu items and manipulating with the virtual objects using natural hand gestures. With a Kinect depth camera, the system tracks the joints of the user, mapping them to a full virtual body which follows the move of the tracked user. The movements of the feet can be detected to determine whether the user is in walking state, so that the walking of model in the virtual world can be activated and stopped by means of animation control in Unity engine. This method frees the hands of users comparing to traditional navigation way using hand-held device. We use the point cloud data getting from Kinect depth camera to recognize the gestures of users, such as swiping, pressing and manipulating virtual objects. Combining the full body tracking and gestures recognition using Kinect, we achieve our interactive VR system in Unity engine with a high degree of presence.

  19. Delayed Stimulus-Specific Improvements in Discourse Following Anomia Treatment Using an Intentional Gesture

    Science.gov (United States)

    Altmann, Lori J. P.; Hazamy, Audrey A.; Carvajal, Pamela J.; Benjamin, Michelle; Rosenbek, John C.; Crosson, Bruce

    2014-01-01

    Purpose: In this study, the authors assessed how the addition of intentional left-hand gestures to an intensive treatment for anomia affects 2 types of discourse: picture description and responses to open-ended questions. Method: Fourteen people with aphasia completed treatment for anomia comprising 30 treatment sessions over 3 weeks. Seven…

  20. Music as a mnemonic to learn gesture sequences in normal aging and Alzheimer’s disease

    Directory of Open Access Journals (Sweden)

    Aline eMoussard

    2014-05-01

    Full Text Available Strong links between music and motor functions suggest that music could represent an interesting aid for motor learning. The present study aims for the first time to test the potential of music to assist in the learning of sequences of gestures in normal and pathological aging. Participants with mild Alzheimer's disease (AD and healthy older adults (Controls learned sequences of meaningless gestures that were either accompanied by music or a metronome. We also manipulated the learning procedure such that participants had to imitate the gestures to-be-memorized in synchrony with the experimenter or after the experimenter during encoding. Results show different patterns of performance for the two groups. Overall, musical accompaniment had no impact on the Controls' performance, but improved those of AD participants. Conversely, synchronization of gestures during learning helped Controls but seemed to interfere with retention in AD. We discuss these findings regarding their relevance for a better understanding of auditory-motor memory, and we propose recommendations to maximize the mnemonic effect of music for motor sequence learning for dementia care.

  1. Different Social Motives in the Gestural Communication of Chimpanzees and Human Children

    Science.gov (United States)

    Bullinger, Anke F.; Zimmermann, Felizitas; Kaminski, Juliane; Tomasello, Michael

    2011-01-01

    Both chimpanzees and human infants use the pointing gesture with human adults, but it is not clear if they are doing so for the same social motives. In two studies, we presented chimpanzees and human 25-month-olds with the opportunity to point for a hidden tool (in the presence of a non-functional distractor). In one condition it was clear that…

  2. Gestures in Prelinguistic Turkish Children with Autism, Down Syndrome, and Typically Developing Children

    Science.gov (United States)

    Toret, Gokhan; Acarlar, Funda

    2011-01-01

    The purpose of this study was to examine gesture use in Turkish children with autism, Down syndrome, and typically developing children. Participants included 30 children in three groups: Ten children with Down syndrome, ten children with autism between 24-60 months of age, and ten typically developing children between 12-18 months of age.…

  3. Captive gorillas' manual laterality: The impact of gestures, manipulators and interaction specificity.

    Science.gov (United States)

    Prieur, Jacques; Barbu, Stéphanie; Blois-Heulin, Catherine; Pika, Simone

    2017-12-01

    Relationships between humans' manual laterality in non-communicative and communicative functions are still poorly understood. Recently, studies showed that chimpanzees' manual laterality is influenced by functional, interactional and individual factors and their mutual intertwinement. However, what about manual laterality in species living in stable social groups? We tackled this question by studying three groups of captive gorillas (N=35) and analysed their most frequent manual signals: three manipulators and 16 gesture types. Our multifactorial investigation showed that conspecific-directed gestures were overall more right-lateralized than conspecific-directed manipulators. Furthermore, it revealed a difference between conspecific- and human-directed gestural laterality for signallers living in one of the study groups. Our results support the hypothesis that gestural laterality is a relevant marker of language left-brain specialisation. We suggest that components of communication and of manipulation (not only of an object but also of a conspecific) do not share the same lateralised cerebral system in some primate species. Copyright © 2017 Elsevier Inc. All rights reserved.

  4. Visual context enhanced. The joint contribution of iconic gestures and visible speech to degraded speech comprehension.

    NARCIS (Netherlands)

    Drijvers, L.; Özyürek, A.

    2017-01-01

    Purpose: This study investigated whether and to what extent iconic co-speech gestures contribute to information from visible speech to enhance degraded speech comprehension at different levels of noise-vocoding. Previous studies of the contributions of these 2 visual articulators to speech

  5. On Semiotics and Jumping Frogs: The Role of Gesture in the Teaching of Subtraction

    Science.gov (United States)

    Farrugia, Marie Therese

    2017-01-01

    In this article, I describe a research/teaching experience I undertook with a class of 5-year-old children in Malta. The topic was subtraction on the number line. I interpret the teaching/learning process through a semiotic perspective. In particular, I highlight the role played by the gesture of forming "frog jumps" on the number line.…

  6. On Becoming Subtle: Age and Social Rank Changes in the Use of a Facial Gesture

    Science.gov (United States)

    Zivin, Gail

    1977-01-01

    This report uses data from videotaped naturalistic interactions and peers' hierarchy rankings of 23 4- to 5-year-olds and 26 7- to 10-year-olds to examine changes in children's use of facial gestures which resemble the dominance-related threat stares of primates. (Author/JMB)

  7. Does comprehension of symbolic gestures and corresponding-in-meaning words make use of motor simulation?

    Science.gov (United States)

    Campione, Giovanna Cristina; De Stefani, Elisa; Innocenti, Alessandro; De Marco, Doriana; Gough, Patricia M; Buccino, Giovanni; Gentilucci, Maurizio

    2014-02-01

    The present study aimed at determining whether or not the comprehension of symbolic gestures, and corresponding-in-meaning words, makes use of cortical circuits involved in movement execution control. Participants were presented with videos of an actress producing meaningful or meaningless gestures, pronouncing corresponding-in-meaning words or pseudo-words; they were required to judge whether the signal was meaningful or meaningless. Single pulse TMS was applied to forearm primary motor cortex area 150-200 ms after the point when the stimulus meaning could be understood. MEPs were significantly greater when processing meaningless signals as compared to a baseline condition presenting a still-and-silent actress. In contrast, this was not the case for meaningful signals whose motor activation did not differ from that for the baseline stimulus. MEPs were significantly greater for meaningless than meaningful signals and no significant difference was found between gesture and speech. On the basis of these results, we hypothesized that the observation-of/listening-to meaningless signals recruits motor areas. In contrast, this did not occur when the signals were meaningful. Overall, the data suggest that the processes related to comprehension of symbolic gestures and communicative words do not involve primary motor area and probably use brain areas involved in semantics. Copyright © 2013 Elsevier B.V. All rights reserved.

  8. Cognitive Disorders of Infantile Autism: A Study Employing the WISC, Spatial Relationship Conceptualization, and Gesture Imitations.

    Science.gov (United States)

    Ohta, Masataka

    1987-01-01

    Analysis of the conceptions of spatial relations, size comparisons, and gesture imitations of 16 autistic children (ages 6-14 years) with IQs over 70 indicated an inability to acquire concepts of size comparison and spatial relationships through verbal instructions, suggesting a cognitive deficit of impaired symbolic-representational functioning.…

  9. Compensation for Coarticulation: Disentangling Auditory and Gestural Theories of Perception of Coarticulatory Effects in Speech

    Science.gov (United States)

    Viswanathan, Navin; Magnuson, James S.; Fowler, Carol A.

    2010-01-01

    According to one approach to speech perception, listeners perceive speech by applying general pattern matching mechanisms to the acoustic signal (e.g., Diehl, Lotto, & Holt, 2004). An alternative is that listeners perceive the phonetic gestures that structured the acoustic signal (e.g., Fowler, 1986). The two accounts have offered different…

  10. Effects of Integrated Physical Exercises and Gestures on Preschool Children's Foreign Language Vocabulary Learning

    Science.gov (United States)

    Mavilidi, Myrto-Foteini; Okely, Anthony D.; Chandler, Paul; Cliff, Dylan P.; Paas, Fred

    2015-01-01

    Research suggests that integrating human movement into a cognitive learning task can be effective for learning due to its cognitive and physiological effects. In this study, the learning effects of enacting words through whole-body movements (i.e., physical exercise) and part-body movements (i.e., gestures) were investigated in a foreign language…

  11. Hand Gesture Modeling and Recognition for Human and Robot Interactive Assembly Using Hidden Markov Models

    Directory of Open Access Journals (Sweden)

    Fei Chen

    2015-04-01

    Full Text Available Gesture recognition is essential for human and robot collaboration. Within an industrial hybrid assembly cell, the performance of such a system significantly affects the safety of human workers. This work presents an approach to recognizing hand gestures accurately during an assembly task while in collaboration with a robot co-worker. We have designed and developed a sensor system for measuring natural human-robot interactions. The position and rotation information of a human worker's hands and fingertips are tracked in 3D space while completing a task. A modified chain-code method is proposed to describe the motion trajectory of the measured hands and fingertips. The Hidden Markov Model (HMM method is adopted to recognize patterns via data streams and identify workers' gesture patterns and assembly intentions. The effectiveness of the proposed system is verified by experimental results. The outcome demonstrates that the proposed system is able to automatically segment the data streams and recognize the gesture patterns thus represented with a reasonable accuracy ratio.

  12. Visual Context Enhanced: The Joint Contribution of Iconic Gestures and Visible Speech to Degraded Speech Comprehension

    Science.gov (United States)

    Drijvers, Linda; Ozyurek, Asli

    2017-01-01

    Purpose: This study investigated whether and to what extent iconic co-speech gestures contribute to information from visible speech to enhance degraded speech comprehension at different levels of noise-vocoding. Previous studies of the contributions of these 2 visual articulators to speech comprehension have only been performed separately. Method:…

  13. Image as Gesture: Notes on Aernout Mik's Communitas and the Modern Political Film

    NARCIS (Netherlands)

    Pisters, P.

    2015-01-01

    Walking through Aernout Mik’s exhibition Communitas, dwelling in the encounter with the people and situations in front of and on the screens, one is struck by the cinematic and political qualities of the gestures in these powerful video installations. In this article I will develop some

  14. Co-speech gestures influence neural activity in brain regions associated with processing semantic information.

    Science.gov (United States)

    Dick, Anthony Steven; Goldin-Meadow, Susan; Hasson, Uri; Skipper, Jeremy I; Small, Steven L

    2009-11-01

    Everyday communication is accompanied by visual information from several sources, including co-speech gestures, which provide semantic information listeners use to help disambiguate the speaker's message. Using fMRI, we examined how gestures influence neural activity in brain regions associated with processing semantic information. The BOLD response was recorded while participants listened to stories under three audiovisual conditions and one auditory-only (speech alone) condition. In the first audiovisual condition, the storyteller produced gestures that naturally accompany speech. In the second, the storyteller made semantically unrelated hand movements. In the third, the storyteller kept her hands still. In addition to inferior parietal and posterior superior and middle temporal regions, bilateral posterior superior temporal sulcus and left anterior inferior frontal gyrus responded more strongly to speech when it was further accompanied by gesture, regardless of the semantic relation to speech. However, the right inferior frontal gyrus was sensitive to the semantic import of the hand movements, demonstrating more activity when hand movements were semantically unrelated to the accompanying speech. These findings show that perceiving hand movements during speech modulates the distributed pattern of neural activation involved in both biological motion perception and discourse comprehension, suggesting listeners attempt to find meaning, not only in the words speakers produce, but also in the hand movements that accompany speech.

  15. The Function of Gesture in Lexically Focused L2 Instructional Conversations

    Science.gov (United States)

    Smotrova, Tetyana; Lantolf, James P.

    2013-01-01

    The purpose of the present study is to investigate the mediational function of the gesture-speech interface in the instructional conversation that emerged as teachers attempted to explain the meaning of English words to their students in two EFL classrooms in the Ukraine. Its analytical framework is provided by Vygotsky's sociocultural psychology…

  16. Gesture-Based Robot Control with Variable Autonomy from the JPL Biosleeve

    Science.gov (United States)

    Wolf, Michael T.; Assad, Christopher; Vernacchia, Matthew T.; Fromm, Joshua; Jethani, Henna L.

    2013-01-01

    This paper presents a new gesture-based human interface for natural robot control. Detailed activity of the user's hand and arm is acquired via a novel device, called the BioSleeve, which packages dry-contact surface electromyography (EMG) and an inertial measurement unit (IMU) into a sleeve worn on the forearm. The BioSleeve's accompanying algorithms can reliably decode as many as sixteen discrete hand gestures and estimate the continuous orientation of the forearm. These gestures and positions are mapped to robot commands that, to varying degrees, integrate with the robot's perception of its environment and its ability to complete tasks autonomously. This flexible approach enables, for example, supervisory point-to-goal commands, virtual joystick for guarded teleoperation, and high degree of freedom mimicked manipulation, all from a single device. The BioSleeve is meant for portable field use; unlike other gesture recognition systems, use of the BioSleeve for robot control is invariant to lighting conditions, occlusions, and the human-robot spatial relationship and does not encumber the user's hands. The BioSleeve control approach has been implemented on three robot types, and we present proof-of-principle demonstrations with mobile ground robots, manipulation robots, and prosthetic hands.

  17. Effects of imitating gestures during encoding or during retrieval of novel verbs on children's test performance.

    Science.gov (United States)

    de Nooijer, Jacqueline A; van Gog, Tamara; Paas, Fred; Zwaan, Rolf A

    2013-09-01

    Research has shown that observing and imitating gestures can foster word learning and that imitation might be more beneficial than observation, which is in line with theories of Embodied Cognition. This study investigated when imitation of gestures is most effective, using a 2×2×2×3 mixed design with between-subjects factors Imitation during Encoding (IE; Yes/No) and Imitation during Retrieval (IR; Yes/No), and within-subjects factors Time of Testing (Immediate/Delayed) and Verb Type (Object manipulation/Locomotion/Abstract). Primary school children (N=115) learned 15 novel verbs (five of each type). They were provided with a verbal definition and a video of the gesture. Depending on assigned condition, they additionally received no imitation instructions, instructions to imitate the gesture immediately (i.e., during encoding; IE), instructions to imitate (from memory) during the first posttest (i.e., during retrieval; IR), or both (IE-IR). Based on the literature, all three imitation conditions could be predicted to be more effective than no imitation. On an immediate and delayed posttest, only the object-manipulation verbs were differentially affected by instructional method, with IE and IR being more effective than no imitation on the immediate test; IE-IR and no imitation did not differ significantly. After a one week delay, only IR was more effective than no imitation, suggesting that imitation during retrieval is most effective for learning object-manipulation words. Copyright © 2013 Elsevier B.V. All rights reserved.

  18. Pointing Gestures as a Cognitive Tool in Young Children: Experimental Evidence

    Science.gov (United States)

    Delgado, Begona; Gomez, Juan Carlos; Sarria, Encarnacion

    2011-01-01

    This article explores the possible cognitive function associated with pointing gestures from a Vygotskian perspective. In Study 1, 39 children who were 2-4 years of age were observed in a solitary condition while solving a mnemonic task with or without an explicit memory demand. A discriminant analysis showed that children used noncommunicative…

  19. The Role of Speech-Gesture Congruency and Delay in Remembering Action Events

    Science.gov (United States)

    Galati, Alexia; Samuel, Arthur G.

    2011-01-01

    When watching others describe events, does information from their speech and gestures affect our memory representations for the gist and surface form of the described events? Does our reliance on these memory representations change over time? Forty participants watched videos of stories narrated by an actor. Each story included three target events…

  20. Using the Hand to Choreograph Instruction: On the Functional Role of Gesture in Definition Talk

    Science.gov (United States)

    Belhiah, Hassan

    2013-01-01

    This article examines the coordination of speech and gesture in teachers' definition talk, that is, vocabulary explanations addressed to language learners. By analyzing one ESL teacher's spoken definitions, the study demonstrates in the details of the unfolding talk how a teacher crafts and choreographs his definitions moment by moment, while…