The astronaut buckles up and slides deeper into his pilot's chair; he’s ready. Then the rocket takes off. Earth slowly disappears while Mars gets bigger and Jupiter appears in the distance. The journey begins ... What sounds like a space adventure – and looks like one – actually takes place on the university campus in Golm in a small, carefully constructed prototype lab. At the Laboratory for Oral Language Acquisition (LOLA), linguists want to use innovative methods to delve deeper into the secrets of language acquisition. They are especially interested in how children learn to master their language – a challenge not only from a scientific perspective.
“Doing scientific research on speech development in children is quite challenging,” says Dr. Aude Noiray, principal investigator at LOLA. “When you want to conduct experiments with children, it is important to stimulate their interest, to take them on a journey.” This is exactly what researchers at LOLA do. Without hesitation, they redesigned the technical equipment at the lab to make it look like the cockpit of a spaceship and they embedded the experimental procedure into an interstellar journey. The young participants who come here hardly notice that they are traveling in the name of science. At the beginning of the journey, they select a little puppet character to bring with them on their interplanetary adventures...
Investigating language from where it is produced – in the mouth
It is not only the child-friendly setting at LOLA that is innovative but also the techniques employed and the expertise of the team. The laboratory has combined audio and video analysis with ultrasound imaging techniques rarely used in developmental psycholinguistic research to create a sophisticated platform for speech production analysis. Ultrasound imaging represents the highlight of the platform; it allows for recording the articulatory activity in an area that cannot be examined with standard video equipment: the inner part of the mouth (the ‘oral cavity’). This setup is used to investigate the complex interactions of the tongue and the lips during speech production in adults and children. These are mostly unexplored in children. “Most of what we know about articulation in children comes from the acoustic analysis of sound recordings,” explains Noiray. “In the past, linguists would record a few children (sometimes their own child) and spend many hours transcribing these recordings and analyzing acoustic data to identify speech patterns. “Nowadays there are new technical possibilities, and we want to use them. It is becoming increasingly obvious that it is not enough to examine only the acoustic aspects of speech production. This is the tip of the iceberg. We also need to look below the surface or into the mouth – and explore the movement of the articulators, especially those of the tongue.”
Speech sounds are mainly produced in the mouth via the interaction of mobile articulators (e.g., the lips, the tongue, the jaw) in relation to rather immobile parts (e.g., the palate) that provide places of articulation. The tongue plays a major role in spoken language (development). The tongue apex, blade, dorsum, and root characterize important anatomical and functional parts involved in the formation of most consonants and all vowels in the world’s languages. Up to now, their movements have been difficult to investigate, because there was not any method for observing the oral cavity non-invasively, i.e. from the outside. The “imaging method” used with the ultrasound device offers new possibilities. “We are finally able to make the invisible processes of speech production visible,” says Noiray with excitement. “I think ultrasound imaging has enormous potential in linguistics for fundamental and applied research, particularly in the clinical field but also for studying first and second language acquisition.”
The young linguist gained experience with ultrasound technology as a postdoc fellow at Haskins Laboratories (affiliated with Yale University) in the USA. She first studied speech production in adults from various languages. Over the past year, she has worked intensively on optimizing this technology for research on language acquisition in children. Capitalizing on her experience with preschoolers and school-aged children, she now aims to target an even more challenging population: infants and investigate the emergence of speech sounds (phonemes) and syllables in the first year of life.
Optimizing ultrasound technique to study children’s speech
Ultrasound imaging is a standard method used in the medical field to visualize various parts of the body (e.g., muscles, nerves, the fetus in pregnant women). At LOLA, Dr. Noiray and her team optimized the technique to be used for the study of speech production in children. Since no instrument or platform existed for imaging the speech apparatus with ultrasound, Noiray designed one and constructed the prototype. “I like designing, building, and testing things. I think it is an exciting part of our work as scientists. Science is a domain in which we can be very creative and use our imagination to develop innovative objects, much like art,” she says. “I also think the methodological aspects of our research are crucial for the success of a project – they need to be considered from the formulation of the initial research question to the interpretation of the results. I wanted to develop an experimental platform that would be fun for kids and at the same time allow us to collect solid data. At the beginning of the project, we brought the team together to brainstorm. We tried to find innovative solutions to the experimental challenges we had to face and that is how we designed our SOLLAR platform (Sonographic and Optical Linguo-Labial Articulation Recording system).” The most sophisticated piece of the SOLLAR platform is an ultrasound transducer that emits ultrasound waves and receives their echo. One of the main challenges is that the transducer must be placed near the tongue without restraining the participant’s lower jaw to allow for natural speech production. Since Noiray did not want to fix the probe to the jaw to avoid perturbing speech, she developed a probe holder onto which the ultrasound transducer is mounted. “That was a bit tricky,” says Dr. Noiray. "The probe holder has to be flexible enough to move with the natural jaw movement while speaking. Together with a mechanical designer from Yale School of Medicine, we played around with ideas for quite some time. We took measurements, designed plans and came out with a prototype that we tested and improved until we got convincing results. I learned a lot during this collaborative work, not only about design and technical specifics, but also about the fundamental links between empirical and theoretical research. How important it is to have all of the details of the research questions, goals and constraints crystal clear in order to come up with an appropriate tool. In the end, we designed a customized probe holder mounted in a customized pedestal that fit the specifics of our ultrasound device and provided a good trade-off between image quality and the constraints associated with working with children.” The ultrasound transducer is placed in the probe holder below the chin of participants (adults or children) in such a way that of the transducer moves up and down together with the jaw when speaking. It is thus always directly below the tongue and records the best possible image of the movement, which is then displayed on a computer screen, as is familiar to us from medical studies. “The whole apparatus is integrated in a spaceship with a video camera and microphone so it is entertaining for children and for us too!”.
While ultrasound devices used in hospitals and clinics, are usually quite large, the LOLA device is comparatively small and portable. For a good reason: “One day we want to use the device in fieldwork,” says Noiray, “because there are many potential applications: going to various countries to draw cross-linguistic comparisons, or investigating endangered languages in communities that live in remote areas. During my postdoc, I once went to British Columbia to record speakers of Tahltan for a documentation project in collaboration with Professor Whalen (CUNY, USA) and Professor Shaw (UBC, Canada). This type of work is very crucial to understand the variety of languages and dialects spoken in the world. But it can often only be done at the places where they are spoken. In these situations, a portable ultrasound imaging system is required.”
Ultrasound makes it possible to see how language is produced
In addition to optimize ultrasound technique to image the tongue of children, Dr. Noiray and her team developed a platform that integrates standard audio and video analyses of the main articulators used for speech. “In our experiments, we record the acoustic speech signal with a microphone, the movements of the lips and the jaw with a video camera and those of the tongue with ultrasound all simultaneously,” explains the researcher. The data are then analyzed with custom-made software developed at LOLA. The program filters out unusable datasets semi-automatically. If, for example, the ultrasound images are blurred because young participants turned their heads, this can be verified based on video recordings. The system also tracks movement of the head and the jaw. For this, blue dots are placed on the participants’ faces before the experiment. Software developed at the lab detects the colored dots and track their movement when a participant speaks or moves his head.
It takes at least a decade for children to speak their native language fluently the way adults do. Noiray describes it as a dance. “Dancing is the dynamic coordination of the body. When we dance, we coordinate various limbs in space but also over time to create specific movements and movements are organized in a continuous flow. The same applies to speech. To produce a sound (e.g., “i”), we coordinate the lips and the tongue to produce specific movements. To produce a syllable or a word, articulatory movements overlap with one another; we say they are coarticulated. An interesting aspect is that the temporal and spatial organization of those movements varies across languages. Changing such dynamics changes the sounds produced. When learning to speak their native language, children learn to control their speech articulators to produce specific movement coordinations that take into account the regularities of their native language.” In their experiments, whether as space travel or not, the researchers ask children to articulate target words. The combinatorial approach used at the lab allows them to investigate the dynamics of speech. In other words, they examine the coordination of the jaw, the lips and the tongue involved in the production of sounds, syllables and words in adults and track how those develop in children. Some analyses address questions about how speech motor control develops with age and how this interacts with the development of the lexicon and the phonological system of the native language; how variability in word production decreases with practice; how syllables and words are planned in the brain before being produced aloud or how children develop anticipatory coarticulation. “In a syllable such as “gi”, the lips can anticipate the configuration for the vowel “i” during the production of the consonant “g”, while the tongue is positioned in the back of the oral cavity for the consonant”, Noiray says. “It gets even more exciting when words and sentences are created from the combination of these individual sounds, because the specific sequence intended by the child influences the way articulators are coordinated over time. This is an important mechanism we want to investigate.” In a current project, the LOLA team is investigating the interplay of consonants (C1, C2) and vowels (V1, V2) in word pairs of the form “a C1V1C2V2”, for example the pseudo-word “eine bode”.
Elina Rubertus is even going a step further in her PhD project, which is part of the study. She investigates whether coarticulation mechanisms also extend beyond word boundaries. She examines the final sound (phoneme) of the first word, (e.g., the last “e” in the German indefinite article “eine”). This sound, a so-called schwa, is formed by a neutral tongue position, so to speak, that often corresponds to the tongue’s resting position. “Using ultrasound imaging, I want to find out whether this neutral tongue position changes due to the following vowel,” she explains, i.e. if the schwa before the word ‘bide’ has a different tongue position from its position before, for example, the word ‘bade’. For the production of an “i” the tongue has a relatively high position in the oral cavity while it is low for the production of “a.” If the schwa already demonstrates such differences in tongue position, we can assume that the vowel is already prepared during the previous syllable to allow fluent speech production.” The researchers will test for similar anticipatory processes with the lips to examine both lingual – tongue-related – coarticulation patterns as well as labial – lip-related – ones. “The comparison of various age groups will be very interesting,” says Rubertus. “Will children exhibit different patterns from adults? Can we observe a gradual development in the various age groups of our young participants?”
Focus on language acquisition in children
The experiments carried out at LOLA focus on spoken language development in children and how this goes hand in hand with lexical and phonological development. Their main interests regard how children become proficient speakers in their native language, and what stands in their way. “A main goal in speaking is to produce specific sounds,” summarizes Noiray. “You can achieve this in various ways. This is one reason why children initially have very different speech production strategies and must learn to reduce them to use the ones that are mostly used by adults in their native language.” Understanding how language develops in typically developing children could ultimately help diagnose and treat children with speech and/or language disorders.
In a large-scale study funded by the German Research Association, the LOLA team is investigating children aged 3-7 years compared to a control group of adults. Fifteen children per age group are invited to participate in the space travel. In a second part of the study, cognitive and language-related abilities are assessed using standard developmental assessment batteries. Comparing various age groups provide crucial insights into how language develops over time, exposure and practice with the native language. The youngest cohort presents particular challenges. “We were not sure we would succeed in examining three year olds,” says Noiray. “At 3 years of age, they are quite shy and cannot stay focused for long. Also, it is not easy to keep them interested in the tasks. Students involved in the project have to be good storytellers! Again, we brainstormed to find interesting scenarios and we optimized the experimental protocol – we practiced and it worked!”
The coarticulation study is designed to provide normative data in a field that lacks direct articulatory insight on language development. In German as well as in most languages, there is not any comparable database yet. Noiray initially planed on working with children with speech or/and language difficulties or with delayed language development. “But we cannot do that without first understanding how language develops when there are no problems.” Nonetheless, Noiray already has new plans. While the cross-sectional study on coarticulation is still in progress, a new project on children with a reading disability (often termed ‘developmental dyslexia’) has already started. LOLA cooperates with the research group of the Marie Curie Innovative Training Network “PredictAble,” led by Barbara Höhle, a professor of psycholinguistics in the Department of Linguistics. Together with colleagues from European labs and universities, they are researching whether it is possible to identify language disorders in the early language development of children in bilingual Europe. Within this collaborative grant, LOLA team investigates the relations between spoken language and reading acquisition in typically developing children compared to children at risk or diagnosed with developmental dyslexia. Noiray leads this project jointly with Dr. Ken Pugh (Haskins Laboratories, Yale University, USA).
For Noiray, this is only the beginning of the adventure. She already has ideas for other projects in mind. “And I have a team of clever, rigorous, and very enthusiastic students by my side,” she adds with a smile. LOLA is gaining momentum; the journey can begin.
The Project
Methodology for investigating the development of coarticulation in German children (DFG Grant No: 1098)
Participating: Dr. Aude Noiray (Principal Investigator), Jan Ries (Lab Manager), Helene Killmer (Speech Language therapist). Students involved: Dzhuma Abakarova, Elina Rubertus, Michelle Golchert (all at University of Potsdam). Partners: Dr. Mark Tiede (Haskins Laboratories); Dr. Khalil Iskarous (USC, USA)
Duration: Late 2014–late 2017
Funding: German Research Association (Deutsche Forschungsgemeinschaft, DFG)
http://www.uni-potsdam.de/lola/index.html
PredictAble: Understanding and predicting developmental language abilities and disorders in multilingual Europe (Marie Skłodowska-Curie Actions Innovative Training Network (ITN), No: 641858)
Participants: Prof. Höhle (Principal Investigator), Prof. Wartenburger (investigator), Dr. Noiray (investigator). Students involved: Stefanos Tserkezis, Stella Krüger, Gwendolyn Kaiser (all at University of Potsdam). Partner: Prof. Pugh (Haskins Laboratories).
Collaboration with other European partners (see website). https://predictable.eu/index.php/en/
Duration: Late 2015–late 2019
Funding: Marie Skłodowska-Curie
The Researcher
Dr. Aude Noiray studied English, Language, Letters and Foreign Civilization and obtained a master’s degree in Language Sciences at the Université Stendhal, Grenoble (France). During her PhD at GIPSA lab – a CNRS laboratory in Grenoble – she visited the Laboratoire de Phonétique in Montréal and McGill University (Canada) to conduct cross-linguistic experiments with Pr. Lucie Ménard. After earning her doctoral degree, she worked as a postdoc at Haskins Laboratories (affiliated with Yale University, USA). Since 2012, she has been teaching and conducting research at the University of Potsdam. She is principal investigator of a DFG grant project and co-investigator of a Marie Curie ITN grant project. She created LOLA lab (Laboratory for Oral language Acquisition) in February 2015.
Universität Potsdam
Department für Linguistik
Karl-Liebknecht-Straße 24–25
14476 Potsdam
Email: anoirayuuni-potsdampde
Elina Rubertus completed her bachelor’s degree in General Theoretical Linguistics and obtained a master’s degree in Experimental and Clinical Linguistics (IECL) at the University of Potsdam. Since October 2015, she teaches in the Department of Linguistics while conducting her PhD project at LOLA.
Email: rubertusuuni-potsdampde
Text: Matthias Zimmermann
Translation: Susanne Voigt
Published online by: Agnetha Lang
Kontakt zur Online-Redaktion: onlineredaktionuuni-potsdampde