Lauren Klein, a computer science Ph.D. candidate in the USC Viterbi School of Engineering, has long been interested in tackling healthcare problems. Her latest approach: robot toys.
“I strongly believe that human-robot interaction is a research topic that is promising for the future of healthcare,” said Klein, a member of the USC Interaction Lab.
Last fall, Klein’s research team won an award in the “CS for Social Good” white paper competition sponsored by the Computing Community Consortium and Schmidt Futures.
Their paper, “A Computational Approach to Earlier Detection and Intervention for Infants with Developmental Disabilities,” received a $7,500 grant to support future research.
That team includes her PhD advisor, Maja Matarić; Chan Soon-Shiong Chair, Distinguished Professor of Computer Science, Neuroscience and Pediatrics, and Interim Vice President of Research at USC; Beth Smith, an assistant research professor in the USC Division of Biokinesiology and Physical Therapy; and Fei Sha, associate professor of computer science and biological sciences and Zohrab A. Kaprielian Fellow in Engineering.
Together, they are researching ways that robots could make a difference in the lives of children with developmental disorders.
Their work aims to help earlier diagnose children with conditions ranging from learning disabilities to Autism Spectrum Disorder. Earlier diagnoses, experts say, allow for earlier interventions and better outcomes.
In their paper, Klein, Matarić, Smith and Sha propose using a robot toy to interact with an infant to encourage certain behaviors.
These behaviors are known as exploratory motor movements — important infant behaviors such as reaching, touching, grasping and kicking that help them learn to control their bodies and interact with their surroundings.
Exploratory movements are believed to be important for healthy cognitive, motor and social development.
“Based on this, we can look for infants who make decreased exploratory movements and design and evaluate interactions that could increase these movements,” Klein said.
“These interactions are aimed toward children at risk for developmental disabilities, though we anticipate it may be supportive for typically developing infants as well due to the importance of early exploratory motor movements.”
The team’s past research placed an infant in a chair across from a humanoid Nao robot, which interacted with infants by responding to movement. Whenever the infant kicked their leg, the Nao robot would also kick one of its legs.
Twelve infants between the ages of 6 and 8 months participated in this first study, which has been published in a paper titled “Socially Assistive Infant-Robot Interaction: Using Robots to Encourage Infant Leg-Motion.”
The study observed that once babies made the connection between their own movement and the movement of the robot, they increased their kicking.
Babies at risk for developmental disorders, such as ADHD or ASD, may perform differently in this paradigm.
They may demonstrate difficulty learning the connection between their movement and the robot response, supporting its use in early detection.
Alternatively, they might respond very well to the robot, supporting its use as an early intervention tool.
“Our preliminary study gave us a lot of insight, which helped to inform the research proposed in the white paper,” Klein said.
Klein and her team’s white paper outlined how they plan to build upon their current work with the Nao robots and pursue future research, one possible avenue being by exploring the use of Sphero robots in encouraging infant motor movement.
In past studies, the team used the Nao robot platform as an effective socially assistive robot to both provide contingent rewards and allow researchers to evaluate whether infants would imitate the robot, but they had some limitations that Klein’s team hope Sphero robots can address.
Nao robots cost thousands of dollars, while Sphero robots are much more affordable, at about $150 per robot.
Additionally, Nao robots can only move in certain directions, which limits the range of motion that can be encouraged, while Sphero robots can safely roll around the baby, encouraging a wider range of motion while simultaneously engaging the infant’s attention more effectively.
Their paper outlines their plan to record interactions between the infant and the robot on video, use software to characterize the movements of the infant’s limbs and head, and sort these movements to classify if the infant is at risk for a developmental disability.
Infant interacts with Nao Robot. Image is credited to 2019 IEEE INTERNATIONAL SYMPOSIUM ON ROBOT AND HUMAN INTERACTIVE COMMUNICATION.
Their white paper also proposes analyzing infant-caregiver interaction during play, as social interactions between parent and caregiver are essential to child development. This work is currently ongoing.
“We are analyzing videos of infant-mother interaction with infants at various ages to create computational models of these interactions, and potentially use these models to help characterize infant development and responsiveness in infant-mother communication,” Klein said.
“The ultimate goal of our research would be to create an approach for affordable, in-home interventions utilizing socially assistive robots that use play to improve healthy development in young children,” Klein said.
Added Smith: “The potential to have a positive impact during infancy and to lay the foundations for a positive developmental trajectory are very exciting to us,” Smith said. “We very much appreciate that the award will help us to move its development forward.”
Social skill deficits are a core challenge for those diagnosed with autism spectrum disorders (ASD) [1]. Therapies that address these challenges are available; however, a single program or regimen is not effective for all individuals with ASD.
This may be due to large variations in skill level, cognitive ability, coping ability, and the type and number of specific challenges manifested in each individual with this spectrum disorder [1,2].
The employment of educational robots for children with ASD, in particular, is rapidly increasing, and evidence is growing documenting its positive impact in addressing core features of ASD, including communication and social relationships.
It is of theoretical and practical value for educators and researchers to have an overview of the literature describing in which conditions educational robotics or other interactive computerized therapies (ICT) may be most effective when applied to ASD.
Approximately four in 10 people with autism have a learning disability, which is a lifelong disorder diagnosed in childhood [3].
Although ASD is not a learning disability, it does affect learning. This is why it is necessary to study ASD and learning disability together. Learning disabilities may affect activities such as acquisition, organization, retention, and understanding or use of verbal or nonverbal information [4,5,6,7].
These distinguish from global intellectual deficiency, which describes those who otherwise demonstrate at least average abilities essential for thinking and/or reasoning. Learning disabilities can cause problems with speaking, reading, writing, math, concentration, organization, time, social interactions, or speech comprehension.
Often, children have more than one kind of learning disability, such as attention deficit hyperactivity disorder (ADHD), which can make learning even more of a challenge [7].
Despite the abundance of literature in the area of ASD, there is limited research on ASD about users with learning disability. Some guidelines suggest that provision of therapies in early childhood offer the child with ASD enhanced benefits over alternative therapies offered later in life [8].
Identifying ASD as early as possible and providing evidence-based therapies known to produce positive outcomes is therefore crucial. One of the most important therapies to offer the child with ASD addresses social skill competency, pivotal to help those with ASD overcome social/communication challenges, a core deficit in this disorder [1,2,8,9,10,11,12,13].
These programs are not standardized in their curriculum and are offered by various professional and nonprofessional instructors and therapists [14,15,16,17].
Programs differ by skills taught, participant age-bands, medium utilized (e.g., video modeling, in-person sessions, and web-based interactive sessions), duration of therapy, and length of sessions. Social skill therapies are offered as individual or group sessions and are available in schools, clinician offices, public meeting places, or live electronic video environments.
Social skills therapies include interventions such as play therapy, didactic social skills instructions, cognitive behavioral therapy, modeling, and practice therapies [13,14,15]. Behavioral interventions that address atypical or disruptive actions such as stereotypies, anxiety tantrums, aggressive behaviors, and defiance may also be considered social skill training, as the ability to self-manage one’s negative behaviors assists in social acceptance. Speech therapies, offering receptive and expressive instruction, including pragmatic skills, may be considered a form of social skills training as well, as it teaches effective communication with others.
Multiple professional disciplines provide differing therapies to assist the child with ASD. A team approach is commonly required to concurrently address multiple issues [15,16]. Comprehensive treatment by a team of professionals including teachers, speech therapists, occupational and physical therapists, mental health providers, behaviorists, social workers, and medical clinicians, along with parents, other family members, and caregivers, all participate to influence the current and future potential of these children.
Rapid progress in the field of technology, especially robotics, offers tremendous possibilities for innovation in treatment or even education for individuals with ASD.
Recent studies have shown that computer-based learning has made a huge contribution for children with intellectual disability. It enables pupils to take charge of their own learning, as they find stimulation through “enjoyable repetition” and a gradual increase in level of challenge [18].
Blamires argues that enabling technology provides access to educational opportunities, life experiences, and facilitates engagement with knowledge and people [19].
The assistive technologies combine speech, pictures, words, and animation in interactive ways to structure concepts that suit the level of understanding of learners and their interest [20].
More importantly, computer-based instruction and game-based learning can make a very real contribution to teaching essential life skills for those with ASD [21].
Computer-assisted and robot-assisted therapy is infiltrating the social skills teaching environment, being trialed or incorporated into therapy by a variety of professions to help teach the child with ASD [16,17,22].
Validation of effectiveness of computer-aided therapies to teach social skills is warranted to justify the quality of these interventions. Useful technologies will likely proliferate further into therapy regimens, offering new models and assistance to those who serve these children and their families.
The key question is where can we find the appropriate tools and assistive technologies, e.g., computer-assisted therapies to support those with ASD and learning disabilities?
Furthermore, how do we prepare the learning information such that teachers, therapists, and parents may easily find and use them for children with multiple disabilities?
In this paper, we identify the types of information technology platforms and evaluate the computer- and robot-assisted therapies in regards to their appropriateness for children with ASD and learning disabilities.
The purpose of this literature review includes the following aims.
(1) To answer the research question: What types of Information Technology (IT) platforms are being evaluated in computer and robot-assisted therapies for children with ASD?
(2) To identify the various disciplines studying computer and robot-assisted social skill therapies.
(3) To identify the outcomes being evaluated in each trial.
(4) To determine if results demonstrate benefits to children with autism.
We systematically collected information on available review literature on the topics of ASD, social skills, communication training/interventions, and technology [23,24,25,26,27].
Based on these reviews, we identified a gap in the existing literature/research on the topic of what computer game and robot-assisted therapies are being used to aid social and intellectual functioning of children with ASD.
Results
IT Platform categorization activity revealed two mediums being utilized, with eleven studies being robot interventions and seven being serious computer game interventions. Summarizing each of the reviewed studies within these two domains helps reveal the penetration of these computerized assistance platforms and their effectiveness as a group in teaching children with ASD social skills.
Robot Interventions
The increasing deployment of robots in recent decades has inspired new boundaries for different therapies. Animal-like robots have received especially notable acceptance in therapeutic settings.
Robots were featured as the intervention in eleven unique studies. A robot is a mechanical or virtual agent capable of moving independently and performing complex actions [29,30]. This paper mainly deals with humanoid robots. A humanoid robot is a robot with its overall appearance based on that of the human body [31].
To simulate an autonomous behavior model, researchers in New Zealand [19] used a semiautonomous parrot-inspired robot (KiliRo). Robot-supported therapy using adapted model–rival method was experimentally tested with nine children with ASD for five consecutive days in a clinical setting.
Facial expressions of the children were analyzed when they contacted KiliRo using an application program interface called the Oxford emotion API (Application Programming Interface). The results showed signs that children with ASD were attracted and were happy to interact with the parrot-inspired robot.
The notion that children with ASD prefer robots as tutors to improve their social interaction and communication abilities is supported by recent studies.
Indeed, the research focused on developing a very promising form of intervention called robot-assisted therapy. This therapy has a number of challenges, e.g., the necessary flexibility and adaptability to real unrestricted therapeutic settings.
Pennisi et al. reviewed studies in the period of 2006 to 2016, asking if social robots could be a useful tool in autism therapy [23].
The most frequent deficiency to children with autism and mental disability is social attention, which includes the difficulty of focusing good visual attention. Di Nuovo and his colleagues [32] examined the use of a new deep learning neural network architectures to automatically determine whether a child focused on visual attention during a therapeutic session, indicating their commitment.
They used the Nao humanoid robot for their research and have proposed the use of computer intelligence techniques to increase robot capabilities for greater adaptability and flexibility, enabling the robot to be integrated into any therapeutic environment, according to the specific needs of the therapist and the individual child.
Their article represented a step forward in this direction, as the authors dealt with the problem of evaluating the child’s visual focal point from the low-resolution video footage of the robot camera.
A study by Huskens et al. utilized a robot-mediated intervention based on LEGO® Therapy to study impact on collaborative play behavior [33,34]. The study took place in the Netherlands. The population included three sibling pairs, one sibling with ASD, the other without.
The siblings had to be within 5 years in age from their brother/sister partner. The siblings with ASD all had IQs greater than 80. Sibling pairs were randomly assigned to different baseline lengths of three, four, or five sessions.
The dependent/outcome variables included collaborative behaviors exhibited as
(1) interaction initiations,
(2) responses to questions or instructions from typically developing siblings, and
(3) play together actions to achieve a common goal. During five 30-minute sessions once a week, a robot would instruct one child of the sibling pair to be the guide, the other the LEGO® builder.
The guide received the LEGO® instruction booklet and the builder collected the LEGO® bricks and put them together as instructed by the guide.
The robot reinforced collaboration and offered prompts. The humanoid 57cm “social robot” in this trial was named NAO (Aldebaran Robotics, n.d.) [35]. Robot features included preprogrammed speech, a female voice, was Dutch-speaking, and able to move its arms, legs, and fingers.
The robot had eyes and a mouth, but no nose. It was controlled by a human trainer using a laptop. Results demonstrated no statistically significant changes in collaborative behaviors in any of the three measurement targets for the children with ASD. Of interest is the social validity measure result, indicating that the children with ASD reported the robot sessions were less enjoyable than the nonrobot sessions while their typically developing siblings reported the opposite.
Social attention skills were studied with a robot intervention by Srinivasan et al. [36]. Thirty-six children with ASD aged 5–12 years participated in this randomized controlled pilot study evaluating the effects of novel movement-based interventions to current standards of care. In addition, human trainer vs. robot trainer effects were compared.
Thirty-two intervention sessions were delivered over 8 weeks. The human trainers consisted of physical therapists, a kinesthesia graduate student, and 2 parents.
The robot trainers were the humanoid robot NAO controlled mainly by a human trainer using a laptop and Rovio™ (WowWee) [37], a wi-fi enabled mobile webcam. All groups performed joint action based gross and/or fine motor activities that prompted social skills.
The rhythm and robotic intervention groups incorporated movement-based games promoting both gross and fine motor skills. The comparison group utilized sedentary activities promoting fine motor skills.
The rhythm and comparison groups demonstrated joint attention improvements. Social attention was improved most in the rhythm group, followed in sequence by the robot group and comparison group.
The authors reported that rhythmic and whole-body interpersonal synchrony games led to high levels of social attention compared to sedentary activities. The robot intervention was noted to be hindered by technical limitations, suggesting advances in autonomy and contingent responding would make the robot a more effective tool to assist children with ASD.
In another similar study by Srinivasan et al. [38] utilizing the previously reported rhythm and robotic therapy intervention with NAO and Rovio, the outcome targets were repetitive behaviors and affective states in children with ASD.
Repetitive, disruptive behaviors negatively impact social acceptance and thus are often addressed as part of social skills therapy. Following thirty-two 45-minute intervention sessions, rhythm and robotic therapy was compared to the standard of care intervention in 36 children with ASD, age 5–12 years.
The outcomes targeted were frequencies of sensory, negative, and stereotyped behaviors. In addition, the duration of time the children displayed positive, negative, and interested effect was measured. Results indicated that early in the sessions, the rhythm and robot groups exhibited greater negative behaviors than the control group.
The control group exhibited greater sensory behaviors. After training, the rhythm group reduced negative behaviors. The other groups did not. Affective state results indicated the rhythm and robotic groups demonstrated greater interested affect across all sessions. Negative affect was decreased and interested affect increased in the rhythm group after training. The robot intervention however displayed reduction in positive affect.
The authors concluded their results suggest rhythm-based interventions are socially engaging treatment tools to address core challenges in children with ASD.
Taheri et al. [39] introduced clinical interventions with social humanoid robots in the treatment of autistic Iranian twins in a pilot study based on a single subject design experiment. The robot-assisted interventions for a pair of fraternal twins—one with HFA and one with LFA—are described. Either the robot and/or the therapist gave the instructions for each game to the children and their parents.
The treatments were held in a 5 × 5 × 3 m room. The experimental setup was made up of one or occasionally two humanoid robots. In addition, there were two laptops, two cameras (for filming sessions), Microsoft Kinect Sensor, a video projector, and a whiteboard and chairs for all involved. Results of the 2.5-month robotic treatment demonstrated improvement in the HFA subject’s social and communication skills, while the LFA subject showed improvement in decreased stereotyped behaviors. While noting several study limitations, the researchers observed that robot group games had the potential to improve not only communication but social skills too.
In another research Taheri et al. [40] observed significantly increased verbal communications of paired-groups following a robot-assisted group games program. The six male participants with autism consisted of three pairs:
(1) a pair of 7-year-old fraternal twins, one of whom with high-functioning and the other one with low-functioning autism;
(2) two siblings with high-functioning ASD, one age 15 years, the other age 10 years; and
(3) two high-functioning classmates aged 6 and 7 years old. The participants took part in games at each session in different modes: Robot–Child or Robot–Child–Peer/Parent/Therapist interactions.
(i) Real-time imitation by the robot in upper body movements of the child (in Robot–Child mode).
(ii) Teaching imitation/motor skills by the robot to the children through individual/paired-group exercise and dances (in Robot–Child and Robot–Child–Peer/Parent modes).
(iii) Playing a real xylophone (in Robot–Child mode) pointing to far/near points and showing the cards/objects by the robot/child (in Robot–Child mode).
(iv) Kinect-based recognition game and classification of animals, fruits, places, and objects by pointing to different baskets on the screen (in Robot–Child and Robot–Child–Parent modes).
(v) Playing a developed Kinect-based virtual xylophone on the screen (in Child–Parent/Therapist modes).
Pour et al. [41] studied facial expression recognition in a 2-part study using a humanoid robot named “Mina”. The first part of the study measured reaction and acceptance of the robot’s facial responses by children with ASD. Fourteen Iranian children ages 3–7 years participated with a robot-acceptance rate of 78%.
A second stage of the study compared the children’s performance reciprocating facial gestures modeled by the Mina robot versus by a human mediator. Results showed the subjects with ASD had better performance mimicking the human mediators than the robot.
A study by van Straten et al. [42] evaluated task performance and affective state outcomes of children with ASD aged 4–8 years playing puzzle games with a robot.
The researchers studied the effects of the robot’s intonation and bodily appearance, noting that both impacted the children’s affective states but not their task performance.
Researchers stated the robot’s human-like body appearance as compared to mechanical bodily appearance led to a higher degree of interest by the child in the child–robot interaction and that congruence of bodily appearance and intonation triggered a higher degree of happiness in the children.
David et al. [43] investigated use of a social robot in Cluj-Napoca, Romania. Five children with ASD participated in their research study. The researchers hypothesized that the more social cues the robot uses in child–robot interaction sessions (i.e., head orientation, pointing, and verbal indication), the better the child would perform maintaining joint attention. The results met the hypothesis.
A study by Desideri et al. [44] reported the results of a pilot test conducted using a social robot intervention targeting developmental and social skills.
This study evaluated the educational sessions’ impact on engagement and learning achievement in two 9-year-old male children with ASD and intellectual disability. Results suggested that interaction with a social robot enhanced engagement and goal achievement in one participant while the 2nd participant demonstrated only enhanced goal achievement.
A creative study combining virtual reality technology and social robotics for tutoring children with ASD was undertaken by Saadatzi et al. [45].
Research subjects included three children with ASD, ages 6–8 years. The tutoring system featured a virtual teacher instructing sight words, and included a humanoid robot emulating a peer. Results indicated that participants acquired, maintained, and generalized 100% of the words explicitly instructed to them, made fewer errors while learning the words common between them and the robot peer, and vicariously learned 94% of the words solely instructed to the robot.
Researchers observed that participants responded positively to the robot peer’s performance (e.g., “thank you” and “nice job”). One of the participants consistently greeted the robot and hugged it when the session was completed.
Another participant began imitating the robot’s happy gestures. The researchers suggested that similar package may serve as a context under which learners can safely practice the performance of critical social responses.
So et al. [46] also utilized robot intervention in their research. They examined whether Chinese-speaking preschool children with ASD in their early childhood could catch up to the level of gestural production found in typically developing, age-matched children and whether they showed an increase in verbal imitation after the completion of robot-based training intervention.
Comparison was made to a waitlist control group. Results were favorable during the trial and were maintained in delayed post-tests. The researchers concluded that robot-based intervention may reduce the gestural delay in children with ASD in their early childhood.
These studies highlight uses of robot-assisted interventions to teach social skills to children with ASD. The systematic review by Grossard and colleagues [24] reported excellent state of the art in the topic ICT and autism care in the period of 2017 to 2018.
They analyzed serious games and social robots. The authors noted children with ASD have a specific need for predictability, visual support, and a sequential presentation of information, which aligns well with the use of social robots. They concluded that social robots offer clinicians new ways to interact and work with people with ASD.
Source:
USC
