Researchers create video game to teach children oral and reading skills using body movements

As many as 9.3% of Spanish students received specialized educational support in the 2020/2021 academic year. This includes children with specific needs, especially learning disorders (38.1%), such as dyslexia (reading difficulties due to problems identifying speech sounds) and dysorthography (writing ability disorder), according to figures from the Spanish Ministry of Education and Vocational Training.

In order to improve the oral and reading skills of schoolchildren and prevent possible language disorders, Esteban Peñaherrera, a student on the doctoral programme in Education and ICT (E-learning), and an entrepreneur participating in Edutech Emprèn, has created WIN, an interactive video game designed for children between 3 and 6 years old. The project, which was a finalist in the SpinUOC entrepreneurship programme organized by the Hubbik platform, uses artificial intelligence models to detect body movements and speech to let players interact with the game. "Our potential clients are parents and teachers in public and private schools who use it as a complement in education, and speech therapists," said Peñaherrera, who developed WIN while working towards his PhD, under the supervision of Llorenç Andreu, an expert in learning difficulties and language disorders, professor in the UOC's Faculty of Psychology and Education Sciences and the leader of the Cognition and Language Research Group (GRECIL), affiliated to the eHealth Center.

Video with Esteban Peñaherrera presenting WIN at the SpinUOC

Main oral skills

The project is based on three oral skills that are the main predictors of a person's reading level: phonological awareness, morphosyntax and vocabulary. "The video game is based on the PREVENIR methodology which I developed in my doctoral research, which explains the oral skills that children need to be taught, and how to do it. There is scientific evidence that practising oral skills improves reading levels," said Peñaherrera.

Phonological awareness is the ability to identify and manipulate phonemes and syllables. The video game makes this interactive: "for example, the video game asks the child: 'do the Catalan words casa (house) and cama (leg) begin in the same way?' And they have to answer yes or no. They can do this by moving their body (jumping), selecting the option with their hand, or orally," he explained. The second skill, morphosyntax, consists of learning word structure and order. "One of the goals in the video game is for the children to order sequential images with their hands, with each one representing the past, present and future, so that they can practise the tenses." For vocabulary, which is knowledge of and the ability to use words and to express their message correctly, the children have several exercises: "for example, different colours appear on the screen, and they have to select them and drag them into a bucket. The system tells them whether they are correct or incorrect, providing positive or neutral feedback. Or it'll show them three images, such as images of a mouse, of different sizes, and ask them to select the ratonzote (the biggest mouse) or the ratoncito (the smallest)."

The game has around 30 activities for each age group: 3 to 4 years old, 4 to 5 years old and 5 to 6 years old, which are aimed at improving their reading level and preventing possible language disorders. "Around 7% of children worldwide suffer from specific language impairment (SLI), or in other words, problems related to speaking and understanding language. This means that half of them will end up having difficulty learning to read," says Llorenç Andreu in his book El Trastorno Específico del Lenguaje (Specific Language Impairment).

Peñaherrera created this tool because he suffered from an obsessive-compulsive disorder as a child, which he overcame thanks to psychological help. "I've always felt indebted to child psychology, and as I'm a software developer, I decided that the best thing to do was to create a video game to help people with a disorder, in this case a language disorder," he said.

Artificial intelligence-powered interaction

The WIN video game will include gamification techniques to make it challenging, entertaining and motivating. "My idea is for children not to think of the video game as a class, but as something that's fun. The sound and visual effects enhance the players' motivation and interest. And incorporating physical movement into the game enhances its benefits, as there's research that shows that physical movement encourages the body to secrete serotonin, and enhances the child's learning," he explained.

The game is based on two models of artificial intelligence: one of them uses a camera built into the computer that tracks points on the human body, so that the video game knows where the parts of the human body are; the other model is based on voice recognition. "I've adapted these two models to the needs of video gaming, to make it a new and disruptive tool. The idea is that the child stands one to two metres in front of a computer with a camera, and plays by moving their body: selecting or dragging images using their hand, jumping, crouching, running from left to right, etc." He added that the speech recognition model allows exercises with phonemes and syllables which are part of the work done by speech therapists: "for example, the system may ask: tell me a word that begins in the same way as casa (house). Depending on the response it receives, it will give positive or neutral feedback."

The video game also features Luni, a character from another planet, who empathizes and interacts with the player. "This helps the child to become more involved, because in addition to playing and learning, they have the goal of helping the character," said Peñaherrera.

This UOC project promotes UN Sustainable Development Goals (SDGs) 3, Good Health and Well-being and 4, Quality Education. 

 

Press contact

Núria Bigas
nbigasf@uoc.edu
619 416 930

UOC R&I

The UOC's research and innovation (R&I) is helping overcome pressing challenges faced by global societies in the 21st century by studying interactions between technology and human & social sciences with a specific focus on the network society, e-learning and e-health.

Over 500 researchers and more than 50 research groups work in the UOC's seven faculties, its eLearning Research programme and its two research centres: the Internet Interdisciplinary Institute (IN3) and the eHealth Center (eHC).

The university also develops online learning innovations at its eLearning Innovation Center (eLinC), as well as UOC community entrepreneurship and knowledge transfer via the Hubbik platform.

Open knowledge and the goals of the United Nations 2030 Agenda for Sustainable Development serve as strategic pillars for the UOC's teaching, research and innovation. More information: research.uoc.edu.