Big data analytics help teaching staff improve university students' performance

Juan Pedro Cerro, a member of faculty at the Universitat Oberta de Catalunya (UOC) and researcher at Edul@b, has demonstrated the effectiveness of big data when applied to education. What is known as learning analytics can be used to improve teaching as well as the academic results of university students. Cerro, a computer engineer, developed the DIANA application (an acronym for DIalogue ANAlysis) over the course of completing his PhD in e-learning, supervised by researchers Montse Guitert and Teresa Romeu. The application analyses communicative interaction in the UOC's online forums by accessing the messages that students have exchanged and using a series of indicators and metrics to produces reports for the teaching staff. This is all done in compliance with Spanish data protection law and with full respect for student privacy.

The application was tested for a period of one and a half years on a course shared by all the UOC's bachelor's degrees. The 40 classes in which it was used showed a 6% decrease in the dropout rate. There was also an increase by almost one point in the average grade of the students and a reduction in the disparity of those same grades. "The level of the group was more consistent because the teacher was far more in touch with what was happening in the classrooms, and so was able to respond to certain situations and remedy the course of some of the students," Cerro said. 90% of students rated the DIANA application very highly. The UOC's education innovation centre, the eLearning Innovation Center (eLinC), helped implement the tool in pilot classrooms with more than 2,300 students.

Metadata to guide teaching

DIANA analyses the metadata in the messages exchanged by students in the virtual classroom. In total, it calculates about 30 metrics. "Some of them are very clear, such as the average number of words students send in their messages, or the number of messages and replies that are posted. But others are more complex, such as the comparison of the words used in the communication by means of a small thesaurus defined by the teacher, which serves to find out if the conversation is focused on the activity's topics or if, on the contrary, it has lost focus," Cerro explained.

The application provides two types of information. The first concerns each student individually and includes aspects such as how their participation is spread out over time to detect whether, for instance, the student is very engaged at the beginning and then disengages, or vice versa. The second type of information concerns the group as a whole; the DIANA interface shows how students organize themselves, using metrics such as the number of messages sent per week, the most used time slot, the conversation's word cloud or even the degree to which the discourse matches the semantic field defined by the teacher through certain keywords.

The tool also allows for a series of alerts to be set to warn the teacher whenever the conversation becomes too dispersed or if a student is inactive for too long. This is all done with a clear aim in mind: to improve academic performance and to demonstrate the effectiveness of learning analytics, a discipline that has at times been questioned by some, who express concerns about the role of the teacher being reduced to simple data collector.

This interdisciplinary research covering pedagogy and technology shows quite the opposite. It has been proven that learning analytics facilitate the monitoring and assessment work for teachers. And this can be seen in Cerro's own experience teaching classes: "When a teacher has a large number of students and the learning process involves students working together online, it can be difficult to assess each student individually."

DIANA is listed in the Intellectual Property Registry of Catalonia in all its versions and the idea is to offer it as an open knowledge resource, with a GPL 3.0. licence. As a matter of fact, the application is already open source and there is a link in Cerro's thesis through which you can download the source code so it can be rolled out at any institution. "We have always been open to collaborating with whoever may be interested in expanding the tool's scope," he said.

A new R&D project is under way to further enhance DIANA

With this in mind, following on from the research and in order to check whether the findings can be extrapolated to other educational contexts with different learning methodologies, Montse Guitert and Teresa Romeu, researchers from the Edul@b group at the UOC's Faculty of Psychology and Educational Sciences, are leading a new research project: Using learning analytics in digital environments: Impact on the improvement of university teaching practice. This initiative is funded by the Spanish Ministry of Science and Innovation.

It began in September 2021 and will last for three years. It will include the participation of a further six universities together with the UOC. "We are going to build on the work already carried out in my thesis," explained Cerro, who is also a member of the research team. "We want to improve the definition of the learning analytics' indicators and metrics, so they better assist teachers. We will share this information with experts from other master's degree programmes at different universities, we will analyse the academic impact of the application of these learning analytics in other digital environments to contrast them with our results and, finally, we will provide recommendations on the use of the tool in teaching practice. All of this will mean adapting the DIANA tool to other contexts, so it can be modified and improved."

Juan Pedro Cerro, who studied on the UOC doctoral programme in Education and ICT (e-Learning), won the UOC 2021 Award for Best Interdisciplinary Thesis, and the International Journal of Educational Technology in Higher Education (ETHE) published an article and several infographics on the project's main findings.

This project from the UOC supports the Sustainable Development Goals (SDG) 4, Quality Education, and 9, Industry, Innovation and Infrastructure.

Reference articles:

Cerro Martínez, J.P., Guitert Catasús, M. & Romeu Fontanillas, T. (2020). "Impact of using learning analytics in asynchronous online discussions in higher education." International Journal of Educational Technology in Higher Education (ETHE), 17, 39 (2020). doi: https://doi.org/10.1186/s41239-020-00217-y

Open access doctoral thesis in the UOC's O2 repository: Cerro, Juan Pedro: Seguimiento y evaluación de actividades colaborativas en línea a través de las analíticas del aprendizaje (15 July 2021)

UOC R&I

The UOC's research and innovation (R&I) is helping overcome pressing challenges faced by global societies in the 21st century, by studying interactions between technology and human & social sciences with a specific focus on the network society, e-learning and e-health.

Over 500 researchers and 52 research groups work among the University's seven faculties and two research centres: the Internet Interdisciplinary Institute (IN3) and the eHealth Center (eHC).

The University also cultivates online learning innovations at its eLearning Innovation Center (eLinC), as well as UOC community entrepreneurship and knowledge transfer via the Hubbik platform.

The United Nations' 2030 Agenda for Sustainable Development and open knowledge serve as strategic pillars for the UOC's teaching, research and innovation. More information: research.uoc.edu #UOC25years