Juan Vilá
He won the award for the best interdisciplinary doctoral thesis at the UOC in 2021. He combines pedagogy and technology in his research, and has even developed his own software.
Juan Pedro Cerro has a degree in Computer Engineering, but decided to work in education. His PhD thesis – Seguimiento y evaluación de actividades colaborativas en línea a través de las analíticas del aprendizaje [Monitoring and evaluation of online collaborative activities through learning analytics] – won the 2021 UOC Award for Best Interdisciplinary Thesis.
As you say in the introduction to your thesis, you know about the topic you have investigated from your own experience.
I have more than twenty years of experience as a teacher in the classroom and twelve, online. Though I started out studying in a technical field, I've always been interested in facilitating students' learning and improving the conditions in which teachers teach, especially when learning is not only face-to-face, but also uses digital or online media.
You completed two master's degrees before your PhD.
That was where my thesis originated. In fact, I completed my Master's Degree in Education and ICT (E-learning) with research into how educational technology is used in virtual classrooms. I then did the Master's Degree in Free Software, which I concluded with research focusing on the use of applications that facilitate teachers' monitoring of learning activities.
Why did you focus on collaborative activities?
As a teacher, monitoring and evaluating collaborative activities online raises some difficulties because students adopt different roles within the work team, and it's not easy to know what each person is doing. But learning analytics provide a solution to this problem.
What are learning analytics?
It's a discipline derived from Big Data; it involves a series of techniques and instruments that help us collect the information which students produce in virtual environments when they interact with each other, with the platform or with the teaching materials. After analysing the data, we can intervene in the teaching process and enhance learning.
Why are they sometimes called into question?
They are criticized by some scientists on the grounds that teachers become mere consumers of data. One of the aims of my research was to disprove those criticisms, and to show that learning analytics make a positive contribution to monitoring and evaluating learning activities.
Part of your thesis consisted of creating a tool for monitoring and evaluating online discussions.
One of the results of my research is a learning analysis tool – a computer application that I called DIANA – an acronym for DIalogue ANAlysis. It analyses the communicative interaction that takes place in the UOC's messaging spaces, accesses all the messages that students have exchanged, and calculates a series of indicators and metrics that are reported to the teacher.
Did any privacy issues arise?
One of the critical aspects of the research was to ensure that it complied with Spanish data protection law. We were very careful to make sure that the data were handled and analysed by each teacher within their educational context. This meant that the students' information was not disclosed, and its integrity was preserved.
What are the main conclusions of your thesis?
My thesis makes two major contributions. The first is technological – the DIANA application – and the second is pedagogical, and involves three areas. The first is a reduction in the students' dropout rate of almost 6% when learning analytics are used. The second is an improvement in the students' academic performance: we found an average increase in classroom grades of almost one point, and grades didn't vary as widely. In other words, we made the students' performance more uniform thanks to the extraordinary information provided by the analytical tool. The third area is the high level of satisfaction (almost 90%) that students reported with the personalized feedback that the teacher sent them.
How would you describe the process involved in preparing your thesis?
There are always ups and downs in a doctoral thesis. It's a bit like a roller coaster, and sometimes it's difficult to see the light at the end of the tunnel. It sounds like a cliché, but for me, the thesis has been both a personal and professional challenge. A PhD requires a willingness to spend many hours on research, with the uncertainty of whether the path you have chosen will lead you to short-term results or not. And you have to have an open mind and act with scientific rigour when you come across results that you didn't expect or weren't able to predict.
What did you find most difficult?
The most difficult part for me was the initial stage – the planning. It took me a year to decide how I was going to organize the field work.
What gave you the most satisfaction?
The conclusions, because it enabled us to publish an article in a high-impact international journal.
How long did the whole process take?
It took five years, working part-time. I combined the research with my main work.
What would you highlight about the UOC?
Scientists often work in isolated laboratories. However, this research has enabled me to meet other people who are also passionate about teaching. I'd particularly like to mention the Edul@b research group, because they constantly helped me with the pilot tests.
Did the pandemic have any impact on your research and the thesis preparation process?
It didn't have a significant impact on undertaking the research, because the academic management processes were being carried out at a completely online university. But from the point of view of the teachers' perception, interest in my research did increase a great deal, as the pandemic changed the paradigm from mostly face-to-face education to online teaching.
What has the thesis given you?
The interest that my thesis supervisors Montse Guitert and Teresa Romeu showed in the tool that I designed led them to propose an R&D project to the Spanish Ministry of Science and Innovation. This means my work will continue, and will expand the research to other contexts and other universities. I'll be working as a research specialist for two and a half years, which was one of my main motivations when doing the thesis. This means that far from being the end of the road, my PhD has become the starting point for new projects.
What has the award for the best thesis meant to you?
It's really satisfying. It gives you an extraordinary boost and motivates you to continue dedicating many hours to research and to stay connected to the educational community and the most innovative research groups, which are the essence of any breakthrough that can happen in society.
What advice would you give someone who is starting a thesis?
Above all, be very patient, particularly if you have negative results. You have to persevere, because effort and work always receive their just reward.