On this beautiful morning a few weeks back, I presented our team’s study on vocational training to the public sector. I used a fun example of Star Wars characters to illustrate our data on the importance of communicating learning objectives between teachers, employers and students.

Essentially, in a behavioural science randomised control trial, researchers will focus on a particular issue affecting a sub-population, such as students not receiving adequate feedback on their training progress. We obtain ethics and work with stakeholders to determine what solution or ‘treatment’ might improve outcomes. This might be different types of incentives, tweaks to information or environmental cues, or some other novel way to encourage changes towards a desired behaviour. Using statistics, we work out the best way to generate a representative sample to test that treatement.

Some people are put into a control group. They receive the same level of service and support as they did before the study began. For them, it’s ‘business as usual.’ The rest of the sample go into the intervention group (that is, they receive the treatment). For example, a message prompt with some tips for employers to give better feedback to better support their students.

In a lab, you can control the delivery of the treatment and other dynamics so there’s less outside influences. The trade off is that how people behave in an experiment is not the same as how they might behave at school, at work or in the place that you’re trying to effect change.

Testing in the real world is more realistic (though the researcher still has more control than if the organisation was simply testing a new change without scientific rigour). But in realworld contexts, people also talk to one another or they can see things changing around them when interventions are being tested. So people in the control group might hear about what happening to colleagues in the intervention group.

In our case, employers had some students they supervised in the control group, while some of their other students were in the intervention group. This means there’s potential for employers to have modified their behaviour towards both groups, even though we tried to minimise this happening. Researchers cannot oversee such contamination in large-scale public studies, so we control for it in our analysis. This is known as the spillover effect. In our presentation, I used the training of jedi, siths, Chewbacca and Hans Solo to illustrate spillover for our study.

It is a true joy and privilege to be able to do research in real-world settings and to communicate ongoing findings of our team’s work to stakeholders.