I’m on leave. The day before I went on leave, everybody wanted everything from me, because I was going on leave, even though I’m only away for three business days. In order to go on leave, I had to deal with a request for a “quick trial,” providing advice on null results, and managing staff remotely.

No such thing as a “quick trial”

One of the hectic developments is that one of our trial partners came to us and asked whether we could put together a new randomised control trial into the field by early next week, so that it would be finished before the end of year holidays begin. It’s difficult to achieve. The ethics application alone can take several weeks to complete.

We didn’t see it as viable, but we still drafted a trial design to see how we’d do it, and whether we could do it on time. Thankfully, it’s now off the table.

In the past, we had someone promote to our team the myth of the “quick trial.” They are possible, but tough to do well, especially with meagre resources, and in a way that meets ethics requirements. To do a randomised control trial quickly, you would need to double the number of people doing the work. It’s easier to do in a nimble consultancy, but we rarely have this luxury, as our staff are already committed on other projects.

Nevertheless, when we’re asked to scope a “quick” trial, we still map it out and develop options, to see what’s possible.

Null results are worthwhile

Another frenetic thing that happened was that some of my colleagues are writing up the results for a trial that I helped to design. The analysis shows null results. This means that our hypothesis was not proven. I left the project when I came back from medical leave and all the people who are working on the project at the moment were not part of the original project design. All the original people working on it have left the team, or in my case, I stepped off the project. The new project team lacked the context to interpret the results.

The intervention did successfully change behaviour. What was in place before our trial was not leading to behavioural change. That is, people weren’t taking up diversity training in the current context. Our trial changed processes to make it easier and more attractive to join training, and we improved communication, so that many people took up the training. However, we wanted to test which messages would lead to higher training.

We tested two behavioural messages against a control group, and the interventions did not lead to a statistically significant result in comparison to the control group. Instead, it looks like all three groups performed the same. Meaning that all three messages led to more people doing training. This is a terrific result for our partners, but for a group of scientists wanting to measure clear results, null results are often an unhappy outcome.

The control group was not a true “control,” as that message was also improved on via our trial. It’s a long story how this came to be the case. We already had evidence that the existing message (the true control) was not increasing training. As it so often happens, once our partners see our behavioural messages, they want to improve the “control.” This, together with our process enhancements, makes it difficult to detect a significant effect between the messages we tested.

People who are not scientists don’t know that academic publications rarely publish null results. Most of what the public hears about are studies that have found statistical significance. There are a lot of problems with this system, because scientists don’t publish the so-called “unsuccessful” trials. First, people don’t know what doesn’t work. Second, this provides a perverse incentive to never learn from mistakes, or what makes trials less effective.

Because we’re applied researchers working in policy, we have a policy of transparency. We publish our results regardless of what the outcome is. For us, a null result is nothing to be ashamed about because we learned a lot. This trial, as with all our trials, improves processes. In fact, with this one, we did improve behaviour.

But it was stressful for my colleagues to have to report null results to their partners. I didn’t want to leave them to try and communicate what all this means to senior executives who don’t understand science the way we do.

Remote managing

Then, there’s the regular challenges of managing a team doing remote work. Our team is very lucky to still be able to do remote work. It means that I’m giving guidance, reviewing people’s work, and providing tasks that need to be done while I’m away, for people who work different hours and days than me. There’s a lot of work involved in making sure that staff feel supported and encouraged.

One team member whom I’m managing will be presenting her work for the first time to members of our team. They lack confidence to present in my absence, even though it’s only a 30-minute meeting with one other person from our team. Preparing them and trying to boost confidence takes a lot of work.

All this means is that going on holidays for a senior manager, you end you end up doing more work, so that you can have a couple of days off. This is true of everybody, no matter what sector they’re in. In the end, I probably worked an extra day, so the three days that I’m having off are probably only really two days.

That brings us to my leave!

“On leave”

Leave has been mostly great, except that I have had to keep an eye out on my important and urgent emails. With one of my trials in the field, I can never fully switch off, because, unfortunately, there are some issues that other team members can’t answer. For example, urgent questions from partners delivering our intervention, or requests from our senior stakeholders.

One fun work thing I did today was to film my segment for an end of year thank you video that one of my colleagues is editing. It’s going out to our trial partners. (Yes—work again, during my leave!) I was thanking partners for a project that I’m super proud of; the scale up that I’ve told you about before. We tested it last year and we followed the results for 12 months, and it has been statistically significant.

Apart from the emergency emails over the last couple of days it’s been quite blissful.

Yesterday I caught up on lots of things around the house which was highly gratifying. Today, I did mostly nothing. I had to shift my plans around because the weather is miserable. Tomorrow should be a beach day. Then, on the weekend, I hopefully will be able to bring you some of what I get up to on my Instagram.