The last two days I’ve continued working with our colleagues to refine the messages that were going to be testing as part of our randomised control trial. A randomised controlled trial is where you have two conditions people can be randomly allocated into: either a control group or an treatment group. You have an equal chance of being allocated to either of those.

Researchers will balance, or stratify as needed, for things like gender, age and other demographics that might have an impact on the outcome of what you’re trying to measure.

In our case, the control group still has access to the same resources, support and opportunities as the treatment group, but they will receive a “business as usual” standard communication that they normally would. However, the two treatment groups get two variations of that message, each of which includes a different prompt using behavioural principles. We’re testing to see whether these prompts can improve behavioural outcomes.

One of the unique aspects about our team is that capability building is essential to our methodology. That means that we’re trying to upskill our partners as part of our project methodology.

We also have co-design at the heart of everything we do. Our partners are part of every stage of our research. With these behavioural interventions, we bring the knowledge from the academic literature and we obviously generate additional evidence that we test as part of the experiment. We inform our trial design through our field work. We then build the intervention using the expertise of our partners. In this case, we’re working with them to shape the control group messages, as well as getting their input on the intervention messages.

We held an online workshop with our partners to take them through how to come up with a draft message for the control condition. We took them through an example using the communications material that they already have. They going to come back to us later with the first draft of the control message.

In the meantime, these past two days, my colleague and I have been refining the intervention messages. We started with nine different message frames based on the academic literature, our fieldwork and a survey of users. We got external input to reduce these down to six messages. We have continued to test these messages internally, seeking feedback the rest of our team and our partners. We have finally whittled the possible message frames down to two messages that we can test.

We’ve now finally finessed the text for the intervention, which is amazing. I feel ecstatic!

The next steps are that we will get back the draft message for the control condition from our paterners. We’ll workshop the control and two treatment messages, to make sure that they all fit our randomised control trial methodology.

I then package everything together, including all our literature and other evidence that supports how we arrived at these three messages. I’ll send it to the principal adviser who will do quality control. Then the package of materials comes back to me. I make whatever revisions are necessary. Then I publish the final materials for further user testing.

User testing is a really important part of how we work. In the first instance, it’ll go to the rest of our team who haven’t been working on this project and they’ll look at it from a fresh perspective, focusing on whether it holds up to the behavioural science principles that we’re trying to test. Then, the materials go out to other specialists, plus a data expert, as well as two other teams who will look at the message frames, to make sure that everything makes sense from a lay perspective. This includes plain language and that all the instructions are easy to follow.

This is just all the prep work to design the trial. Then there’s a whole heap of additional work to plan the implementation. This includes liaising with various internal stakeholders, including senior Executives who need to review and approve our methods and message frames. We’ve been working with our data and analytics team so they can provide us de-identified data which we then randomise to allocate people into control or treatment. We need to work with the internal communications team, human resources and other areas to carry out the test. All of this work is technical and time consuming. We need to also do a test run with a couple of teams and make sure all the systems involved work smoothly. Data need to be cleaned and other processes need to be put in place so we can effectively measure our outcome.

Even though this trial is relatively simple in terms of the intervention (testing message frames), no randomised control trial is quick. It takes many different sets of expertise to make sure the intervention works in the field.