Pilot Structure
Accurx and the NHS App team have concluded a three-month pilot to test the effectiveness of the NHS App as a channel for individual messaging, in comparison to SMS . We wanted to learn how we could integrate individual messages into the NHS App, with minimal disruption, and save significant fragment costs.
We ran a 12-week pilot, integrating Individual Accurx messaging into the NHS App. We set a fallback to SMS of 60 mins, and measured response rates in both groups at 15 min intervals. We recruited 1,211 practices across England and randomised them to be in the treatment (NHS App + SMS) or control group (SMS only).
Over 12 weeks, we gathered significant data and conducted in-depth interviews. The findings show that iterations to the app, greater patient uptake, and further data collection are all necessary before we can roll-out this functionality nationally. This means the aim to make the NHS App the default channel for all GP patient messaging will take longer than planned to achieve. Below, we outline the findings of the pilot.
Findings
When compared to response rates in the control group (SMS-only), patient response rates in the treatment group (using the NHS App) were significantly lower.
Self-book response rates at 60 minutes were 48% lower in the treatment group compared to the control group.
Message with reply response rates at 60 minutes were 31% lower in the treatment group compared to the control group.
Questionnaire response rates at 60 minutes were 23% lower in the treatment group compared to the control group.
By the three-hour mark (2 hours after a fallback SMS is sent), the treatment group recovers, with response rates matching the control group.
By the seven-day mark, the control group performs slightly better, with response rates being 5% higher.
NHS App adoption can be initially challenging for patients, but has long term benefits.
The NHS App login process was the main pain point identified by patients and practices.
Some practices invested heavily in patient education, by running workshops, training and helping patients get set up on the App. Once set-up, patients found features like being able to view their health record or test results beneficial.
Patients often lacked awareness that the NHS App could be used as a messaging channel, and weren’t expecting to receive communications via the App. This caused confusion, particularly in the early weeks of the pilot.
The pilot did not significantly increase NHS app adoption, with adoption rates similar in both treatment and control groups.
Practices that remained in the pilot recognised that, while change is always difficult, transitioning to individual messaging into the app would be beneficial in the long run. These practices overcame the initial pain, as patients grew more confident using the App.
Using the NHS app for time-sensitive messages was challenging for some practices
Some patients missed time-sensitive messages.
Practices using the total triage model to manage inbound requests were particularly impacted, as they rely on timely communication with patients. These practices were more likely to exit the pilot early.
Practices that complemented time-sensitive messages with additional phone calls were less impacted by individual messages going via NHS App.
8.3% of messages did not require an SMS fallback
This was an average saving of £95 saved on fragments during the pilot, or a £412 saving annually, per practice.
As NHS App adoption and engagement with the App increases, we would expect this to rise in the future