The last couple of weeks have not been easy for health secretary Matt Hancock. Last week he included trainees and part time workers in the figures for GP recruitment in an attempt to dress them up as rising. Many publicly called him out on this falsehood – a cursory glance at the available numbers immediately proved him wrong. His assertion was withdrawn, but the distrust from members of the public and those in the NHS going forward will be less easy to repair.
Breach of ministerial code?
Earlier this week the health secretary extolled the benefits of technology in the NHS, in an interview published in the London Evening Standard supplement, paid for by private health company Babylon. Hancock’s photograph appeared next to the company’s logo as he maintained:
Hancock has proudly and publicly made known his allegiance to the “brilliant” app GP at Hand, answering some of his first questions in parliament by saying how convenient being signed up was for him. A government minister should not be publicly endorsing a private company in this way and last night the Labour Party wrote to the Prime Minister demanding an investigation into an alleged breach of the ministerial code. It will be very interesting to see how this all plays out.
It’s all about trust
After news outlet Buzzfeed asked questions around the Secretary of State’s endorsement of Babylon, the branding was withdrawn from the online news article. This follows the same pattern in that fundamental trust in the health secretary is being eroded. When Hancock replaced Hunt a few months ago many were hopeful of more transparency; yet this minister with links to pro-privatisation lobbyists the Institute for Economic Affairs (IEA) was always going to be a concern, no matter how many night-shifts he shadowed.
As we have previously reported, private company Babylon’s GP at Hand application has faced fierce criticism from health campaigners and clinicians alike. It’s often-dubious algorithms have been called out as potentially dangerous, and the way it ‘cherry-picks’ the fitter patients away from struggling NHS primary care practices deprives them of much needed funding paid per patient. Signing up to the service also de-registers patients from their regular GP practice. Patients are left in a very vulnerable position without comprehensive access to a physician for more complex cases.
The Science is clear – GP at Hand is heavily criticised by The Lancet
Private digital health company Babylon launched its GP at Hand service in partnership with the NHS in November 2017, yet until June there had been no trialling of its effectiveness. Letting a private company seeking to make a profit loose upon vulnerable patients is ill-considered at best, and at worst irresponsible. However, health secretary Matt Hancock has a penchant for digital technologies and seems to regard them as a one-size fits all strategy for the future of the NHS. Now though, respected medical journal The Lancet casts serious doubts upon the validity of this application and potentially raises questions therefore about all technology which operates in a similar vein.
Babylon’s own trial
The issue with Babylon’s GP at Hand (aside from the concerns over private companies encroaching into NHS provision) is that their methods had not been independently tested, trialled or researched prior to implementation. In June Babylon conducted its own trial into the reliability of its user interface. The results were positive for the company and they maintained, fair. However, there were a number of flaws to their testing. The trial was conducted internally which left it wide open to an obvious bias and the results were published in a non-peer reviewed journal (arXiv.org) so results are not open to critique. In the journal abstract Babylon maintained that:
“We found that the triage advice recommended by the AI System was, on average, safer than that of human doctors, when compared to the ranges of acceptable triage provided by independent expert judges, with only a minimal reduction in appropriateness.”
Babylon tested the app using diagnostic questions from trainee GP exams and reported that its AI scored 81% compared to an average mark for real-life doctors of 72%. The Royal College of General Practitioners (RCGP) said the claims were ‘dubious’.
The Lancet’s findings
On 6th of November, The Lancet published its findings on the service – the first peer-reviewed paper on the subject; and they are pretty damning. Authors Hamish Fraser, Enrico Coiera and David Wong maintained of Babylon’s trial:
“…the results…were met with scepticism because of methodological concerns. In particular, data in the trials were entered by doctors, not the intended lay users, and no statistical significance testing was performed. Comparisons between the Babylon Diagnostic and Triage System and seven doctors were sensitive to outliers; poor performance of just one doctor skewed results in favour of the Babylon Diagnostic and Triage System.”
For a new service providing access to healthcare to really be credible it must be independently assessed and reviewed. Without such a process it will never be trusted or respected by medics and this trust is crucial to the long-term establishment and survival of such an intervention. Doctors and academics are rightly sceptical of a private company that seeks to make a profit at the expense of established GP practices and is too scared to open themselves to proper scrutiny or trusted academic processes. The Lancet states:
“Babylon’s study does not offer convincing evidence that its…Diagnostic and Triage System can perform better than doctors in any realistic situation, and there is a possibility that it might perform significantly worse… Further clinical evaluation is necessary to ensure confidence in patient safety.”
Indeed, as its parting comment The Lancet remarks how, far from being the saviour of the NHS, new technologies which are not first subject to proper and rigorous testing may have the opposite effect in actually becoming a burden on the service through irresponsible practice:
“There is currently minimal regulatory oversight of these technologies. Without such structure, commercial entities have little incentive to develop a culture that supports peer-reviewed independent evaluation… Symptom checkers have great potential to improve diagnosis, quality of care, and health system performance worldwide. However, systems that are poorly designed or lack rigorous clinical evaluation can put patients at risk and likely increase the load on health systems.”
Despite these findings in the respected Lancet, there has been no announcement that the application is to be curbed, looked into further or indeed withdrawn. It is clear that apps (and particularly this one powered by Babylon), are in no way the answer to the current crisis caused by underfunding and government neglect, there is no such thing as a quick fix. Their rapid roll-out without due diligence and proper scrutiny sets a worrying precedent in terms of all future technological development that may be planned in the NHS.
Samantha Wathen, Press and Media Officer, Keep Our NHS Public
Leave a Reply