The importance of digitising unstructured patient data

Healthcare digitisation specialist, Ted Reynolds from Restore Digital, discusses how the NHS could be missing out on the important health insight unstructured patient data can deliver.

The digital transformation of the NHS is well underway.

Unsurprisingly, over the years interest in, and the uptake of, Electronic Patient Records (EPRs) has grown markedly.

Particularly, following the Secretary of State for Health and Social Care, Sajid Javid, announcing his ambition earlier this year for 90 per cent of NHS trusts to have EPR in place by December 2023 – with the remaining 10 per cent expected to be in the ‘implementation’ stage by that time.

This isn’t a case of replacing paper-based patient records with a like-for-like digitised version – the opportunity, and in turn the challenge, goes far beyond that.

There is a real need for Trusts to consider how they will manage, and benefit from, the huge amount on unstructured data currently only retained in paper-based records.

Over the last two decades the volume of paper-based records have grown considerably. And that’s not just because the population is living longer, or that cases of chronic health conditions have increased.

Each patient’s record has expanded in size due to increased governance requirements and enhanced clinical observation techniques. But that’s not even the biggest challenge.

The standing joke that a patient couldn’t read a clinician handwriting lost its humour when prescriptions became digital. But, most paper-based patient records are indeed handwritten and due to poor handling and storage, often unreadable.

We call information such as this in the industry ‘unstructured data’ – and unsurprisingly it’s a huge hurdle to overcome so the NHS can achieve their digitisation goals.

EPRs work well because they capture data in a structured form. For example, assessments are completed in a systematic, workflow driven way, and the data captured has qualities that allow it to be shared and used multiple times in differing scenarios.

Persistent data, such as height, weight, alcohol consumption, and smoking status for example, is especially valuable in this context. This data provides vital health insight, all of which helps clinicians provide better patient outcomes.

Sharing this data is clearly vital on a number of fronts. Not least because it helps reduce the need for a patient to ‘tell the same story’ multiple times during an episode of care. But it also ensures clinicians have consistent data to aid with decision making and diagnosis.

On the other hand, EPRs do not work well with unstructured data. But it’s often in the unstructured data, the handwritten notes and additions, that care professionals can pick up on important health insights.

Electronic document management systems provide this access, though it’s not always straight forward.

In an ideal world EPRs would be able to access the unstructured data in external systems directly, but the reality is that an intermediate stage is required where that data is ingested into an electronic document management system then classified or indexed.

In a fully interoperable scenario, an EPR would access the patient case note in the system, in patient context.

This can happen in a ‘window in window’ environment though it presents challenges, including the need to prevent the patient context being lost and clinical risk being introduced.

When legacy patient records are captured as images, and if classification is applied, the metadata that this generates is used to identify and access the images.  This is the basis of how electronic document management systems ‘find’ information.

The system can also use optical character recognition (OCR) to identify key words and phrases and to retrieve cohorts of documents for users to view.

An extension of this uses patterns of key words to identify images of documents ‘on the fly’, offering a rudimentary type of classification, sometimes referred to as dynamic tagging.

Where information exists as data in an XML file, for instance as in the Cancer Outcomes and Services Data Set – Pathology, this data can be captured into a digital template that is then classified and indexed, allowing it to be viewed in the electronic document management system as a ‘form’.

The capture of data held in disparate databases, end of life systems or archival repositories is not the sole domain of the electronic document management system providers.

It is inherently linked to the choices those implementing electronic document management systems need to make regarding record structure, data metatags and legacy data accessibility.  It sits front and centre with digitising paper.

Another challenge those looking for information face is that historically, paper records were scanned and stored in multiple, disparate systems.  Similarly, data generated by some digital systems was never exported to a central repository.

How many environments are there which contain terabytes of data for which the supporting index files cannot be accessed without spending thousands on professional services, effectively making the data inaccessible?

Trusts historically scanned patient records into standalone digital systems some of which have gone end of life, or have operating systems with vulnerabilities, posing a security threat.

These remain isolated from main networks and the resultant images and data files sit in silos, again inaccessible to users.

The project to digitise all patient record data is vast, but it is a vitally important part of the transformation of the NHS.

From a digitisation perspective tackling the sheer volume of unstructured data included within patient records is an extra hurdle the NHS must overcome.

But once achieved, it could help unlock vital insight that can be seamlessly shared throughout the health ecosystem for the benefit of patients.