Published On: 12/4/2023

by Rob Sullivan, Chief Data & Analytics Officer

Overcoming Scalability Challenges in Real-World Data Processing for Transformative Patient Care

Researchers have massive amounts of real-world healthcare data at our fingertips and when approaching a real-world evidence (RWE) deep analytics task, electronic health records are typically the first stop in terms of data gathering. However, there are a number of other important data sources, including structured data like healthcare claims and unstructured data such as patient-reported outcomes that should be considered and included for a complete view of the patient experience. Deep analysis of this information is used to generate novel insights and data repositories leading to RWE.

Recent draft guidance documents from the FDA offer an opportunity to use these types of data to support regulatory submissions.^1,2,3 The FDA guidance documents elevate the importance of real-world data (RWD) in decision-making and bring rigor to the capture and analysis of RWD. The guidance also highlights how unstructured data often contains critical information for understanding the patient’s healthcare journey and is therefore an essential component in generating evidence.

Accurate and timely processing of such data is imperative to creating robust analytical datasets that can be used in the RWE setting. Quantity of data is not an issue. When it comes to the sheer volume of RWD available to us, it’s often an embarrassment of riches. Challenges lie in dealing with the scope and variability of health data. 

Combined with the complexity and volume associated with a single patient’s EHR data, comes the completeness challenge. When patients routinely visit multiple providers and specialists, electronic health records viewed individually do not present a complete picture. Data must be linked for a specific patient from different sources, incrementally adding data as new sources become available, deriving inferences for missing data and creating derived data elements from existing data. 

Long-term, observational data from disease state registries mean that new information is constantly added over months and years – from the files of thousands of patients. The sheer volume of information involved with this type of research means that manual approaches simply are not feasible. Scalability must be incorporated into the initial study design stages to address the intricacy of this research. Privacy is fundamental and patients have come to expect it. Securing informed consent and protecting patient confidentiality from the start of data collection is an important consideration.

Processing patient data when the number of patients continues to increase over longer timeframes requires a scalable approach that maintains accuracy and completeness. Manual curation cannot support scale. Efficient and accurate ingestion, standardization, normalization and linking of patient data from multiple discrete sources generates a comprehensive view of the patient for analytical purposes. 

With that said, even the most sophisticated systems and models cannot accurately capture all the nuances that exist in complex data sets such as EHR. Human processing is naturally less scalable than computerized systems and so this scarce and valuable resource needs to be allocated appropriately.

When done well, scalability can yield truly meaningful results and generate insights that can change the trajectory of a drug or research program. More importantly, insights borne of rich and comprehensive analysis can ultimately help improve patient care. 

What is an ideal and practical approach to overcoming scalability challenges with RWD? For a more in-depth look, download the whitepaper, Maximizing Real-World Evidence Outputs: Bringing Healthcare Data Processing to Scale, now available from Target RWE. 

 

¹ Real-World Data: Assessing Electronic Health Records and Medical Claims Data To Support Regulatory Decision-Making for Drug and Biological Products

² Considerations for the Use of Real-World Data and Real-World Evidence to support Regulatory Decision-Making for Drug and Biological Products

³ Real-World Data: Assessing Registries to Support Regulatory Decision-Making for Drug and Biological Products