Merijn Rijk

The rationale behind this thesis
Lower respiratory tract infections (LRTI) are among the most common infections encountered in general practice with around three percent of adults consulting their general practitioner (GP) with an LRTI annually. Uncomplicated LRTIs generally run a favourable course in the absence of ‘risk factors’ for a complicated trajectory. Nevertheless, adverse outcomes, such as hospitalisation or mortality, do occur – especially in patients with more severe LRTI such as community-acquired pneumonia (CAP). A prediction model to aid GPs in identifying LRTI patients at highest risk of adverse outcomes could improve clinical decision-making but is currently lacking in everyday practice.

Complications from LRTIs reach beyond a complicated disease trajectory in terms of hospitalisation or mortality. Evidence consistently indicates that LRTIs can trigger acute cardiovascular events, such as myocardial infarction (MI), stroke, venous thromboembolism (VTE), and atrial fibrillation (AF). Although previous studies found that LRTIs increase the risk of cardiovascular events up to five-fold, none reported estimates of absolute risk. This leaves the population impact of cardiovascular risk associated with LRTIs unknown.

Throughout this thesis, we focussed on improving the prognostication of patients presenting to general practice with LRTI. While doing so, we considered a broad range of relevant outcomes, ranging from adverse outcomes such as hospitalisation and mortality to acute cardiovascular events. In the various chapters of this thesis, we applied state-of-the-art prognostic and etiologic research methods to electronic health records (EHR) data. In addition, we evaluated whether data derived from EHR unstructured clinical notes by natural language processing (NLP) improved prognostication of LRTI patients in general practice.

Findings of this thesis
Chapter 2 reports the findings of a systematic literature review to identify and synthesise the available evidence base on relevant existing prognostic factors and prediction models for all-cause hospitalisation and mortality within 90 days for adults with a GP-diagnosed LRTI. Increasing age, sex, current smoking, a history of diabetes, stroke, cancer, or heart failure, previous hospitalisation, influenza vaccination status, current use of systemic corticosteroids, recent antibiotic use, respiratory rate ≥25/minute, and a clinical diagnosis of pneumonia were identified as promising prognostic factors to be considered when developing or updating a prediction model. Currently available prediction models were considered not suitable for implementation in everyday clinical practice due to high risk of bias and incomplete assessment of model performance.

Chapter 3 describes the rationale for, and design of the development and external validation of two EHR-based prediction models that predict individual risk of (1) 30-day all-cause hospitalisation or mortality and (2) 90-day cardiovascular events in patients presenting to the GP with LRTI. For the first model, the prespecified design was followed with the findings reported in Chapter 5. The observed low event rate of cardiovascular events following LRTI (as reported in Chapter 8) precluded development of a clinically meaningful model. As an alternative to the intended prediction model development study, an etiologic study design was used to quantify the excess cardiovascular risk attributable to LRTI with the findings reported in Chapter 8.

In Chapter 4, recording patterns of signs, symptoms, and vital sign measurements in EHR clinical notes of patients presenting to the GP with an LRTI are evaluated to inform future prediction modelling studies. For these potentially relevant predictors, a substantial amount of nonrecorded values were noted, and when recorded, this was likely selective with abnormal findings more likely to be recorded. This underlines the challenges of using unstructured EHR data in prediction research and implies that, prior to including signs, symptoms, and vital sign measurements from clinical notes, recording patterns of such unstructured data should be carefully assessed to tailor missing data handling techniques.

Chapter 5 reports the stepwise development and external validation of a model predicting individual risk of 30-day all-cause hospitalisation or mortality in adults aged ≥40 years presenting to the GP with an LRTI. Predictors of the ‘final’ model, based on structured EHR data, include demographics (age and sex), cardiometabolic diseases (history of diabetes, heart failure, ischaemic heart disease, stroke, transient ischaemic attack (TIA) pulmonary embolism (PE), deep venous thrombosis (DVT), atrial fibrillation (AF), and peripheral artery disease), other medical history (history of pneumonia, non-dermatological malignancies, chronic obstructive pulmonary disease, asthma, dementia, hospitalisation in previous year, and influenza vaccination in previous year year), current medication use (antibiotic prescription in previous month, immunosuppressants, inhalation medication, and antidepressants), and a clinical diagnosis of pneumonia. The model was developed in a pre-COVID-19 pandemic cohort (2016–2019, outcome rate: 7.8%) from the region of Utrecht, the Netherlands, and was externally validated in a post-COVID-19 pandemic cohort (2022–2023, outcome rate: 11.4%) from the region of Amsterdam, the Netherlands. External validation yielded a c-statistic of 0.71 (95% confidence interval (CI) 0.69 – 0.73), a calibration intercept of 0.28 (95% CI 0.20 – 0.36), and a calibration slope of 0.95 (95% CI 0.85 – 1.05), with predicted risks ranging from 1.5% to 51.3% (median 6.9%). In addition, the incremental predictive value of cardiometabolic diseases beyond age and sex was found to be low (Dc-statistic +0.02). The ‘final’ structured EHR data-based model holds promise to aid GPs in identifying LRTI patients at highest risk of 30-day hospitalisation or mortality. However, before adoption in everyday general practice, future research should assess the model’s impact on individual patient outcomes.

Chapter 6 reports the fine-tuning and evaluation of the performance of two Dutch large language models (LLM) (MedRoBERTa.nl and RobBERT) for extraction of candidate predictor information on signs and symptoms – identified as promising in Chapter 2 – from EHR clinical notes of patients presenting to the GP with LRTI. These models demonstrated good performance when implemented as direct classifier – labelling signs and symptoms as either present, absent, or not reported – with an average F1-score of 0.74 (range 0.56 – 0.87) and 0.69 (range 0.46 – 0.86) using 1,600 manually annotated training samples, respectively. Model performance varied substantially across signs and symptoms, with performance decreasing with increasing missingness and class-imbalance. Nevertheless, these findings support the feasibility of using LLMs for automated extraction of signs and symptoms captured in Dutch EHR clinical notes.

In Chapter 7, the incremental predictive value of signs and symptoms beyond structured EHR data is evaluated by adding these as candidate predictors to the models presented in Chapter 5. These signs and symptoms were derived from unstructured EHR clinical notes by NLP using the LLM (MedRoBERTa.nl) that was fine-tuned as reported in Chapter 6. Although signs and symptoms derived from unstructured EHR clinical notes improved model performance beyond age and sex, their incremental value beyond structured EHR-based predictors for prediction of 30-day all-cause hospitalisation or mortality was limited (Dc-statistic +0.02, without substantial improvement of model calibration). In addition, decision curve analysis revealed limited net benefit of adding the NLP-derived predictors when compared to the model including structured EHR data only. This implies that signs and symptoms derived from unstructured EHR clinical notes do not add substantial predictive information in LRTI patients, provided that GPs adequately incorporate all relevant structured information available from patients’ EHR.

Chapter 8 focusses on the temporary increased risk of acute cardiovascular events following an LRTI. In a population-based nested self-controlled case-series, the absolute excess cardiovascular risk associated with LRTI in adults presenting to the GP with LRTI was estimated for a broad range of cardiovascular events to quantify the population impact of LRTIs on cardiovascular events. The risk of major adverse cardiac and cerebrovascular events (a composite of MI, stroke, and TIA), venous thromboembolism (VTE, including PE and DVT) and new-onset AF (nAF) was particularly increased during the first week following LRTI diagnosis. Incidence rate ratios of these cardiovascular events within 90 days following LRTI diagnosis were 1.3 (95% CI 1.0 – 1.8), 6.8 (95% CI 4.4 – 10.4), and 2.9 (95% CI 2.3 – 3.7) per 1,000 LRTI patients, respectively. This culminates in approximately five to nine excess cardiovascular events attributable to LRTI per 1,000 LRTI patients. Incidence of major adverse cardiac and cerebrovascular events and nAF within 90 days following LRTI diagnosis increased with age and was particularly high in patients with a history of heart failure, diabetes or hypertension, whereas incidence of VTE was most pronounced among younger patients. Given the large number of LRTI patients during seasonal outbreaks, this implies a need for targeted preventative interventions and patient-focussed campaigns to increase seasonal vaccine uptake and awareness of LRTI as trigger for cardiovascular events.

Discussion of this thesis
In the general discussion of this thesis (Chapter 9), I reflect upon the thesis’ implications in the context of the GP’s consultation room. I discuss how the ‘final’ model reported in Chapter 5 based on structured EHR data may be used to guide close monitoring strategies in LRTI patients in general practice while reflecting upon common challenges and pitfalls of implementing prediction models in clinical practice. I argue that well-developed prediction models do not necessarily lead to successful implementation. Therefore, apart from assessment of the model’s impact on individual patient outcomes, GPs and other relevant stakeholders should inform clinically relevant risk thresholds and downstream intervention strategies to ensure successful adoption. Subsequently, I focus on the increased cardiovascular risk attributable to LRTIs and place these risk estimates in the context of traditional cardiovascular risk factors. This illustrates that the risks reported in Chapter 8 warrant, at a minimum, increased awareness among clinicians that LRTIs are an undervalued risk factor for acute cardiovascular events. In addition, I propose potential risk mitigating strategies aimed at early detection and prevention of cardiovascular events, including close monitoring strategies and short-course preventative treatment regimens. I emphasize that input from both patients and GPs on appropriate risk thresholds and acceptability and usability of potentially effective interventions are pivotal for the development and refinement of future risk mitigation strategies. Finally, using an illustrative example of a targeted case-finding strategy for nAF in LRTI patients, I discuss the clinical uncertainties and potential individual and societal adverse effects that should be explored and weighed against potential benefits while developing new cardiovascular risk mitigating interventions for LRTI patients in general practice. I argue that, especially in the case of nAF, targeted case-finding strategies should be tailored to a subgroup of LRTI patients at highest risk of clinically relevant AF to reduce overdiagnosis and (subsequent) overtreatment. In general, when developing interventions to mitigate cardiovascular risk in patients with LRTI, I highlight the importance of carefully balancing benefits with potential risks and adverse effects to adhere to the “first do no harm” principle, both from the individual patient’s perspective and for society as a whole.