Enhancing Machine Learning Prediction to Improve Asthma Care Management – Physician’s Weekly
When managing patients with asthma, a major goal is to reduce hospital visits resulting from the disease. Some healthcare centers are now using machine learning predictive models to determine which patients with asthma are highly likely to experience poor outcomes in the future. Machine learning is a state-of-the-art method for gaining high prediction accuracy, explains Gang Luo, PhD. While it has great potential to improve healthcare, most machine learning models are black boxes and dont explain their predictions, creating a barrier for use in clinical practice. This has been a well-known problem associated with machine learning for many years.
Predicting & Explaining Asthma Hospitalization Risk
Recently, Dr. Luo and colleagues built an extreme gradient boosting (XGBoost) machine learning model to predict asthma hospital visits in the subsequent year for patients with asthma. This XGBoost model was found to be more accurate than previous models, but like most machine learning models, it did not offer explanations as to why patients were at risk for poor outcomes. To overcome this barrier, Dr. Luo and colleagues conducted a studypublished in JMIR Medical Informaticsin which they developed a method to automatically explain the models prediction results and suggest tailored interventions without lowering any of the models performance measures.
The automatic explanation function was able to explain prediction results for 89.7% of patients with asthma who were correctly predicted to incur asthma hospital visits in the subsequent year. This percentage is high enough to support routine clinical use of this method. Of note, the researchers also presented several sample rule-based explanations provided by the function to illustrate how the function worked (Table).
Suggesting Tailored Asthma Interventions
For the first time, our study showed that we can automatically provide rule-based explanations and suggest tailored interventions for predictions from any black-box machine learning predictive model built on tabular data without degrading any of the models performance measures, says Dr. Luo. This occurs regardless of whether the outcome of interest has a skewed distribution. Clinicians were able to understand the rule-based explanations. Among all automatic explanation methods for machine learning predictions, our method is the only one that can automatically suggest interventions.
According to Dr. Luo, clinicians previously needed to manually review long patient records and think of interventions on their own. This consumes a lot of time, is labor intensive, and may lead to missing important information and interventions, he says. Our method can serve as a reminder system to help prevent clinicians from missing these opportunities. It also greatly speeds up processes, because the summary information is presented directly to clinicians and doesnt require sifting through long patient records to make an informed decision.
The study team notes that the automatic explanation function should be viewed as a reminder for decision support rather than a replacement for clinical judgment. It is still the clinicians responsibility to use their own judgment to decide whether to use the models prediction results and apply suggested interventions to their patients. If there are any doubts, clinicians are recommended to check their patients records before making final decisions on any recommendations.
Impacting Clinician Use of Machine Learning for Patients With Asthma
After further improvement of model accuracy, using the asthma outcome prediction model together with the automatic explanation function could help with decision support to guide the allocation of limited asthma care management resources. This could help boost asthma outcomes and reduce resource use and costs.
Predicting hospital visits for patients with asthma is an urgent need for asthma care management, which is widely used to improve outcomes, Dr. Luo says. Researchers have been working on this problem for at least two decades but have repeatedly encountered problems with low prediction accuracy. Our model significantly improved prediction accuracy. In addition, we can now automatically explain the prediction results. These are important factors that impact the willingness of clinicians to use our model in clinical practice. In future research, we plan to test our automatic explanation method on more predictive modeling problems, such as in different prediction targets and diseases.
Luo G, Johnson MD, Nkoy FL, He S, Stone BL. Automatically explaining machine learning prediction results on asthma hospital visits in patients with asthma: secondary analysis. JMIR Med Inform.2020;8(12):e21965. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7808890/.
Luo G, He S, Stone BL, Nkoy FL, Johnson MD. Developing a model to predict hospital encounters for asthma in asthmatic patients: secondary analysis. JMIR Med Inform. 2020;8(1):e16080.
Luo G. Automatically explaining machine learning prediction results: a demonstration on type 2 diabetes risk prediction. Health Inf Sci Syst. 2016;4:2.
Luo G, Stone BL, Sakaguchi F, Sheng X, Murtaugh MA. Using computational approaches to improve risk-stratified patient management: rationale and methods. JMIR Res Protoc. 2015;4(4):e128.
Read the original here:
Enhancing Machine Learning Prediction to Improve Asthma Care Management - Physician's Weekly