Publications by authors named "Yizhao Ni"

37 Publications

External validation and comparison of risk score models in pediatric heart transplants.

Pediatr Transplant 2021 Dec 8:e14204. Epub 2021 Dec 8.

Cardiothoracic Surgery, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio, USA.

Background: Pediatric heart transplant (PHT) patients have the highest waitlist mortality of solid organ transplants, yet more than 40% of viable hearts are unutilized. A tool for risk prediction could impact these outcomes. This study aimed to compare and validate the PHT risk score models (RSMs) in the literature.

Methods: The literature was reviewed to identify RSMs published. The United Network for Organ Sharing (UNOS) registry was used to validate the published models identified in a pediatric cohort (<18 years) transplanted between 2017 and 2019 and compared against the Scientific Registry of Transplant Recipients (SRTR) 2021 model. Primary outcome was post-transplant 1-year mortality. Odds ratios were obtained to evaluate the association between risk score groups and 1-year mortality. Area under the curve (AUC) was used to compare the RSM scores on their goodness-of-fit, using Delong's test.

Results: Six recipient and one donor RSMs published between 2008 and 2021 were included in the analysis. The validation cohort included 1,003 PHT. Low-risk groups had a significantly better survival than high-risk groups as predicted by Choudhry (OR = 4.59, 95% CI [2.36-8.93]) and Fraser III (3.17 [1.43-7.05]) models. Choudhry's and SRTR models achieved the best overall performance (AUC = 0.69 and 0.68, respectively). When adjusted for CHD and ventricular assist device support, all models reported better predictability [AUC > 0.6]. Choudhry (AUC = 0.69) and SRTR (AUC = 0.71) remained the best predicting RSMs even after adjustment.

Conclusion: Although the RSMs by SRTR and Choudhry provided the best prediction for 1-year mortality, none demonstrated a strong (AUC ≥ 0.8) concordance statistic. All published studies lacked advanced analytical approaches and were derived from an inherently limited dataset.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1111/petr.14204DOI Listing
December 2021

Understanding Pediatric Surgery Cancellation: Geospatial Analysis.

J Med Internet Res 2021 09 10;23(9):e26231. Epub 2021 Sep 10.

Department of Pediatrics, University of Cincinnati, Cincinnati, OH, United States.

Background: Day-of-surgery cancellation (DoSC) represents a substantial wastage of hospital resources and can cause significant inconvenience to patients and families. Cancellation is reported to impact between 2% and 20% of the 50 million procedures performed annually in American hospitals. Up to 85% of cancellations may be amenable to the modification of patients' and families' behaviors. However, the factors underlying DoSC and the barriers experienced by families are not well understood.

Objective: This study aims to conduct a geospatial analysis of patient-specific variables from electronic health records (EHRs) of Cincinnati Children's Hospital Medical Center (CCHMC) and of Texas Children's Hospital (TCH), as well as linked socioeconomic factors measured at the census tract level, to understand potential underlying contributors to disparities in DoSC rates across neighborhoods.

Methods: The study population included pediatric patients who underwent scheduled surgeries at CCHMC and TCH. A 5-year data set was extracted from the CCHMC EHR, and addresses were geocoded. An equivalent set of data >5.7 years was extracted from the TCH EHR. Case-based data related to patients' health care use were aggregated at the census tract level. Community-level variables were extracted from the American Community Survey as surrogates for patients' socioeconomic and minority status as well as markers of the surrounding context. Leveraging the selected variables, we built spatial models to understand the variation in DoSC rates across census tracts. The findings were compared to those of the nonspatial regression and deep learning models. Model performance was evaluated from the root mean squared error (RMSE) using nested 10-fold cross-validation. Feature importance was evaluated by computing the increment of the RMSE when a single variable was shuffled within the data set.

Results: Data collection yielded sets of 463 census tracts at CCHMC (DoSC rates 1.2%-12.5%) and 1024 census tracts at TCH (DoSC rates 3%-12.2%). For CCHMC, an L2-normalized generalized linear regression model achieved the best performance in predicting all-cause DoSC rate (RMSE 1.299%, 95% CI 1.21%-1.387%); however, its improvement over others was marginal. For TCH, an L2-normalized generalized linear regression model also performed best (RMSE 1.305%, 95% CI 1.257%-1.352%). All-cause DoSC rate at CCHMC was predicted most strongly by previous no show. As for community-level data, the proportion of African American inhabitants per census tract was consistently an important predictor. In the Texas area, the proportion of overcrowded households was salient to DoSC rate.

Conclusions: Our findings suggest that geospatial analysis offers potential for use in targeting interventions for census tracts at a higher risk of cancellation. Our study also demonstrates the importance of home location, socioeconomic disadvantage, and racial minority status on the DoSC of children's surgery. The success of future efforts to reduce cancellation may benefit from taking social, economic, and cultural issues into account.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.2196/26231DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8463951PMC
September 2021

Machine Learning for Detection of Correct Peripherally Inserted Central Catheter Tip Position from Radiology Reports in Infants.

Appl Clin Inform 2021 08 8;12(4):856-863. Epub 2021 Sep 8.

Division of Neonatal and Pulmonary Biology, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio, United States.

Background: In critically ill infants, the position of a peripherally inserted central catheter (PICC) must be confirmed frequently, as the tip may move from its original position and run the risk of hyperosmolar vascular damage or extravasation into surrounding spaces. Automated detection of PICC tip position holds great promise for alerting bedside clinicians to noncentral PICCs.

Objectives: This research seeks to use natural language processing (NLP) and supervised machine learning (ML) techniques to predict PICC tip position based primarily on text analysis of radiograph reports from infants with an upper extremity PICC.

Methods: Radiographs, containing a PICC line in infants under 6 months of age, were manually classified into 12 anatomical locations based on the radiologist's textual report of the PICC line's tip. After categorization, we performed a 70/30 train/test split and benchmarked the performance of seven different (neural network, support vector machine, the naïve Bayes, decision tree, random forest, AdaBoost, and K-nearest neighbors) supervised ML algorithms. After optimization, we calculated accuracy, precision, and recall of each algorithm's ability to correctly categorize the stated location of the PICC tip.

Results: A total of 17,337 radiographs met criteria for inclusion and were labeled manually. Interrater agreement was 99.1%. Support vector machines and neural networks yielded accuracies as high as 98% in identifying PICC tips in central versus noncentral position (binary outcome) and accuracies as high as 95% when attempting to categorize the individual anatomical location (12-category outcome).

Conclusion: Our study shows that ML classifiers can automatically extract the anatomical location of PICC tips from radiology reports. Two ML classifiers, support vector machine (SVM) and a neural network, obtained top accuracies in both binary and multiple category predictions. Implementing these algorithms in a neonatal intensive care unit as a clinical decision support system may help clinicians address PICC line position.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1055/s-0041-1735178DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8426077PMC
August 2021

Seasonality, mediation and comparison (SMAC) methods to identify influences on lung function decline.

MethodsX 2021 21;8:101313. Epub 2021 Mar 21.

Division of Biostatistics and Epidemiology, Cincinnati Children's Hospital Medical Center, 3333 Burnet Ave, Cincinnati, OH 45229, United States.

This study develops a comprehensive method to assess seasonal influences on a longitudinal marker and compare estimates between cohorts. The method extends existing approaches by (i) combining a sine-cosine model of seasonality with a specialized covariance function for modeling longitudinal correlation; (ii) performing mediation analysis on a seasonality model. An example dataset and R code are provided. The bundle of methods is referred to as seasonality, mediation and comparison (SMAC). The case study described utilizes lung function as the marker observed on a cystic fibrosis cohort but SMAC can be used to evaluate other markers and in other disease contexts. Key aspects of customization are as follows.•This study introduces a novel seasonality model to fit trajectories of lung function decline and demonstrates how to compare this model to a conventional model in this context.•Steps required for mediation analyses in the seasonality model are shown.•The necessary calculations to compare seasonality models between cohorts, based on estimation coefficients, are derived in the study.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1016/j.mex.2021.101313DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8374306PMC
March 2021

Automated detection of substance use information from electronic health records for a pediatric population.

J Am Med Inform Assoc 2021 09;28(10):2116-2127

Department of Pediatrics, College of Medicine, University of Cincinnati, Cincinnati, Ohio, USA.

Objective: Substance use screening in adolescence is unstandardized and often documented in clinical notes, rather than in structured electronic health records (EHRs). The objective of this study was to integrate logic rules with state-of-the-art natural language processing (NLP) and machine learning technologies to detect substance use information from both structured and unstructured EHR data.

Materials And Methods: Pediatric patients (10-20 years of age) with any encounter between July 1, 2012, and October 31, 2017, were included (n = 3890 patients; 19 478 encounters). EHR data were extracted at each encounter, manually reviewed for substance use (alcohol, tobacco, marijuana, opiate, any use), and coded as lifetime use, current use, or family use. Logic rules mapped structured EHR indicators to screening results. A knowledge-based NLP system and a deep learning model detected substance use information from unstructured clinical narratives. System performance was evaluated using positive predictive value, sensitivity, negative predictive value, specificity, and area under the receiver-operating characteristic curve (AUC).

Results: The dataset included 17 235 structured indicators and 27 141 clinical narratives. Manual review of clinical narratives captured 94.0% of positive screening results, while structured EHR data captured 22.0%. Logic rules detected screening results from structured data with 1.0 and 0.99 for sensitivity and specificity, respectively. The knowledge-based system detected substance use information from clinical narratives with 0.86, 0.79, and 0.88 for AUC, sensitivity, and specificity, respectively. The deep learning model further improved detection capacity, achieving 0.88, 0.81, and 0.85 for AUC, sensitivity, and specificity, respectively. Finally, integrating predictions from structured and unstructured data achieved high detection capacity across all cases (0.96, 0.85, and 0.87 for AUC, sensitivity, and specificity, respectively).

Conclusions: It is feasible to detect substance use screening and results among pediatric patients using logic rules, NLP, and machine learning technologies.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1093/jamia/ocab116DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8449626PMC
September 2021

Quantitative disease risk scores from EHR with applications to clinical risk stratification and genetic studies.

NPJ Digit Med 2021 Jul 23;4(1):116. Epub 2021 Jul 23.

Department of Biostatistics, Columbia University, New York, NY, USA.

Labeling clinical data from electronic health records (EHR) in health systems requires extensive knowledge of human expert, and painstaking review by clinicians. Furthermore, existing phenotyping algorithms are not uniformly applied across large datasets and can suffer from inconsistencies in case definitions across different algorithms. We describe here quantitative disease risk scores based on almost unsupervised methods that require minimal input from clinicians, can be applied to large datasets, and alleviate some of the main weaknesses of existing phenotyping algorithms. We show applications to phenotypic data on approximately 100,000 individuals in eMERGE, and focus on several complex diseases, including Chronic Kidney Disease, Coronary Artery Disease, Type 2 Diabetes, Heart Failure, and a few others. We demonstrate that relative to existing approaches, the proposed methods have higher prediction accuracy, can better identify phenotypic features relevant to the disease under consideration, can perform better at clinical risk stratification, and can identify undiagnosed cases based on phenotypic features available in the EHR. Using genetic data from the eMERGE-seq panel that includes sequencing data for 109 genes on 21,363 individuals from multiple ethnicities, we also show how the new quantitative disease risk scores help improve the power of genetic association studies relative to the standard use of disease phenotypes. The results demonstrate the effectiveness of quantitative disease risk scores derived from rich phenotypic EHR databases to provide a more meaningful characterization of clinical risk for diseases of interest beyond the prevalent binary (case-control) classification.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1038/s41746-021-00488-3DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8302667PMC
July 2021

DeepImmuno: deep learning-empowered prediction and generation of immunogenic peptides for T-cell immunity.

Brief Bioinform 2021 11;22(6)

Cincinnati Children's Hospital Medical Center, USA.

Cytolytic T-cells play an essential role in the adaptive immune system by seeking out, binding and killing cells that present foreign antigens on their surface. An improved understanding of T-cell immunity will greatly aid in the development of new cancer immunotherapies and vaccines for life-threatening pathogens. Central to the design of such targeted therapies are computational methods to predict non-native peptides to elicit a T-cell response, however, we currently lack accurate immunogenicity inference methods. Another challenge is the ability to accurately simulate immunogenic peptides for specific human leukocyte antigen alleles, for both synthetic biological applications, and to augment real training datasets. Here, we propose a beta-binomial distribution approach to derive peptide immunogenic potential from sequence alone. We conducted systematic benchmarking of five traditional machine learning (ElasticNet, K-nearest neighbors, support vector machine, Random Forest and AdaBoost) and three deep learning models (convolutional neural network (CNN), Residual Net and graph neural network) using three independent prior validated immunogenic peptide collections (dengue virus, cancer neoantigen and SARS-CoV-2). We chose the CNN as the best prediction model, based on its adaptivity for small and large datasets and performance relative to existing methods. In addition to outperforming two highly used immunogenicity prediction algorithms, DeepImmuno-CNN correctly predicts which residues are most important for T-cell antigen recognition and predicts novel impacts of SARS-CoV-2 variants. Our independent generative adversarial network (GAN) approach, DeepImmuno-GAN, was further able to accurately simulate immunogenic peptides with physicochemical properties and immunogenicity predictions similar to that of real antigens. We provide DeepImmuno-CNN as source code and an easy-to-use web interface.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1093/bib/bbab160DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8135853PMC
November 2021

DeepImmuno: Deep learning-empowered prediction and generation of immunogenic peptides for T cell immunity.

bioRxiv 2020 Dec 24. Epub 2020 Dec 24.

Division of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA.

T-cells play an essential role in the adaptive immune system by seeking out, binding and destroying foreign antigens presented on the cell surface of diseased cells. An improved understanding of T-cell immunity will greatly aid in the development of new cancer immunotherapies and vaccines for life threatening pathogens. Central to the design of such targeted therapies are computational methods to predict non-native epitopes to elicit a T cell response, however, we currently lack accurate immunogenicity inference methods. Another challenge is the ability to accurately simulate immunogenic peptides for specific human leukocyte antigen (HLA) alleles, for both synthetic biological applications and to augment real training datasets. Here, we proposed a beta-binomial distribution approach to derive epitope immunogenic potential from sequence alone. We conducted systematic benchmarking of five traditional machine learning (ElasticNet, KNN, SVM, Random Forest, AdaBoost) and three deep learning models (CNN, ResNet, GNN) using three independent prior validated immunogenic peptide collections (dengue virus, cancer neoantigen and SARS-Cov-2). We chose the CNN model as the best prediction model based on its adaptivity for small and large datasets, and performance relative to existing methods. In addition to outperforming two highly used immunogenicity prediction algorithms, DeepHLApan and IEDB, DeepImmuno-CNN further correctly predicts which residues are most important for T cell antigen recognition. Our independent generative adversarial network (GAN) approach, DeepImmuno-GAN, was further able to accurately simulate immunogenic peptides with physiochemical properties and immunogenicity predictions similar to that of real antigens. We provide DeepImmuno-CNN as source code and an easy-to-use web interface.

Data Availability: DeepImmuno Python3 code is available at https://github.com/frankligy/DeepImmuno . The DeepImmuno web portal is available from https://deepimmuno.herokuapp.com . The data in this article is available in GitHub and supplementary materials.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1101/2020.12.24.424262DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC7781330PMC
December 2020

The Generalizability of a Medication Administration Discrepancy Detection System: Quantitative Comparative Analysis.

JMIR Med Inform 2020 Dec 2;8(12):e22031. Epub 2020 Dec 2.

Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH, United States.

Background: As a result of the overwhelming proportion of medication errors occurring each year, there has been an increased focus on developing medication error prevention strategies. Recent advances in electronic health record (EHR) technologies allow institutions the opportunity to identify medication administration error events in real time through computerized algorithms. MED.Safe, a software package comprising medication discrepancy detection algorithms, was developed to meet this need by performing an automated comparison of medication orders to medication administration records (MARs). In order to demonstrate generalizability in other care settings, software such as this must be tested and validated in settings distinct from the development site.

Objective: The purpose of this study is to determine the portability and generalizability of the MED.Safe software at a second site by assessing the performance and fit of the algorithms through comparison of discrepancy rates and other metrics across institutions.

Methods: The MED.Safe software package was executed on medication use data from the implementation site to generate prescribing ratios and discrepancy rates. A retrospective analysis of medication prescribing and documentation patterns was then performed on the results and compared to those from the development site to determine the algorithmic performance and fit. Variance in performance from the development site was further explored and characterized.

Results: Compared to the development site, the implementation site had lower audit/order ratios and higher MAR/(order + audit) ratios. The discrepancy rates on the implementation site were consistently higher than those from the development site. Three drivers for the higher discrepancy rates were alternative clinical workflow using orders with dosing ranges; a data extract, transfer, and load issue causing modified order data to overwrite original order values in the EHRs; and delayed EHR documentation of verbal orders. Opportunities for improvement were identified and applied using a software update, which decreased false-positive discrepancies and improved overall fit.

Conclusions: The execution of MED.Safe at a second site was feasible and effective in the detection of medication administration discrepancies. A comparison of medication ordering, administration, and discrepancy rates identified areas where MED.Safe could be improved through customization. One modification of MED.Safe through deployment of a software update improved the overall algorithmic fit at the implementation site. More flexible customizations to accommodate different clinical practice patterns could improve MED.Safe's fit at new sites.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.2196/22031DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC7744260PMC
December 2020

Integrating and Evaluating the Data Quality and Utility of Smart Pump Information in Detecting Medication Administration Errors: Evaluation Study.

JMIR Med Inform 2020 Sep 2;8(9):e19774. Epub 2020 Sep 2.

Division of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, United States.

Background: At present, electronic health records (EHRs) are the central focus of clinical informatics given their role as the primary source of clinical data. Despite their granularity, the EHR data heavily rely on manual input and are prone to human errors. Many other sources of data exist in the clinical setting, including digital medical devices such as smart infusion pumps. When incorporated with prescribing data from EHRs, smart pump records (SPRs) are capable of shedding light on actions that take place during the medication use process. However, harmoniz-ing the 2 sources is hindered by multiple technical challenges, and the data quality and utility of SPRs have not been fully realized.

Objective: This study aims to evaluate the quality and utility of SPRs incorporated with EHR data in detecting medication administration errors. Our overarching hypothesis is that SPRs would contribute unique information in the med-ication use process, enabling more comprehensive detection of discrepancies and potential errors in medication administration.

Methods: We evaluated the medication use process of 9 high-risk medications for patients admitted to the neonatal inten-sive care unit during a 1-year period. An automated algorithm was developed to align SPRs with their medica-tion orders in the EHRs using patient ID, medication name, and timestamp. The aligned data were manually re-viewed by a clinical research coordinator and 2 pediatric physicians to identify discrepancies in medication ad-ministration. The data quality of SPRs was assessed with the proportion of information that was linked to valid EHR orders. To evaluate their utility, we compared the frequency and severity of discrepancies captured by the SPR and EHR data, respectively. A novel concordance assessment was also developed to understand the detec-tion power and capabilities of SPR and EHR data.

Results: Approximately 70% of the SPRs contained valid patient IDs and medication names, making them feasible for data integration. After combining the 2 sources, the investigative team reviewed 2307 medication orders with 10,575 medication administration records (MARs) and 23,397 SPRs. A total of 321 MAR and 682 SPR dis-crepancies were identified, with vasopressors showing the highest discrepancy rates, followed by narcotics and total parenteral nutrition. Compared with EHR MARs, substantial dosing discrepancies were more commonly detectable using the SPRs. The concordance analysis showed little overlap between MAR and SPR discrepan-cies, with most discrepancies captured by the SPR data.

Conclusions: We integrated smart infusion pump information with EHR data to analyze the most error-prone phases of the medication lifecycle. The findings suggested that SPRs could be a more reliable data source for medication error detection. Ultimately, it is imperative to integrate SPR information with EHR data to fully detect and mitigate medication administration errors in the clinical setting.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.2196/19774DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC7495258PMC
September 2020

Finding warning markers: Leveraging natural language processing and machine learning technologies to detect risk of school violence.

Int J Med Inform 2020 07 25;139:104137. Epub 2020 Apr 25.

Department of Pediatrics, College of Medicine, University of Cincinnati, Cincinnati, OH, United States; Division of Child and Adolescent Psychiatry, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, United States.

Introduction: School violence has a far-reaching effect, impacting the entire school population including staff, students and their families. Among youth attending the most violent schools, studies have reported higher dropout rates, poor school attendance, and poor scholastic achievement. It was noted that the largest crime-prevention results occurred when youth at elevated risk were given an individualized prevention program. However, much work is needed to establish an effective approach to identify at-risk subjects.

Objective: In our earlier research, we developed a risk assessment program to interview subjects, identify risk and protective factors, and evaluate risk for school violence. This study focused on developing natural language processing (NLP) and machine learning technologies to automate the risk assessment process.

Material And Methods: We prospectively recruited 131 students with or without behavioral concerns from 89 schools between 05/01/2015 and 04/30/2018. The subjects were interviewed with two risk assessment scales and a questionnaire, and their risk of violence were determined by pediatric psychiatrists based on clinical judgment. Using NLP technologies, different types of linguistic features were extracted from the interview content. Machine learning classifiers were then applied to predict risk of school violence for individual subjects. A two-stage feature selection was implemented to identify violence-related predictors. The performance was validated on the psychiatrist-generated reference standard of risk levels, where positive predictive value (PPV), sensitivity (SEN), negative predictive value (NPV), specificity (SPEC) and area under the ROC curve (AUC) were assessed.

Results: Compared to subjects' sociodemographic information, use of linguistic features significantly improved classifiers' predictive performance (P < 0.01). The best-performing classifier with n-gram features achieved 86.5 %/86.5 %/85.7 %/85.7 %/94.0 % (PPV/SEN/NPV/SPEC/AUC) on the cross-validation set and 83.3 %/93.8 %/91.7 %/78.6 %/94.6 % (PPV/SEN/NPV/SPEC/AUC) on the test data. The feature selection process identified a set of predictors covering the discussion of subjects' thoughts, perspectives, behaviors, individual characteristics, peers and family dynamics, and protective factors.

Conclusions: By analyzing the content from subject interviews, the NLP and machine learning algorithms showed good capacity for detecting risk of school violence. The feature selection uncovered multiple warning markers that could deliver useful clinical insights to assist personalizing intervention. Consequently, the developed approach offered the promise of an accurate and scalable computerized screening service for preventing school violence.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1016/j.ijmedinf.2020.104137DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC7257261PMC
July 2020

Influences of environmental exposures on individuals living with cystic fibrosis.

Expert Rev Respir Med 2020 07 26;14(7):737-748. Epub 2020 Apr 26.

Eudowood Division of Pediatric Respiratory Sciences, Department of Pediatrics, Johns Hopkins University School of Medicine , Baltimore, MD, USA.

Introduction: Natural, social, and constructed environments play a critical role in the development and exacerbation of respiratory diseases. However, less is known regarding the influence of these environmental/community risk factors on the health of individuals living with cystic fibrosis (CF), compared to other pulmonary disorders.

Areas Covered: Here, we review current knowledge of environmental exposures related to CF, which suggests that environmental/community risk factors do interact with the respiratory tract to affect outcomes. Studies discussed in this review were identified in PubMed between March 2019 and March 2020. Although the limited data available do not suggest that avoiding potentially detrimental exposures other than secondhand smoke could improve outcomes, additional research incorporating novel markers of environmental exposures and community characteristics obtained at localized levels is needed.

Expert Opinion: As we outline, some environmental exposures and community characteristics are modifiable; if not by the individual, then by policy. We recommend a variety of strategies to advance understanding of environmental influences on CF disease progression.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1080/17476348.2020.1753507DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC7424635PMC
July 2020

Mining patient-specific and contextual data with machine learning technologies to predict cancellation of children's surgery.

Int J Med Inform 2019 09 8;129:234-241. Epub 2019 Jun 8.

College of Medicine, University of Cincinnati, Cincinnati, OH, USA; Department of Anesthesia, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA. Electronic address:

Background: Last-minute surgery cancellation represents a major wastage of resources and can cause significant inconvenience to patients. Our objectives in this study were: 1) To develop predictive models of last-minute surgery cancellation, utilizing machine learning technologies, from patient-specific and contextual data from two distinct pediatric surgical sites of a single institution; and 2) to identify specific key predictors that impact children's risk of day-of-surgery cancellation.

Methods And Findings: We extracted five-year datasets (2012-2017) from the Electronic Health Record at Cincinnati Children's Hospital Medical Center. By leveraging patient-specific information and contextual data, machine learning classifiers were developed to predict all patient-related cancellations and the most frequent four cancellation causes individually (patient illness, "no show," NPO violation and refusal to undergo surgery by either patient or family). Model performance was evaluated by the area under the receiver operating characteristic curve (AUC) using ten-fold cross-validation. The best performance for predicting all-cause surgery cancellation was generated by gradient-boosted logistic regression models, with AUC 0.781 (95% CI: [0.764,0.797]) and 0.740 (95% CI: [0.726,0.771]) for the two campuses. Of the four most frequent individual causes of cancellation, "no show" and NPO violation were predicted better than patient illness or patient/family refusal. Models showed good cross-campus generalizability (AUC: 0.725/0.735, when training on one site and testing on the other). To synthesize a human-oriented conceptualization of pediatric surgery cancellation, an iterative step-forward approach was applied to identify key predictors which may inform the design of future preventive interventions.

Conclusions: Our study demonstrated the capacity of machine learning models for predicting pediatric patients at risk of last-minute surgery cancellation and providing useful insight into root causes of cancellation. The approach offers the promise of targeted interventions to significantly decrease both healthcare costs and also families' negative experiences.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1016/j.ijmedinf.2019.06.007DOI Listing
September 2019

Development and Preliminary Evaluation of a Visual Annotation Tool to Rapidly Collect Expert-Annotated Weight Errors in Pediatric Growth Charts.

Stud Health Technol Inform 2019 Aug;264:853-857

Department of Biomedical Informatics, University of Cincinnati, Cincinnati, OH, USA.

Patient weights can be entered incorrectly into electronic health record (EHR) systems. These weight errors can cause significant patient harm especially in pediatrics where weight-based dosing is pervasively used. Determining weight errors through manual chart reviews is impractical in busy clinics, and current EHR alerts are rudimentary. To address these issues, we seek to develop an advanced algorithm to detect weight errors using supervised machine learning techniques. The critical first step is to collect labelled weight errors for algorithm training. In this paper, we designed and preliminarily evaluated a visual annotation tool using Agile software development to achieve the goal of supporting the rapid collection of expert-annotated weight errors. The design was based on the fact that weight errors are infrequent and medical experts can easily spot potential errors. The results show positive user feedback and prepared us for the formal user-centered evaluation as the next step.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.3233/SHTI190344DOI Listing
August 2019

A Real-Time Automated Patient Screening System for Clinical Trials Eligibility in an Emergency Department: Design and Evaluation.

JMIR Med Inform 2019 Jul 24;7(3):e14185. Epub 2019 Jul 24.

Cincinnati Children's Hospital Medical Center, Cincinnati, OH, United States.

Background: One critical hurdle for clinical trial recruitment is the lack of an efficient method for identifying subjects who meet the eligibility criteria. Given the large volume of data documented in electronic health records (EHRs), it is labor-intensive for the staff to screen relevant information, particularly within the time frame needed. To facilitate subject identification, we developed a natural language processing (NLP) and machine learning-based system, Automated Clinical Trial Eligibility Screener (ACTES), which analyzes structured data and unstructured narratives automatically to determine patients' suitability for clinical trial enrollment. In this study, we integrated the ACTES into clinical practice to support real-time patient screening.

Objective: This study aimed to evaluate ACTES's impact on the institutional workflow, prospectively and comprehensively. We hypothesized that compared with the manual screening process, using EHR-based automated screening would improve efficiency of patient identification, streamline patient recruitment workflow, and increase enrollment in clinical trials.

Methods: The ACTES was fully integrated into the clinical research coordinators' (CRC) workflow in the pediatric emergency department (ED) at Cincinnati Children's Hospital Medical Center. The system continuously analyzed EHR information for current ED patients and recommended potential candidates for clinical trials. Relevant patient eligibility information was presented in real time on a dashboard available to CRCs to facilitate their recruitment. To assess the system's effectiveness, we performed a multidimensional, prospective evaluation for a 12-month period, including a time-and-motion study, quantitative assessments of enrollment, and postevaluation usability surveys collected from the CRCs.

Results: Compared with manual screening, the use of ACTES reduced the patient screening time by 34% (P<.001). The saved time was redirected to other activities such as study-related administrative tasks (P=.03) and work-related conversations (P=.006) that streamlined teamwork among the CRCs. The quantitative assessments showed that automated screening improved the numbers of subjects screened, approached, and enrolled by 14.7%, 11.1%, and 11.1%, respectively, suggesting the potential of ACTES in streamlining recruitment workflow. Finally, the ACTES achieved a system usability scale of 80.0 in the postevaluation surveys, suggesting that it was a good computerized solution.

Conclusions: By leveraging NLP and machine learning technologies, the ACTES demonstrated good capacity for improving efficiency of patient identification. The quantitative assessments demonstrated the potential of ACTES in streamlining recruitment workflow and improving patient enrollment. The postevaluation surveys suggested that the system was a good computerized solution with satisfactory usability.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.2196/14185DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6685132PMC
July 2019

Data Challenges With Real-Time Safety Event Detection And Clinical Decision Support.

J Med Internet Res 2019 05 22;21(5):e13047. Epub 2019 May 22.

Department of Pediatrics, College of Medicine, University of Cincinnati, Cincinnati, OH, United States.

Background: The continued digitization and maturation of health care information technology has made access to real-time data easier and feasible for more health care organizations. With this increased availability, the promise of using data to algorithmically detect health care-related events in real-time has become more of a reality. However, as more researchers and clinicians utilize real-time data delivery capabilities, it has become apparent that simply gaining access to the data is not a panacea, and some unique data challenges have emerged to the forefront in the process.

Objective: The aim of this viewpoint was to highlight some of the challenges that are germane to real-time processing of health care system-generated data and the accurate interpretation of the results.

Methods: Distinct challenges related to the use and processing of real-time data for safety event detection were compiled and reported by several informatics and clinical experts at a quaternary pediatric academic institution. The challenges were collated from the experiences of the researchers implementing real-time event detection on more than half a dozen distinct projects. The challenges have been presented in a challenge category-specific challenge-example format.

Results: In total, 8 major types of challenge categories were reported, with 13 specific challenges and 9 specific examples detailed to provide a context for the challenges. The examples reported are anchored to a specific project using medication order, medication administration record, and smart infusion pump data to detect discrepancies and errors between the 3 datasets.

Conclusions: The use of real-time data to drive safety event detection and clinical decision support is extremely powerful, but it presents its own set of challenges that include data quality and technical complexity. These challenges must be recognized and accommodated for if the full promise of accurate, real-time safety event clinical decision support is to be realized.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.2196/13047DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6549472PMC
May 2019

A Comparison of Existing Methods to Detect Weight Data Errors in a Pediatric Academic Medical Center.

AMIA Annu Symp Proc 2018 5;2018:1103-1109. Epub 2018 Dec 5.

Division of Hospital Medicine, Cincinnati Children's Hospital Medical Center, OH.

Dosing errors due to erroneous body weight entry can be mitigated through algorithms designed to detect anomalies in weight patterns. To prepare for the development of a new algorithm for weight-entry error detection, we compared methods for detecting weight anomalies to human annotation, including a regression-based method employed in a real-time web service. Using a random sample of 4,000 growth charts, annotators identified clinically important anomalies with good inter-rater reliability. Performance of the three detection algorithms was variable, with the best performance from the algorithm that takes into account weights collected after the anomaly was recorded. All methods were highly specific, but positive predictive value ranged from < 5% to over 82%. There were 203 records of missed errors, but all of these were either due to no prior data points or errors too small to be clinically significant. This analysis illustrates the need for better weight-entry error detection algorithms.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6371361PMC
November 2019

A Time-and-Motion Study of Clinical Trial Eligibility Screening in a Pediatric Emergency Department.

Pediatr Emerg Care 2019 Dec;35(12):868-873

Department of Biomedical Informatics.

Objective: Challenges with efficient patient recruitment including sociotechnical barriers for clinical trials are major barriers to the timely and efficacious conduct of translational studies. We conducted a time-and-motion study to investigate the workflow of clinical trial enrollment in a pediatric emergency department.

Methods: We observed clinical research coordinators during 3 clinically staffed shifts. One clinical research coordinator was shadowed at a time. Tasks were marked in 30-second intervals and annotated to include patient screening, patient contact, performing procedures, and physician contact. Statistical analysis was conducted on the patient enrollment activities.

Results: We conducted fifteen 120-minute observations from December 12, 2013, to January 3, 2014 and shadowed 8 clinical research coordinators. Patient screening took 31.62% of their time, patient contact took 18.67%, performing procedures took 17.6%, physician contact was 1%, and other activities took 31.0%.

Conclusions: Screening patients for eligibility constituted the most time. Automated screening methods could help reduce this time. The findings suggest improvement areas in recruitment planning to increase the efficiency of clinical trial enrollment.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1097/PEC.0000000000001592DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6445787PMC
December 2019

Automated Risk Assessment for School Violence: a Pilot Study.

Psychiatr Q 2018 12;89(4):817-828

University of Cincinnati, Cincinnati, USA.

School violence has increased over the past ten years. This study evaluated students using a more standard and sensitive method to help identify students who are at high risk for school violence. 103 participants were recruited through Cincinnati Children's Hospital Medical Center (CCHMC) from psychiatry outpatient clinics, the inpatient units, and the emergency department. Participants (ages 12-18) were active students in 74 traditional schools (i.e. non-online education). Collateral information was gathered from guardians before participants were evaluated. School risk evaluations were performed with each participant, and audio recordings from the evaluations were later transcribed and manually annotated. The BRACHA (School Version) and the School Safety Scale (SSS), both 14-item scales, were used. A template of open-ended questions was also used. This analysis included 103 participants who were recruited from 74 different schools. Of the 103 students evaluated, 55 were found to be moderate to high risk and 48 were found to be low risk based on the paper risk assessments including the BRACHA and SSS. Both the BRACHA and the SSS were highly correlated with risk of violence to others (Pearson correlations>0.82). There were significant differences in BRACHA and SSS total scores between low risk and high risk to others groups (p-values <0.001 under unpaired t-test). In particular, there were significant differences in individual SSS items between the two groups (p-value <0.001). Of these items, Previous Violent Behavior (Pearson Correlation = 0.80), Impulsivity (0.69), School Problems (0.64), and Negative Attitudes (0.61) were positively correlated with risk to others. The novel machine learning algorithm achieved an AUC of 91.02% when using the interview content to predict risk of school violence, and the AUC increased to 91.45% when demographic and socioeconomic data were added. Our study indicates that the BRACHA and SSS are clinically useful for assessing risk for school violence. The machine learning algorithm was highly accurate in assessing school violence risk.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1007/s11126-018-9581-8DOI Listing
December 2018

Towards phenotyping stroke: Leveraging data from a large-scale epidemiological study to detect stroke diagnosis.

PLoS One 2018 14;13(2):e0192586. Epub 2018 Feb 14.

Department of Neurology and Rehabilitation Medicine, University of Cincinnati, Cincinnati, Ohio, United States of America.

Objective: 1) To develop a machine learning approach for detecting stroke cases and subtypes from hospitalization data, 2) to assess algorithm performance and predictors on real-world data collected by a large-scale epidemiology study in the US; and 3) to identify directions for future development of high-precision stroke phenotypic signatures.

Materials And Methods: We utilized 8,131 hospitalization events (ICD-9 codes 430-438) collected from the Greater Cincinnati/Northern Kentucky Stroke Study in 2005 and 2010. Detailed information from patients' medical records was abstracted for each event by trained research nurses. By analyzing the broad list of demographic and clinical variables, the machine learning algorithms predicted whether an event was a stroke case and, if so, the stroke subtype. The performance was validated on gold-standard labels adjudicated by stroke physicians, and results were compared with stroke classifications based on ICD-9 discharge codes, as well as labels determined by study nurses.

Results: The best performing machine learning algorithm achieved a performance of 88.57%/93.81%/92.80%/93.30%/89.84%/98.01% (accuracy/precision/recall/F-measure/area under ROC curve/area under precision-recall curve) on stroke case detection. For detecting stroke subtypes, the algorithm yielded an overall accuracy of 87.39% and greater than 85% precision on individual subtypes. The machine learning algorithms significantly outperformed the ICD-9 method on all measures (P value<0.001). Their performance was comparable to that of study nurses, with better tradeoff between precision and recall. The feature selection uncovered a subset of predictive variables that could facilitate future development of effective stroke phenotyping algorithms.

Discussion And Conclusions: By analyzing a broad array of patient data, the machine learning technologies held promise for improving detection of stroke diagnosis, thus unlocking high statistical power for subsequent genetic and genomic studies.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0192586PLOS
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5812624PMC
April 2018

Designing and evaluating an automated system for real-time medication administration error detection in a neonatal intensive care unit.

J Am Med Inform Assoc 2018 05;25(5):555-563

Department of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA.

Background: Timely identification of medication administration errors (MAEs) promises great benefits for mitigating medication errors and associated harm. Despite previous efforts utilizing computerized methods to monitor medication errors, sustaining effective and accurate detection of MAEs remains challenging. In this study, we developed a real-time MAE detection system and evaluated its performance prior to system integration into institutional workflows.

Methods: Our prospective observational study included automated MAE detection of 10 high-risk medications and fluids for patients admitted to the neonatal intensive care unit at Cincinnati Children's Hospital Medical Center during a 4-month period. The automated system extracted real-time medication use information from the institutional electronic health records and identified MAEs using logic-based rules and natural language processing techniques. The MAE summary was delivered via a real-time messaging platform to promote reduction of patient exposure to potential harm. System performance was validated using a physician-generated gold standard of MAE events, and results were compared with those of current practice (incident reporting and trigger tools).

Results: Physicians identified 116 MAEs from 10 104 medication administrations during the study period. Compared to current practice, the sensitivity with automated MAE detection was improved significantly from 4.3% to 85.3% (P = .009), with a positive predictive value of 78.0%. Furthermore, the system showed potential to reduce patient exposure to harm, from 256 min to 35 min (P < .001).

Conclusions: The automated system demonstrated improved capacity for identifying MAEs while guarding against alert fatigue. It also showed promise for reducing patient exposure to potential harm following MAE events.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1093/jamia/ocx156DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6018990PMC
May 2018

Using Health Information Technology to Improve Safety in Neonatal Care: A Systematic Review of the Literature.

Clin Perinatol 2017 09;44(3):583-616

Department of Pediatrics, James M. Anderson Center for Health Systems Excellence, Cincinnati Children's Hospital Medical Center, 3333 Burnet Avenue, MLC 7014, Cincinnati, OH 45229, USA.

Health information technology (HIT) interventions may improve neonatal patient safety but may also introduce new errors. The objective of this review was to evaluate the evidence for use of HIT interventions to improve safety in neonatal care. Evidence for improvement exists for interventions like computerized provider order entry in the neonatal population, but is lacking for several other interventions. Many unique applications of HIT are emerging as technology and use of the electronic health record expands. Future research should focus on the impact of these interventions in the neonatal population.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1016/j.clp.2017.04.003DOI Listing
September 2017

Leveraging Food and Drug Administration Adverse Event Reports for the Automated Monitoring of Electronic Health Records in a Pediatric Hospital.

Biomed Inform Insights 2017 8;9:1178222617713018. Epub 2017 Jun 8.

Department of Biomedical Informatics, Cincinnati Children's Hospital Medical Center (CCHMC), Cincinnati, OH, USA.

The objective of this study was to determine whether the Food and Drug Administration's Adverse Event Reporting System (FAERS) data set could serve as the basis of automated electronic health record (EHR) monitoring for the adverse drug reaction (ADR) subset of adverse drug events. We retrospectively collected EHR entries for 71 909 pediatric inpatient visits at Cincinnati Children's Hospital Medical Center. Natural language processing (NLP) techniques were used to identify positive diseases/disorders and signs/symptoms (DDSSs) from the patients' clinical narratives. We downloaded all FAERS reports submitted by medical providers and extracted the reported drug-DDSS pairs. For each patient, we aligned the drug-DDSS pairs extracted from their clinical notes with the corresponding drug-DDSS pairs from the FAERS data set to identify Drug-Reaction Pair Sentences (DRPSs). The DRPSs were processed by NLP techniques to identify ADR-related DRPSs. We used clinician annotated, real-world EHR data as reference standard to evaluate the proposed algorithm. During evaluation, the algorithm achieved promising performance and showed great potential in identifying ADRs accurately for pediatric patients.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1177/1178222617713018DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5467704PMC
June 2017

A Pilot Study on Developing a Standardized and Sensitive School Violence Risk Assessment with Manual Annotation.

Psychiatr Q 2017 09;88(3):447-457

Cincinnati Childrens's Hospital Medical Center and the University of Cincinnati, Cincinnati, USA.

School violence has increased over the past decade and innovative, sensitive, and standardized approaches to assess school violence risk are needed. In our current feasibility study, we initialized a standardized, sensitive, and rapid school violence risk approach with manual annotation. Manual annotation is the process of analyzing a student's transcribed interview to extract relevant information (e.g., key words) to school violence risk levels that are associated with students' behaviors, attitudes, feelings, use of technology (social media and video games), and other activities. In this feasibility study, we first implemented school violence risk assessments to evaluate risk levels by interviewing the student and parent separately at the school or the hospital to complete our novel school safety scales. We completed 25 risk assessments, resulting in 25 transcribed interviews of 12-18 year olds from 15 schools in Ohio and Kentucky. We then analyzed structured professional judgments, language, and patterns associated with school violence risk levels by using manual annotation and statistical methodology. To analyze the student interviews, we initiated the development of an annotation guideline to extract key information that is associated with students' behaviors, attitudes, feelings, use of technology and other activities. Statistical analysis was applied to associate the significant categories with students' risk levels to identify key factors which will help with developing action steps to reduce risk. In a future study, we plan to recruit more subjects in order to fully develop the manual annotation which will result in a more standardized and sensitive approach to school violence assessments.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1007/s11126-016-9458-7DOI Listing
September 2017

Electronic Health Record Based Algorithm to Identify Patients with Autism Spectrum Disorder.

PLoS One 2016 29;11(7):e0159621. Epub 2016 Jul 29.

Harvard Medical School, Pediatrics, Boston, Massachusetts, United States of America.

Objective: Cohort selection is challenging for large-scale electronic health record (EHR) analyses, as International Classification of Diseases 9th edition (ICD-9) diagnostic codes are notoriously unreliable disease predictors. Our objective was to develop, evaluate, and validate an automated algorithm for determining an Autism Spectrum Disorder (ASD) patient cohort from EHR. We demonstrate its utility via the largest investigation to date of the co-occurrence patterns of medical comorbidities in ASD.

Methods: We extracted ICD-9 codes and concepts derived from the clinical notes. A gold standard patient set was labeled by clinicians at Boston Children's Hospital (BCH) (N = 150) and Cincinnati Children's Hospital and Medical Center (CCHMC) (N = 152). Two algorithms were created: (1) rule-based implementing the ASD criteria from Diagnostic and Statistical Manual of Mental Diseases 4th edition, (2) predictive classifier. The positive predictive values (PPV) achieved by these algorithms were compared to an ICD-9 code baseline. We clustered the patients based on grouped ICD-9 codes and evaluated subgroups.

Results: The rule-based algorithm produced the best PPV: (a) BCH: 0.885 vs. 0.273 (baseline); (b) CCHMC: 0.840 vs. 0.645 (baseline); (c) combined: 0.864 vs. 0.460 (baseline). A validation at Children's Hospital of Philadelphia yielded 0.848 (PPV). Clustering analyses of comorbidities on the three-site large cohort (N = 20,658 ASD patients) identified psychiatric, developmental, and seizure disorder clusters.

Conclusions: In a large cross-institutional cohort, co-occurrence patterns of comorbidities in ASDs provide further hypothetical evidence for distinct courses in ASD. The proposed automated algorithms for cohort selection open avenues for other large-scale EHR studies and individualized treatment of ASD.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0159621PLOS
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4966969PMC
August 2017

Developing an Algorithm to Detect Early Childhood Obesity in Two Tertiary Pediatric Medical Centers.

Appl Clin Inform 2016 07 20;7(3):693-706. Epub 2016 Jul 20.

Todd Lingren, Cincinnati Children's Hospital Medical Center, Biomedical Informatics, 3333 Burnet Avenue, MLC 7024 Cincinnati, OH 45229-3039, Phone: 513-803-9032, Fax: 513-636-2056, Email:

Objective: The objective of this study is to develop an algorithm to accurately identify children with severe early onset childhood obesity (ages 1-5.99 years) using structured and unstructured data from the electronic health record (EHR).

Introduction: Childhood obesity increases risk factors for cardiovascular morbidity and vascular disease. Accurate definition of a high precision phenotype through a standardize tool is critical to the success of large-scale genomic studies and validating rare monogenic variants causing severe early onset obesity.

Data And Methods: Rule based and machine learning based algorithms were developed using structured and unstructured data from two EHR databases from Boston Children's Hospital (BCH) and Cincinnati Children's Hospital and Medical Center (CCHMC). Exclusion criteria including medications or comorbid diagnoses were defined. Machine learning algorithms were developed using cross-site training and testing in addition to experimenting with natural language processing features.

Results: Precision was emphasized for a high fidelity cohort. The rule-based algorithm performed the best overall, 0.895 (CCHMC) and 0.770 (BCH). The best feature set for machine learning employed Unified Medical Language System (UMLS) concept unique identifiers (CUIs), ICD-9 codes, and RxNorm codes.

Conclusions: Detecting severe early childhood obesity is essential for the intervention potential in children at the highest long-term risk of developing comorbidities related to obesity and excluding patients with underlying pathological and non-syndromic causes of obesity assists in developing a high-precision cohort for genetic study. Further such phenotyping efforts inform future practical application in health care environments utilizing clinical decision support.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.4338/ACI-2016-01-RA-0015DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5052543PMC
July 2016

Will they participate? Predicting patients' response to clinical trial invitations in a pediatric emergency department.

J Am Med Inform Assoc 2016 07 27;23(4):671-80. Epub 2016 Apr 27.

Department of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, Cincinnati, OH 45229-3039, USA Division of Emergency Medicine, Cincinnati Children's Hospital Medical Center, Cincinnati, OH 45229-3039, USA.

Objective: (1) To develop an automated algorithm to predict a patient's response (ie, if the patient agrees or declines) before he/she is approached for a clinical trial invitation; (2) to assess the algorithm performance and the predictors on real-world patient recruitment data for a diverse set of clinical trials in a pediatric emergency department; and (3) to identify directions for future studies in predicting patients' participation response.

Materials And Methods: We collected 3345 patients' response to trial invitations on 18 clinical trials at one center that were actively enrolling patients between January 1, 2010 and December 31, 2012. In parallel, we retrospectively extracted demographic, socioeconomic, and clinical predictors from multiple sources to represent the patients' profiles. Leveraging machine learning methodology, the automated algorithms predicted participation response for individual patients and identified influential features associated with their decision-making. The performance was validated on the collection of actual patient response, where precision, recall, F-measure, and area under the ROC curve were assessed.

Results: Compared to the random response predictor that simulated the current practice, the machine learning algorithms achieved significantly better performance (Precision/Recall/F-measure/area under the ROC curve: 70.82%/92.02%/80.04%/72.78% on 10-fold cross validation and 71.52%/92.68%/80.74%/75.74% on the test set). By analyzing the significant features output by the algorithms, the study confirmed several literature findings and identified challenges that could be mitigated to optimize recruitment.

Conclusion: By exploiting predictive variables from multiple sources, we demonstrated that machine learning algorithms have great potential in improving the effectiveness of the recruitment process by automatically predicting patients' participation response to trial invitations.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1093/jamia/ocv216DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4926740PMC
July 2016

Desiderata for computable representations of electronic health records-driven phenotype algorithms.

J Am Med Inform Assoc 2015 Nov 5;22(6):1220-30. Epub 2015 Sep 5.

Department of Biomedical Informatics, Vanderbilt University, Nashville, TN, USA.

Background: Electronic health records (EHRs) are increasingly used for clinical and translational research through the creation of phenotype algorithms. Currently, phenotype algorithms are most commonly represented as noncomputable descriptive documents and knowledge artifacts that detail the protocols for querying diagnoses, symptoms, procedures, medications, and/or text-driven medical concepts, and are primarily meant for human comprehension. We present desiderata for developing a computable phenotype representation model (PheRM).

Methods: A team of clinicians and informaticians reviewed common features for multisite phenotype algorithms published in PheKB.org and existing phenotype representation platforms. We also evaluated well-known diagnostic criteria and clinical decision-making guidelines to encompass a broader category of algorithms.

Results: We propose 10 desired characteristics for a flexible, computable PheRM: (1) structure clinical data into queryable forms; (2) recommend use of a common data model, but also support customization for the variability and availability of EHR data among sites; (3) support both human-readable and computable representations of phenotype algorithms; (4) implement set operations and relational algebra for modeling phenotype algorithms; (5) represent phenotype criteria with structured rules; (6) support defining temporal relations between events; (7) use standardized terminologies and ontologies, and facilitate reuse of value sets; (8) define representations for text searching and natural language processing; (9) provide interfaces for external software algorithms; and (10) maintain backward compatibility.

Conclusion: A computable PheRM is needed for true phenotype portability and reliability across different EHR products and healthcare systems. These desiderata are a guide to inform the establishment and evolution of EHR phenotype algorithm authoring platforms and languages.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1093/jamia/ocv112DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4639716PMC
November 2015

Automated detection of medication administration errors in neonatal intensive care.

J Biomed Inform 2015 Oct 17;57:124-33. Epub 2015 Jul 17.

Division of Neonatology, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, United States. Electronic address:

Objective: To improve neonatal patient safety through automated detection of medication administration errors (MAEs) in high alert medications including narcotics, vasoactive medication, intravenous fluids, parenteral nutrition, and insulin using the electronic health record (EHR); to evaluate rates of MAEs in neonatal care; and to compare the performance of computerized algorithms to traditional incident reporting for error detection.

Methods: We developed novel computerized algorithms to identify MAEs within the EHR of all neonatal patients treated in a level four neonatal intensive care unit (NICU) in 2011 and 2012. We evaluated the rates and types of MAEs identified by the automated algorithms and compared their performance to incident reporting. Performance was evaluated by physician chart review.

Results: In the combined 2011 and 2012 NICU data sets, the automated algorithms identified MAEs at the following rates: fentanyl, 0.4% (4 errors/1005 fentanyl administration records); morphine, 0.3% (11/4009); dobutamine, 0 (0/10); and milrinone, 0.3% (5/1925). We found higher MAE rates for other vasoactive medications including: dopamine, 11.6% (5/43); epinephrine, 10.0% (289/2890); and vasopressin, 12.8% (54/421). Fluid administration error rates were similar: intravenous fluids, 3.2% (273/8567); parenteral nutrition, 3.2% (649/20124); and lipid administration, 1.3% (203/15227). We also found 13 insulin administration errors with a resulting rate of 2.9% (13/456). MAE rates were higher for medications that were adjusted frequently and fluids administered concurrently. The algorithms identified many previously unidentified errors, demonstrating significantly better sensitivity (82% vs. 5%) and precision (70% vs. 50%) than incident reporting for error recognition.

Conclusions: Automated detection of medication administration errors through the EHR is feasible and performs better than currently used incident reporting systems. Automated algorithms may be useful for real-time error identification and mitigation.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1016/j.jbi.2015.07.012DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4715992PMC
October 2015

An end-to-end hybrid algorithm for automated medication discrepancy detection.

BMC Med Inform Decis Mak 2015 May 6;15:37. Epub 2015 May 6.

Department of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, 3333 Burnet Avenue, MLC 7024, Cincinnati, OH, 45229-3039, USA.

Background: In this study we implemented and developed state-of-the-art machine learning (ML) and natural language processing (NLP) technologies and built a computerized algorithm for medication reconciliation. Our specific aims are: (1) to develop a computerized algorithm for medication discrepancy detection between patients' discharge prescriptions (structured data) and medications documented in free-text clinical notes (unstructured data); and (2) to assess the performance of the algorithm on real-world medication reconciliation data.

Methods: We collected clinical notes and discharge prescription lists for all 271 patients enrolled in the Complex Care Medical Home Program at Cincinnati Children's Hospital Medical Center between 1/1/2010 and 12/31/2013. A double-annotated, gold-standard set of medication reconciliation data was created for this collection. We then developed a hybrid algorithm consisting of three processes: (1) a ML algorithm to identify medication entities from clinical notes, (2) a rule-based method to link medication names with their attributes, and (3) a NLP-based, hybrid approach to match medications with structured prescriptions in order to detect medication discrepancies. The performance was validated on the gold-standard medication reconciliation data, where precision (P), recall (R), F-value (F) and workload were assessed.

Results: The hybrid algorithm achieved 95.0%/91.6%/93.3% of P/R/F on medication entity detection and 98.7%/99.4%/99.1% of P/R/F on attribute linkage. The medication matching achieved 92.4%/90.7%/91.5% (P/R/F) on identifying matched medications in the gold-standard and 88.6%/82.5%/85.5% (P/R/F) on discrepant medications. By combining all processes, the algorithm achieved 92.4%/90.7%/91.5% (P/R/F) and 71.5%/65.2%/68.2% (P/R/F) on identifying the matched and the discrepant medications, respectively. The error analysis on algorithm outputs identified challenges to be addressed in order to improve medication discrepancy detection.

Conclusion: By leveraging ML and NLP technologies, an end-to-end, computerized algorithm achieves promising outcome in reconciling medications between clinical notes and discharge prescriptions.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1186/s12911-015-0160-8DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4427951PMC
May 2015
-->