Publications by authors named "Chunhua Weng"

240 Publications

A systematic review on natural language processing systems for eligibility prescreening in clinical research.

J Am Med Inform Assoc 2021 Nov 2. Epub 2021 Nov 2.

School of Nursing, Columbia University, New York, New York, USA.

Objective: We conducted a systematic review to assess the effect of natural language processing (NLP) systems in improving the accuracy and efficiency of eligibility prescreening during the clinical research recruitment process.

Materials And Methods: Guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) standards of quality for reporting systematic reviews, a protocol for study eligibility was developed a priori and registered in the PROSPERO database. Using predetermined inclusion criteria, studies published from database inception through February 2021 were identified from 5 databases. The Joanna Briggs Institute Critical Appraisal Checklist for Quasi-experimental Studies was adapted to determine the study quality and the risk of bias of the included articles.

Results: Eleven studies representing 8 unique NLP systems met the inclusion criteria. These studies demonstrated moderate study quality and exhibited heterogeneity in the study design, setting, and intervention type. All 11 studies evaluated the NLP system's performance for identifying eligible participants; 7 studies evaluated the system's impact on time efficiency; 4 studies evaluated the system's impact on workload; and 2 studies evaluated the system's impact on recruitment.

Discussion: NLP systems in clinical research eligibility prescreening are an understudied but promising field that requires further research to assess its impact on real-world adoption. Future studies should be centered on continuing to develop and evaluate relevant NLP systems to improve enrollment into clinical studies.

Conclusion: Understanding the role of NLP systems in improving eligibility prescreening is critical to the advancement of clinical research recruitment.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1093/jamia/ocab228DOI Listing
November 2021

A Retrospective Analysis of COVID-19 mRNA Vaccine Breakthrough Infections - Risk Factors and Vaccine Effectiveness.

medRxiv 2021 Oct 7. Epub 2021 Oct 7.

Importance: Little is known about COVID vaccine breakthrough infections and their risk factors.

Objective: To identify risk factors associated with COVID-19 breakthrough infections among vaccinated individuals and to reassess the effectiveness of COVID-19 vaccination against severe outcomes using real-world data.

Design Setting And Participants: We conducted a series of observational retrospective analyses using the electronic health records (EHRs) of Columbia University Irving Medical Center/New York Presbyterian (CUIMC/NYP) up to September 21, 2021. New York adult residence with PCR test records were included in this analysis.

Main Outcomes And Measures: Poisson regression was used to assess the association between breakthrough infection rate in vaccinated individuals and multiple risk factors - including vaccine brand, demographics, and underlying conditions - while adjusting for calendar month, prior number of visits and observational days. Logistic regression was used to assess the association between vaccine administration and infection rate by comparing a vaccinated cohort to a historically matched cohort in the pre-vaccinated period. Infection incident rate was also compared between vaccinated individuals and longitudinally matched unvaccinated individuals. Cox regression was used to estimate the association of the vaccine and COVID-19 associated severe outcomes by comparing breakthrough cohort and two matched unvaccinated infection cohorts.

Results: Individuals vaccinated with Pfizer/BNT162b2 (IRR against Moderna/mRNA-1273 [95% CI]: 1.66 [1.17 - 2.35]); were male (1.47 [1.11 - 1.94%]); and had compromised immune systems (1.48 [1.09 - 2.00]) were at the highest risk for breakthrough infections. Vaccinated individuals had a significant lower infection rate among all subgroups. An increased incidence rate was found in both vaccines over the time. Among individuals infected with COVID-19, vaccination significantly reduced the risk of death (adj. HR: 0.20 [0.08 - 0.49]).

Conclusion And Relevance: While we found both mRNA vaccines were effective, Moderna/mRNA-1273 had a lower incidence rate of breakthrough infections. Both vaccines had increased incidence rates over the time. Immunocompromised individuals were among the highest risk groups experiencing breakthrough infections. Given the rapidly changing nature of the SARS-CoV-2, continued monitoring and a generalizable analysis pipeline are warranted to inform quick updates on vaccine effectiveness in real time.

Key Points: What risk factors contribute to COVID-19 breakthrough infections among mRNA vaccinated individuals? How do clinical outcomes differ between vaccinated but still SARS-CoV-2 infected individuals and non-vaccinated, infected individuals? This retrospective study uses CUIMC/NYP EHR data up to September 21, 2021. Individuals who were vaccinated with Pfizer/BNT162b2, male, and had compromised immune systems had significantly higher incidence rate ratios of breakthrough infections. Comparing demographically matched pre-vaccinated and unvaccinated individuals, vaccinated individuals had a lower incidence rate of SARS-CoV-2 infection among all subgroups. Leveraging real-world EHR data provides insight on who may optimally benefit from a booster COVID-19 vaccination.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1101/2021.10.05.21264583DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8509087PMC
October 2021

The potential role of EHR data in optimizing eligibility criteria definition for cardiovascular outcome trials.

Int J Med Inform 2021 12 25;156:104587. Epub 2021 Sep 25.

Department of Biomedical Informatics, Columbia University, New York, NY 10032, USA. Electronic address:

Background: Cardiovascular outcome trials (CVOTs) include patients with high risks for cardiovascular events based on specific inclusion criteria. Little is known about the impact of such inclusion criteria on patient accrual and the incidence rate of cardiovascular events.

Materials And Methods: We evaluated the impact of criteria on the accrual and the number of cardiovascular events in a cohort of 1544 diabetes patients identified from the clinical data warehouse of New York Presbyterian Hospital / Columbia University Irving Medical Center.

Results: The highest incidence rate of the composite events (i.e., cardiovascular mortality, stroke, and myocardial infarction) was observed when the inclusion criteria seek patients with underlying cardiovascular diseases or age ≥ 60 with at least two of the risk factors including duration of diabetes, hypertension, dyslipidemia, smoking status, and albuminuria.

Conclusion: Our study shows that the electronic health records could be utilized to optimize the inclusion criteria while balancing study inclusiveness and number of events.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1016/j.ijmedinf.2021.104587DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8571001PMC
December 2021

Columbia Open Health Data for COVID-19 Research: Database Analysis.

J Med Internet Res 2021 09 30;23(9):e31122. Epub 2021 Sep 30.

Columbia University, New York, NY, United States.

Background: COVID-19 has threatened the health of tens of millions of people all over the world. Massive research efforts have been made in response to the COVID-19 pandemic. Utilization of clinical data can accelerate these research efforts to combat the pandemic since important characteristics of the patients are often found by examining the clinical data. Publicly accessible clinical data on COVID-19, however, remain limited despite the immediate need.

Objective: To provide shareable clinical data to catalyze COVID-19 research, we present Columbia Open Health Data for COVID-19 Research (COHD-COVID), a publicly accessible database providing clinical concept prevalence, clinical concept co-occurrence, and clinical symptom prevalence for hospitalized patients with COVID-19. COHD-COVID also provides data on hospitalized patients with influenza and general hospitalized patients as comparator cohorts.

Methods: The data used in COHD-COVID were obtained from NewYork-Presbyterian/Columbia University Irving Medical Center's electronic health records database. Condition, drug, and procedure concepts were obtained from the visits of identified patients from the cohorts. Rare concepts were excluded, and the true concept counts were perturbed using Poisson randomization to protect patient privacy. Concept prevalence, concept prevalence ratio, concept co-occurrence, and symptom prevalence were calculated using the obtained concepts.

Results: Concept prevalence and concept prevalence ratio analyses showed the clinical characteristics of the COVID-19 cohorts, confirming the well-known characteristics of COVID-19 (eg, acute lower respiratory tract infection and cough). The concepts related to the well-known characteristics of COVID-19 recorded high prevalence and high prevalence ratio in the COVID-19 cohort compared to the hospitalized influenza cohort and general hospitalized cohort. Concept co-occurrence analyses showed potential associations between specific concepts. In case of acute lower respiratory tract infection in the COVID-19 cohort, a high co-occurrence ratio was obtained with COVID-19-related concepts and commonly used drugs (eg, disease due to coronavirus and acetaminophen). Symptom prevalence analysis indicated symptom-level characteristics of the cohorts and confirmed that well-known symptoms of COVID-19 (eg, fever, cough, and dyspnea) showed higher prevalence than the hospitalized influenza cohort and the general hospitalized cohort.

Conclusions: We present COHD-COVID, a publicly accessible database providing useful clinical data for hospitalized patients with COVID-19, hospitalized patients with influenza, and general hospitalized patients. We expect COHD-COVID to provide researchers and clinicians quantitative measures of COVID-19-related clinical features to better understand and combat the pandemic.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.2196/31122DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8485985PMC
September 2021

A Framework for Systematic Assessment of Clinical Trial Population Representativeness Using Electronic Health Records Data.

Appl Clin Inform 2021 08 8;12(4):816-825. Epub 2021 Sep 8.

Department of Biomedical Informatics, Columbia University, New York, New York, United States.

Background: Clinical trials are the gold standard for generating robust medical evidence, but clinical trial results often raise generalizability concerns, which can be attributed to the lack of population representativeness. The electronic health records (EHRs) data are useful for estimating the population representativeness of clinical trial study population.

Objectives: This research aims to estimate the population representativeness of clinical trials systematically using EHR data during the early design stage.

Methods: We present an end-to-end analytical framework for transforming free-text clinical trial eligibility criteria into executable database queries conformant with the Observational Medical Outcomes Partnership Common Data Model and for systematically quantifying the population representativeness for each clinical trial.

Results: We calculated the population representativeness of 782 novel coronavirus disease 2019 (COVID-19) trials and 3,827 type 2 diabetes mellitus (T2DM) trials in the United States respectively using this framework. With the use of overly restrictive eligibility criteria, 85.7% of the COVID-19 trials and 30.1% of T2DM trials had poor population representativeness.

Conclusion: This research demonstrates the potential of using the EHR data to assess the clinical trials population representativeness, providing data-driven metrics to inform the selection and optimization of eligibility criteria.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1055/s-0041-1733846DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8426045PMC
August 2021

A Comparison between Human and NLP-based Annotation of Clinical Trial Eligibility Criteria Text Using The OMOP Common Data Model.

AMIA Jt Summits Transl Sci Proc 2021;2021:394-403. Epub 2021 May 17.

Department of Biomedical Informatics, Columbia University, New York, NY, USA.

Human annotations are the established gold standard for evaluating natural language processing (NLP) methods. The goals of this study are to quantify and qualify the disagreement between human and NLP. We developed an NLP system for annotating clinical trial eligibility criteria text and constructed a manually annotated corpus, both following the OMOP Common Data Model (CDM). We analyzed the discrepancies between the human and NLP annotations and their causes (e.g., ambiguities in concept categorization and tacit decisions on inclusion of qualifiers and temporal attributes during concept annotation). This study initially reported complexities in clinical trial eligibility criteria text that complicate NLP and the limitations of the OMOP CDM. The disagreement between and human and NLP annotations may be generalizable. We discuss implications for NLP evaluation.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8378608PMC
September 2021

Severity Prediction for COVID-19 Patients via Recurrent Neural Networks.

AMIA Jt Summits Transl Sci Proc 2021;2021:374-383. Epub 2021 May 17.

Department of Biomedical Informatics, Columbia University, New York, N.Y.

The novel coronavirus disease-2019 (COVID-19) pandemic has threatened the health of tens of millions of people worldwide and imposed heavy burden on global healthcare systems. In this paper, we propose a model to predict whether a patient infected with COVID-19 will develop severe outcomes based only on the patient's historical electronic health records (EHR) prior to hospital admission using recurrent neural networks. The model predicts risk score that represents the probability for a patient to progress into severe status (mechanical ventilation, tracheostomy, or death) after being infected with COVID-19. The model achieved 0.846 area under the receiver operating characteristic curve in predicting patients' outcomes averaged over 5-fold cross validation. While many of the existing models use features obtained after diagnosis of COVID-19, our proposed model only utilizes a patient's historical EHR to enable proactive risk management at the time of hospital admission.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8378630PMC
September 2021

Clinical Phenotypic Spectrum of 4095 Individuals with Down Syndrome from Text Mining of Electronic Health Records.

Genes (Basel) 2021 07 28;12(8). Epub 2021 Jul 28.

Center for Cellular and Molecular Therapeutics, Children's Hospital of Philadelphia, Philadelphia, PA 19104, USA.

Human genetic disorders, such as Down syndrome, have a wide variety of clinical phenotypic presentations, and characterizing each nuanced phenotype and subtype can be difficult. In this study, we examined the electronic health records of 4095 individuals with Down syndrome at the Children's Hospital of Philadelphia to create a method to characterize the phenotypic spectrum digitally. We extracted Human Phenotype Ontology (HPO) terms from quality-filtered patient notes using a natural language processing (NLP) approach MetaMap. We catalogued the most common HPO terms related to Down syndrome patients and compared the terms with those from a baseline population. We characterized the top 100 HPO terms by their frequencies at different ages of clinical visits and highlighted selected terms that have time-dependent distributions. We also discovered phenotypic terms that have not been significantly associated with Down syndrome, such as "Proptosis", "Downslanted palpebral fissures", and "Microtia". In summary, our study demonstrated that the clinical phenotypic spectrum of individual with Mendelian diseases can be characterized through NLP-based digital phenotyping on population-scale electronic health records (EHRs).
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.3390/genes12081159DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8393657PMC
July 2021

From clinical trials to clinical practice: How long are drugs tested and then used by patients?

J Am Med Inform Assoc 2021 Oct;28(11):2456-2460

Department of Biomedical Informatics, Columbia University, New York, New York, USA.

Objective: Evidence is scarce regarding the safety of long-term drug use, especially for drugs treating chronic diseases. To bridge this knowledge gap, this research investigated the differences in drug exposure between clinical trials and clinical practice.

Materials And Methods: We extracted drug follow-up times from clinical trials in ClinicalTrials.gov and compared the difference between clinical trials and real-world usage data for 914 drugs taken by 96 645 927 patients.

Results: A total of 17.5% of drugs had longer median exposure in practice than in trials, 6% of patients had extended exposure to at least 1 drug, and drugs treating nervous system disorders and cardiovascular diseases were the most common among drugs with high rates of extended exposure.

Conclusions: For most of patients, the drug use length is shorter than the tested length in clinical trials. Still, a remarkable number of patients experienced extended drug exposure, particularly for drugs treating nervous system disorders or cardiovascular disorders.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1093/jamia/ocab164DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8510283PMC
October 2021

Penetrance of Breast Cancer Susceptibility Genes From the eMERGE III Network.

JNCI Cancer Spectr 2021 Aug 8;5(4):pkab044. Epub 2021 May 8.

Department of Pediatrics, Columbia University Irving Medical Center, New York, NY, USA.

Background: Unbiased estimates of penetrance are challenging but critically important to make informed choices about strategies for risk management through increased surveillance and risk-reducing interventions.

Methods: We studied the penetrance and clinical outcomes of 7 breast cancer susceptibility genes (, , , , , , and ) in almost 13 458 participants unselected for personal or family history of breast cancer. We identified 242 female participants with pathogenic or likely pathogenic variants in 1 of the 7 genes for penetrance analyses, and 147 women did not previously know their genetic results.

Results: Out of the 147 women, 32 women were diagnosed with breast cancer at an average age of 52.8 years. Estimated penetrance by age 60 years ranged from 17.8% to 43.8%, depending on the gene. In clinical-impact analysis, 42.3% (95% confidence interval = 31.3% to 53.3%) of women had taken actions related to their genetic results, and 2 new breast cancer cases were identified within the first 12 months after genetic results disclosure.

Conclusions: Our study provides population-based penetrance estimates for the understudied genes , , and and highlights the importance of using unselected populations for penetrance studies. It also demonstrates the potential clinical impact of genetic testing to improve health care through early diagnosis and preventative screening.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1093/jncics/pkab044DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8346699PMC
August 2021

Misalignment between COVID-19 hotspots and clinical trial sites.

J Am Med Inform Assoc 2021 10;28(11):2461-2466

Department of Biostatistics, Mailman School of Public Health, Columbia University, New York, New York, USA.

Hundreds of interventional clinical trials have been launched in the United States to identify effective treatment strategies for combating the coronavirus disease 2019 (COVID-19) pandemic. However, to date, only a small fraction of these trials have completed enrollment, delaying the scientific investigation of COVID-19 and its treatment options. This study presents novel metrics to examine the geographic alignment between COVID-19 hotspots and interventional clinical trial sites and evaluate trial access over time during the evolving pandemic. Using temporal COVID-19 case data from USAFacts.org and trial data from ClinicalTrials.gov, U.S. counties were categorized based on their numbers of cases and trials. Our analysis suggests that alignment and access have worsened as the pandemic shifted over time. We recommend strategies and metrics to evaluate the alignment between cases and trials. Future studies are warranted to investigate the impact of the misalignment of cases and clinical trial sites on clinical trial recruitment.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1093/jamia/ocab167DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8385938PMC
October 2021

Generalizability of Polygenic Risk Scores for Breast Cancer Among Women With European, African, and Latinx Ancestry.

JAMA Netw Open 2021 Aug 2;4(8):e2119084. Epub 2021 Aug 2.

Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, New York.

Importance: Multiple polygenic risk scores (PRSs) for breast cancer have been developed from large research consortia; however, their generalizability to diverse clinical settings is unknown.

Objective: To examine the performance of previously developed breast cancer PRSs in a clinical setting for women of European, African, and Latinx ancestry.

Design, Setting, And Participants: This cohort study using the Electronic Medical Records and Genomics (eMERGE) network data set included 39 591 women from 9 contributing medical centers in the US that had electronic medical records (EMR) linked to genotype data. Breast cancer cases and controls were identified through a validated EMR phenotyping algorithm.

Main Outcomes And Measures: Multivariable logistic regression was used to assess the association between breast cancer risk and 7 previously developed PRSs, adjusting for age, study site, breast cancer family history, and first 3 ancestry informative principal components.

Results: This study included 39 591 women: 33 594 with European, 3801 with African, and 2196 with Latinx ancestry. The mean (SD) age at breast cancer diagnosis was 60.7 (13.0), 58.8 (12.5), and 60.1 (13.0) years for women with European, African, and Latinx ancestry, respectively. PRSs derived from women with European ancestry were associated with breast cancer risk in women with European ancestry (highest odds ratio [OR] per 1-SD increase, 1.46; 95% CI, 1.41-1.51), women with Latinx ancestry (highest OR, 1.31; 95% CI, 1.09-1.58), and women with African ancestry (OR, 1.19; 95% CI, 1.05-1.35). For women with European ancestry, this association with breast cancer risk was largest in the extremes of the PRS distribution, with ORs ranging from 2.19 (95% CI, 1.84-2.53) to 2.48 (95% CI, 1.89-3.25) for the 3 different PRSs examined for those in the highest 1% of the PRS compared with those in the middle quantile. Among women with Latinx and African ancestries at the extremes of the PRS distribution, there were no statistically significant associations.

Conclusions And Relevance: This cohort study found that PRS models derived from women with European ancestry for breast cancer risk generalized well for women with European, Latinx, and African ancestries across different clinical settings, although the effect sizes for women with African ancestry were smaller, likely because of differences in risk allele frequencies and linkage disequilibrium patterns. These results highlight the need to improve representation of diverse population groups, particularly women with African ancestry, in genomic research cohorts.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1001/jamanetworkopen.2021.19084DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8339934PMC
August 2021

A conceptual framework for external validity.

J Biomed Inform 2021 09 21;121:103870. Epub 2021 Jul 21.

Department of Biomedical Informatics, Columbia University. New York, NY, USA.

Evidence-Based Medicine (EBM) encourages clinicians to seek the most reputable evidence. The quality of evidence is organized in a hierarchy in which randomized controlled trials (RCTs) are regarded as least biased. However, RCTs are plagued by poor generalizability, impeding the translation of clinical research to practice. Though the presence of poor external validity is known, the factors that contribute to poor generalizability have not been summarized and placed in a framework. We propose a new population-oriented conceptual framework to facilitate consistent and comprehensive evaluation of generalizability, replicability, and assessment of RCT study quality.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1016/j.jbi.2021.103870DOI Listing
September 2021

Quantitative disease risk scores from EHR with applications to clinical risk stratification and genetic studies.

NPJ Digit Med 2021 Jul 23;4(1):116. Epub 2021 Jul 23.

Department of Biostatistics, Columbia University, New York, NY, USA.

Labeling clinical data from electronic health records (EHR) in health systems requires extensive knowledge of human expert, and painstaking review by clinicians. Furthermore, existing phenotyping algorithms are not uniformly applied across large datasets and can suffer from inconsistencies in case definitions across different algorithms. We describe here quantitative disease risk scores based on almost unsupervised methods that require minimal input from clinicians, can be applied to large datasets, and alleviate some of the main weaknesses of existing phenotyping algorithms. We show applications to phenotypic data on approximately 100,000 individuals in eMERGE, and focus on several complex diseases, including Chronic Kidney Disease, Coronary Artery Disease, Type 2 Diabetes, Heart Failure, and a few others. We demonstrate that relative to existing approaches, the proposed methods have higher prediction accuracy, can better identify phenotypic features relevant to the disease under consideration, can perform better at clinical risk stratification, and can identify undiagnosed cases based on phenotypic features available in the EHR. Using genetic data from the eMERGE-seq panel that includes sequencing data for 109 genes on 21,363 individuals from multiple ethnicities, we also show how the new quantitative disease risk scores help improve the power of genetic association studies relative to the standard use of disease phenotypes. The results demonstrate the effectiveness of quantitative disease risk scores derived from rich phenotypic EHR databases to provide a more meaningful characterization of clinical risk for diseases of interest beyond the prevalent binary (case-control) classification.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1038/s41746-021-00488-3DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8302667PMC
July 2021

Neptune: an environment for the delivery of genomic medicine.

Genet Med 2021 10 13;23(10):1838-1846. Epub 2021 Jul 13.

Human Genome Sequencing Center, Baylor College of Medicine, Houston, TX, USA.

Purpose: Genomic medicine holds great promise for improving health care, but integrating searchable and actionable genetic data into electronic health records (EHRs) remains a challenge. Here we describe Neptune, a system for managing the interaction between a clinical laboratory and an EHR system during the clinical reporting process.

Methods: We developed Neptune and applied it to two clinical sequencing projects that required report customization, variant reanalysis, and EHR integration.

Results: Neptune has been applied for the generation and delivery of over 15,000 clinical genomic reports. This work spans two clinical tests based on targeted gene panels that contain 68 and 153 genes respectively. These projects demanded customizable clinical reports that contained a variety of genetic data types including single-nucleotide variants (SNVs), copy-number variants (CNVs), pharmacogenomics, and polygenic risk scores. Two variant reanalysis activities were also supported, highlighting this important workflow.

Conclusion: Methods are needed for delivering structured genetic data to EHRs. This need extends beyond developing data formats to providing infrastructure that manages the reporting process itself. Neptune was successfully applied on two high-throughput clinical sequencing projects to build and deliver clinical reports to EHR systems. The software is open source and available at https://gitlab.com/bcm-hgsc/neptune .
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1038/s41436-021-01230-wDOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8487966PMC
October 2021

Comparative effectiveness of medical concept embedding for feature engineering in phenotyping.

JAMIA Open 2021 Apr 16;4(2):ooab028. Epub 2021 Jun 16.

Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, New York 10032, USA.

Objective: Feature engineering is a major bottleneck in phenotyping. Properly learned medical concept embeddings (MCEs) capture the semantics of medical concepts, thus are useful for retrieving relevant medical features in phenotyping tasks. We compared the effectiveness of MCEs learned from knowledge graphs and electronic healthcare records (EHR) data in retrieving relevant medical features for phenotyping tasks.

Materials And Methods: We implemented 5 embedding methods including node2vec, singular value decomposition (SVD), LINE, skip-gram, and GloVe with 2 data sources: (1) knowledge graphs obtained from the observational medical outcomes partnership (OMOP) common data model; and (2) patient-level data obtained from the OMOP compatible electronic health records (EHR) from Columbia University Irving Medical Center (CUIMC). We used phenotypes with their relevant concepts developed and validated by the electronic medical records and genomics (eMERGE) network to evaluate the performance of learned MCEs in retrieving phenotype-relevant concepts. in retrieving phenotype-relevant concepts based on a single and multiple seed concept(s) was used to evaluate MCEs.

Results: Among all MCEs, MCEs learned by using node2vec with knowledge graphs showed the best performance. Of MCEs based on knowledge graphs and EHR data, MCEs learned by using node2vec with knowledge graphs and MCEs learned by using GloVe with EHR data outperforms other MCEs, respectively.

Conclusion: MCE enables scalable feature engineering tasks, thereby facilitating phenotyping. Based on current phenotyping practices, MCEs learned by using knowledge graphs constructed by hierarchical relationships among medical concepts outperformed MCEs learned by using EHR data.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1093/jamiaopen/ooab028DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8206403PMC
April 2021

A Mendelian Randomization Approach Using 3-HMG-Coenzyme-A Reductase Gene Variation to Evaluate the Association of Statin-Induced Low-Density Lipoprotein Cholesterol Lowering With Noncardiovascular Disease Phenotypes.

JAMA Netw Open 2021 Jun 1;4(6):e2112820. Epub 2021 Jun 1.

Division of Clinical Pharmacology, Department of Medicine, Vanderbilt University Medical Center, Nashville, Tennessee.

Importance: Observational studies suggest that statins, which inhibit 3-hydroxy-3-methylglutaryl coenzyme A (HMG-CoA) reductase, may be associated with beneficial effects in many noncardiovascular diseases.

Objective: To construct a weighted HMG-CoA reductase (HMGCR) gene genetic risk score (GRS) using variants in the HMGCR gene affecting low-density lipoprotein cholesterol as an instrumental variable for mendelian randomization analyses to test associations with candidate noncardiovascular phenotypes previously associated with statin use in observational studies.

Design, Setting, And Participants: This cohort study included 53 385 unrelated adults of European ancestry with genome-wide genotypes available from BioVU (a practice-based biobank, used for discovery) and 30 444 unrelated adults with European ancestry available in the Electronic Medical Records and Genomics (eMERGE; a research consortium that conducts genetic research using electronic medical records, used for replication). The study was conducted from February 6, 2015, through April 31, 2019; data analysis was performed from August 26, 2019, through December 22, 2020.

Interventions: An HMGCR GRS was calculated.

Main Outcomes And Measures: The association between the HMGCR GRS and the presence or absence of 22 noncardiovascular phenotypes previously associated with statin use in clinical studies.

Results: Of the 53 385 individuals in BioVU, 29 958 (56.1%) were women; mean (SD) age was 59.9 (15.6) years. The finding between the HMGCR GRS and the noncardiovascular phenotypes of interest in this cohort was significant only for type 2 diabetes. An HMGCR GRS equivalent to a 10-mg/dL decrease in the low-density lipoprotein cholesterol level was associated with an increased risk of type 2 diabetes (odds ratio [OR], 1.09; 95% CI, 1.04-1.15; P = 5.58 × 10-4). The HMGCR GRS was not associated with other phenotypes; the closest were increased risk of Parkinson disease (OR, 1.30; 95% CI, 1.07-1.58; P = .007) and kidney failure (OR, 1.18; 95% CI, 1.05-1.34; P = .008). Of the 30 444 individuals in eMERGE, 16 736 (55.0%) were women; mean (SD) age was 68.7 (15.4) years. The association between the HMGCR GRS and type 2 diabetes was replicated in this cohort (OR, 1.09; 95% CI, 1.01-1.17; P = .02); however, the HMGCR GRS was not associated with Parkinson disease (OR, 0.93; 95% CI, 0.75-1.16; P = .53) and kidney failure (OR, 1.18; 95% CI, 0.98-1.41; P = .08) in the eMERGE cohort.

Conclusions And Relevance: A mendelian randomization approach using variants in the HMGCR gene replicated the association between statin use and increased type 2 diabetes risk but provided no strong evidence for pleiotropic effects of statin-induced decrease of the low-density lipoprotein cholesterol level on other diseases.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1001/jamanetworkopen.2021.12820DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8185593PMC
June 2021

A deep database of medical abbreviations and acronyms for natural language processing.

Sci Data 2021 06 2;8(1):149. Epub 2021 Jun 2.

Department of Biomedical Informatics, Columbia University, New York, NY, USA.

The recognition, disambiguation, and expansion of medical abbreviations and acronyms is of upmost importance to prevent medically-dangerous misinterpretation in natural language processing. To support recognition, disambiguation, and expansion, we present the Medical Abbreviation and Acronym Meta-Inventory, a deep database of medical abbreviations. A systematic harmonization of eight source inventories across multiple healthcare specialties and settings identified 104,057 abbreviations with 170,426 corresponding senses. Automated cross-mapping of synonymous records using state-of-the-art machine learning reduced redundancy, which simplifies future application. Additional features include semi-automated quality control to remove errors. The Meta-Inventory demonstrated high completeness or coverage of abbreviations and senses in new clinical text, a substantial improvement over the next largest repository (6-14% increase in abbreviation coverage; 28-52% increase in sense coverage). To our knowledge, the Meta-Inventory is the most complete compilation of medical abbreviations and acronyms in American English to-date. The multiple sources and high coverage support application in varied specialties and settings. This allows for cross-institutional natural language processing, which previous inventories did not support. The Meta-Inventory is available at https://bit.ly/github-clinical-abbreviations .
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1038/s41597-021-00929-4DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8172575PMC
June 2021

Preferences for Updates on General Research Results: A Survey of Participants in Genomic Research from Two Institutions.

J Pers Med 2021 May 11;11(5). Epub 2021 May 11.

Johns Hopkins University Berman Institute of Bioethics, Baltimore, MD 21205, USA.

There is a need for multimodal strategies to keep research participants informed about study results. Our aim was to characterize preferences of genomic research participants from two institutions along four dimensions of general research result updates: content, timing, mechanism, and frequency.

Methods: We conducted a web-based cross-sectional survey that was administered from 25 June 2018 to 5 December 2018.

Results: 397 participants completed the survey, most of whom (96%) expressed a desire to receive research updates. Preferences with high endorsement included: update content (brief descriptions of major findings, descriptions of purpose and goals, and educational material); update timing (when the research is completed, when findings are reviewed, when findings are published, and when the study status changes); update mechanism (email with updates, and email newsletter); and update frequency (every three months). Hierarchical cluster analyses based on the four update preferences identified four profiles of participants with similar preference patterns. Very few participants in the largest profile were comfortable with budgeting less money for research activities so that researchers have money to set up services to send research result updates to study participants.

Conclusion: Future studies may benefit from exploring preferences for research result updates, as we have in our study. In addition, this work provides evidence of a need for funders to incentivize researchers to communicate results to participants.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.3390/jpm11050399DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8151672PMC
May 2021

Clinical comparison between trial participants and potentially eligible patients using electronic health record data: A generalizability assessment method.

J Biomed Inform 2021 07 25;119:103822. Epub 2021 May 25.

Department of Biomedical Informatics, Columbia University, New York, NY, United States. Electronic address:

Objective: To present a generalizability assessment method that compares baseline clinical characteristics of trial participants (TP) to potentially eligible (PE) patients as presented in their electronic health record (EHR) data while controlling for clinical setting and recruitment period.

Methods: For each clinical trial, a clinical event was defined to identify patients of interest using available EHR data from one clinical setting during the trial's recruitment timeframe. The trial's eligibility criteria were then applied and patients were separated into two mutually exclusive groups: (1) TP, which were patients that participated in the trial per trial enrollment data; (2) PE, the remaining patients. The primary outcome was standardized differences in clinical characteristics between TP and PE per trial. A standardized difference was considered prominent if its absolute value was greater than or equal to 0.1. The secondary outcome was the difference in mean propensity scores (PS) between TP and PE per trial, in which the PS represented prediction for a patient to be in the trial. Three diverse trials were selected for illustration: one focused on hepatitis C virus (HCV) patients receiving a liver transplantation; one focused on leukemia patients and lymphoma patients; and one focused on appendicitis patients.

Results: For the HCV trial, 43 TP and 83 PE were found, with 61 characteristics evaluated. Prominent differences were found among 69% of characteristics, with a mean PS difference of 0.13. For the leukemia/lymphoma trial, 23 TP and 23 PE were found, with 39 characteristics evaluated. Prominent differences were found among 82% of characteristics, with a mean PS difference of 0.76. For the appendicitis trial, 123 TP and 242 PE were found, with 52 characteristics evaluated. Prominent differences were found among 52% of characteristics, with a mean PS difference of 0.15.

Conclusions: Differences in clinical characteristics were observed between TP and PE among all three trials. In two of the three trials, not all of the differences necessarily compromised trial generalizability and subsets of PE could be considered similar to their corresponding TP. In the remaining trial, lack of generalizability appeared present, but may be a result of other factors such as small sample size or site recruitment strategy. These inconsistent findings suggest eligibility criteria alone are sometimes insufficient in defining a target group to generalize to. With caveats in limited scalability, EHR data quality, and lack of patient perspective on trial participation, this generalizability assessment method that incorporates control for temporality and clinical setting promise to better pinpoint clinical patterns and trial considerations.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1016/j.jbi.2021.103822DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8260457PMC
July 2021

Participatory Design of a Clinical Trial Eligibility Criteria Simplification Method.

Stud Health Technol Inform 2021 May;281:984-988

Department of Biomedical Informatics.

Clinical trial eligibility criteria are important for selecting the right participants for clinical trials. However, they are often complex and not computable. This paper presents the participatory design of a human-computer collaboration method for criteria simplification that includes natural language processing followed by user-centered eligibility criteria simplification. A case study on the ARCADIA trial shows how criteria were simplified for structured database querying by clinical researchers and identifies rules for criteria simplification and concept normalization.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.3233/SHTI210325DOI Listing
May 2021

Potential Role of Clinical Trial Eligibility Criteria in Electronic Phenotyping.

Stud Health Technol Inform 2021 May;281:148-152

Department of Biomedical Informatics, Columbia University, New York, NY, USA.

2,719 distinctive phenotyping variables from 176 electronic phenotypes were compared with 57,150 distinctive clinical trial eligibility criteria concepts to assess the phenotype knowledge overlap between them. We observed a high percentage (69.5%) of eMERGE phenotype features and a lower percentage (47.6%) of OHDSI phenotype features matched to clinical trial eligibility criteria, possibly due to the relative emphasis on specificity for eMERGE phenotypes and the relative emphasis on sensitivity for OHDSI phenotypes. The study results show the potential of reusing clinical trial eligibility criteria for phenotyping feature selection and moderate benefits of using them for local cohort query implementation.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.3233/SHTI210138DOI Listing
May 2021

PhenCards: a data resource linking human phenotype information to biomedical knowledge.

Genome Med 2021 05 25;13(1):91. Epub 2021 May 25.

Raymond G. Perelman Center for Cellular and Molecular Therapeutics, Children's Hospital of Philadelphia, Philadelphia, PA, 19104, USA.

We present PhenCards ( https://phencards.org ), a database and web server intended as a one-stop shop for previously disconnected biomedical knowledge related to human clinical phenotypes. Users can query human phenotype terms or clinical notes. PhenCards obtains relevant disease/phenotype prevalence and co-occurrence, drug, procedural, pathway, literature, grant, and collaborator data. PhenCards recommends the most probable genetic diseases and candidate genes based on phenotype terms from clinical notes. PhenCards facilitates exploration of phenotype, e.g., which drugs cause or are prescribed for patient symptoms, which genes likely cause specific symptoms, and which comorbidities co-occur with phenotypes.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1186/s13073-021-00909-8DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8147460PMC
May 2021

A neuro-symbolic method for understanding free-text medical evidence.

J Am Med Inform Assoc 2021 07;28(8):1703-1711

Department of Biomedical Informatics, Columbia University, New York, USA.

Objective: We introduce Medical evidence Dependency (MD)-informed attention, a novel neuro-symbolic model for understanding free-text clinical trial publications with generalizability and interpretability.

Materials And Methods: We trained one head in the multi-head self-attention model to attend to the Medical evidence Ddependency (MD) and to pass linguistic and domain knowledge on to later layers (MD informed). This MD-informed attention model was integrated into BioBERT and tested on 2 public machine reading comprehension benchmarks for clinical trial publications: Evidence Inference 2.0 and PubMedQA. We also curated a small set of recently published articles reporting randomized controlled trials on COVID-19 (coronavirus disease 2019) following the Evidence Inference 2.0 guidelines to evaluate the model's robustness to unseen data.

Results: The integration of MD-informed attention head improves BioBERT substantially in both benchmark tasks-as large as an increase of +30% in the F1 score-and achieves the new state-of-the-art performance on the Evidence Inference 2.0. It achieves 84% and 82% in overall accuracy and F1 score, respectively, on the unseen COVID-19 data.

Conclusions: MD-informed attention empowers neural reading comprehension models with interpretability and generalizability via reusable domain knowledge. Its compositionality can benefit any transformer-based architecture for machine reading comprehension of free-text medical evidence.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1093/jamia/ocab077DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8135980PMC
July 2021

Charting the life course: Emerging opportunities to advance scientific approaches using life course research.

J Clin Transl Sci 2020 Jun 15;5(1):e9. Epub 2020 Jun 15.

Department of Pediatrics, Vanderbilt University Medical Center, Nashville, TN, USA.

Life course research embraces the complexity of health and disease development, tackling the extensive interactions between genetics and environment. This interdisciplinary blueprint, or theoretical framework, offers a structure for research ideas and specifies relationships between related factors. Traditionally, methodological approaches attempt to reduce the complexity of these dynamic interactions and decompose health into component parts, ignoring the complex reciprocal interaction of factors that shape health over time. New methods that match the epistemological foundation of the life course framework are needed to fully explore adaptive, multilevel, and reciprocal interactions between individuals and their environment. The focus of this article is to (1) delineate the differences between lifespan and life course research, (2) articulate the importance of complex systems science as a methodological framework in the life course research toolbox to guide our research questions, (3) raise key questions that can be asked within the clinical and translational science domain utilizing this framework, and (4) provide recommendations for life course research implementation, charting the way forward. Recent advances in computational analytics, computer science, and data collection could be used to approximate, measure, and analyze the intertwining and dynamic nature of genetic and environmental factors involved in health development.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1017/cts.2020.492DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8057465PMC
June 2020

Medical Records-Based Genetic Studies of the Complement System.

J Am Soc Nephrol 2021 08 3;32(8):2031-2047. Epub 2021 May 3.

Division of Nephrology, Department of Medicine, Vagelos College of Physicians and Surgeons, Columbia University, New York, New York

Background: Genetic variants in complement genes have been associated with a wide range of human disease states, but well-powered genetic association studies of complement activation have not been performed in large multiethnic cohorts.

Methods: We performed medical records-based genome-wide and phenome-wide association studies for plasma C3 and C4 levels among participants of the Electronic Medical Records and Genomics (eMERGE) network.

Results: In a GWAS for C3 levels in 3949 individuals, we detected two genome-wide significant loci: chr.1q31.3 (CFH locus; rs3753396-A; =0.20; 95% CI, 0.14 to 0.25; =1.52x10) and chr.19p13.3 (C3 locus; rs11569470-G; =0.19; 95% CI, 0.13 to 0.24; =1.29x10). These two loci explained approximately 2% of variance in C3 levels. GWAS for C4 levels involved 3998 individuals and revealed a genome-wide significant locus at chr.6p21.32 (C4 locus; rs3135353-C; =0.40; 95% CI, 0.34 to 0.45; =4.58x10). This locus explained approximately 13% of variance in C4 levels. The multiallelic copy number variant analysis defined two structural genomic C4 variants with large effect on blood C4 levels: C4-BS (=-0.36; 95% CI, -0.42 to -0.30; =2.98x10) and C4-AL-BS (=0.25; 95% CI, 0.21 to 0.29; =8.11x10). Overall, C4 levels were strongly correlated with copy numbers of C4A and C4B genes. In comprehensive phenome-wide association studies involving 102,138 eMERGE participants, we cataloged a full spectrum of autoimmune, cardiometabolic, and kidney diseases genetically related to systemic complement activation.

Conclusions: We discovered genetic determinants of plasma C3 and C4 levels using eMERGE genomic data linked to electronic medical records. Genetic variants regulating C3 and C4 levels have large effects and multiple clinical correlations across the spectrum of complement-related diseases in humans.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1681/ASN.2020091371DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8455263PMC
August 2021

Characterizing database granularity using SNOMED-CT hierarchy.

AMIA Annu Symp Proc 2020 25;2020:983-992. Epub 2021 Jan 25.

Columbia University, New York, NY, USA.

Multi-center observational studies require recognition and reconciliation of differences in patient representations arising from underlying populations, disparate coding practices and specifics of data capture. This leads to different granularity or detail of concepts representing the clinical facts. For researchers studying certain populations of interest, it is important to ensure that concepts at the right level are used for the definition of these populations. We studied the granularity of concepts within 22 data sources in the OHDSI network and calculated a composite granularity score for each dataset. Three alternative SNOMED-based approaches for such score showed consistency in classifying data sources into three levels of granularity (low, moderate and high), which correlated with the provenance of data and country of origin. However, they performed unsatisfactorily in ordering data sources within these groups and showed inconsistency for small data sources. Further studies on examining approaches to data source granularity are needed.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8075504PMC
June 2021

Impact of IMPACT: Longitudinal Analysis of an Integrated Participant Scheduling System in a Clinical Research Setting.

AMIA Annu Symp Proc 2020 25;2020:283-292. Epub 2021 Jan 25.

Columbia University, New York, NY i0032, USA.

Rapidly increasing costs have been a major threat to our clinical research enterprise. Improvement in appointment scheduling is a crucial means to boost efficiency and save cost in clinical research and has been well studied in the outpatient setting. This study reviews nearly 5 years of usage data of an integrated scheduling system implemented at Columbia University/New York Presbyterian (CUIMC/NYP) called IMPACT and provides original insights into the challenges faced by a clinical research facility. Briefly, the IMPACT data shows that high rates of room and resource changes correlate with rescheduled appointments and that rescheduled visits are more likely to be attended than non-rescheduled visits. We highlight the differing roles of schedulers, coordinators, and investigators, and propose a highly accurate predictive model of participant no-shows in a research setting. This study sheds light on ways to reduce overall cost and improve the care we offer to clinical research participants.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8075470PMC
June 2021

Building an OMOP common data model-compliant annotated corpus for COVID-19 clinical trials.

J Biomed Inform 2021 06 28;118:103790. Epub 2021 Apr 28.

Department of Biomedical Informatics, Columbia University, New York, NY, USA. Electronic address:

Clinical trials are essential for generating reliable medical evidence, but often suffer from expensive and delayed patient recruitment because the unstructured eligibility criteria description prevents automatic query generation for eligibility screening. In response to the COVID-19 pandemic, many trials have been created but their information is not computable. We included 700 COVID-19 trials available at the point of study and developed a semi-automatic approach to generate an annotated corpus for COVID-19 clinical trial eligibility criteria called COVIC. A hierarchical annotation schema based on the OMOP Common Data Model was developed to accommodate four levels of annotation granularity: i.e., study cohort, eligibility criteria, named entity and standard concept. In COVIC, 39 trials with more than one study cohorts were identified and labelled with an identifier for each cohort. 1,943 criteria for non-clinical characteristics such as "informed consent", "exclusivity of participation" were annotated. 9767 criteria were represented by 18,161 entities in 8 domains, 7,743 attributes of 7 attribute types and 16,443 relationships of 11 relationship types. 17,171 entities were mapped to standard medical concepts and 1,009 attributes were normalized into computable representations. COVIC can serve as a corpus indexed by semantic tags for COVID-19 trial search and analytics, and a benchmark for machine learning based criteria extraction.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1016/j.jbi.2021.103790DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8079156PMC
June 2021

Data Quality of Chemotherapy-Induced Nausea and Vomiting Documentation.

Appl Clin Inform 2021 03 21;12(2):320-328. Epub 2021 Apr 21.

School of Nursing, Columbia University, New York, New York, United States.

Objective: The objective of the study was to characterize the completeness and concordance of the electronic health record (EHR) documentation of cancer symptoms among multidisciplinary health care professionals.

Methods: We examined the EHRs of children, adolescents, and young adults who received highly emetogenic chemotherapy and characterized the completeness and concordance of chemotherapy-induced nausea and vomiting (CINV) documentation by clinician type and by the International Classification of Diseases 10th Revision (ICD-10) coding choice.

Results: The EHRs of 127 patients, comprising 870 patient notes, were abstracted and reviewed. A CINV assessment was documented by prescribers in 75% of patients, and by nurses in 58% of patients. Of the 60 encounters where both prescribers and nurses documented, 72% agreed on the presence/absence of CINV.

Conclusion: Most patients receiving highly emetogenic chemotherapy had a documented assessment of CINV; however, many had incomplete or discordant documentation of CINV from different providers by role, implying the importance of incorporating pragmatic knowledge of EHR documentation patterns among multidisciplinary health professionals for EHR phenotyping and clinical decision support systems directed toward cancer-related symptom management.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1055/s-0041-1728698DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8060070PMC
March 2021
-->