Publications by authors named "Inocencio Maramba"

14 Publications

  • Page 1 of 1

Methods of usability testing in the development of eHealth applications: A scoping review.

Int J Med Inform 2019 06 31;126:95-104. Epub 2019 Mar 31.

Peninsula Medical School, Faculty of Medicine and Dentistry, University of Plymouth, Drake Circus, Plymouth, PL4 8AA, United Kingdom.

Background: The number of eHealth applications has exponentially increased in recent years, with over 325,000 health apps now available on all major app stores. This is in addition to other eHealth applications available on other platforms such as PC software, web sites and even gaming consoles. As with other digital applications, usability is one of the key factors in the successful implementation of eHealth apps. Reviews of the literature on empirical methods of usability testing in eHealth were last published in 2015. In the context of an exponentially increasing rate of App development year on year, an updated review is warranted.

Objective: To identify, explore, and summarize the current methods used in the usability testing of eHealth applications.

Methods: A scoping review was conducted on literature available from April 2014 up to October 2017. Four databases were searched. Literature was considered for inclusion if it was (1) focused on an eHealth application (which includes websites, PC software, smartphone and tablet applications), (2) provided information about usability of the application, (3) provided empirical results of the usability testing, (4) a full or short paper (not an abstract) published in English after March 2014. We then extracted data pertaining to the usability evaluation processes described in the selected studies.

Results: 133 articles met the inclusion criteria. The methods used for usability testing, in decreasing order of frequency were: questionnaires (n = 105), task completion (n = 57), 'Think-Aloud' (n = 45), interviews (n = 37), heuristic testing (n = 18) and focus groups (n = 13). Majority of the studies used one (n = 45) or two (n = 46) methods of testing. The rest used a combination of three (n = 30) or four (n = 12) methods of testing usability. None of the studies used automated mechanisms to test usability. The System Usability Scale (SUS) was the most frequently used questionnaire (n = 44). The ten most frequent health conditions or diseases where eHealth apps were being evaluated for usability were the following: mental health (n = 12), cancer (n = 10), nutrition (n = 10), child health (n = 9), diabetes (n = 9), telemedicine (n = 8), cardiovascular disease (n = 6), HIV (n = 4), health information systems (n = 4) and smoking (n = 4). Further iterations of the app were reported in a minority of the studies (n = 41). The use of the 'Think-Aloud' (Pearson Chi-squared test: χ2 = 11.15, p < 0.05) and heuristic walkthrough (Pearson Chi-squared test: χ2 = 4.48, p < 0.05) were significantly associated with at least one further iteration of the app being developed.

Conclusion: Although there has been an exponential increase in the number of eHealth apps, the number of studies that have been published that report the results of usability testing on these apps has not increased at an equivalent rate. The number of digital health applications that publish their usability evaluation results remains only a small fraction. Questionnaires are the most prevalent method of evaluating usability in eHealth applications, which provide an overall measure of usability but do not pinpoint the problems that need to be addressed. Qualitative methods may be more useful in this regard. The use of multiple evaluation methods has increased. Automated methods such as eye tracking have not gained traction in evaluating health apps. Further research is needed into which methods are best suited for the different types of eHealth applications, according to their target users and the health conditions being addressed.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1016/j.ijmedinf.2019.03.018DOI Listing
June 2019

The Evaluation of Physicians' Communication Skills From Multiple Perspectives.

Ann Fam Med 2018 07;16(4):330-337

Cambridge Centre for Health Services Research, University of Cambridge School of Clinical Medicine, Cambridge, United Kingdom.

Purpose: To examine how family physicians', patients', and trained clinical raters' assessments of physician-patient communication compare by analysis of individual appointments.

Methods: Analysis of survey data from patients attending face-to-face appointments with 45 family physicians at 13 practices in England. Immediately post-appointment, patients and physicians independently completed a questionnaire including 7 items assessing communication quality. A sample of videotaped appointments was assessed by trained clinical raters, using the same 7 communication items. Patient, physician, and rater communication scores were compared using correlation coefficients.

Results: Included were 503 physician-patient pairs; of those, 55 appointments were also evaluated by trained clinical raters. Physicians scored themselves, on average, lower than patients (mean physician score 74.5; mean patient score 94.4); 63.4% (319) of patient-reported scores were the maximum of 100. The mean of rater scores from 55 appointments was 57.3. There was a near-zero correlation coefficient between physician-reported and patient-reported communication scores (0.009, = .854), and between physician-reported and trained rater-reported communication scores (-0.006, = .69). There was a moderate and statistically significant association, however, between patient and trained-rater scores (0.35, = .042).

Conclusions: The lack of correlation between physician scores and those of others indicates that physicians' perceptions of good communication during their appointments may differ from those of external peer raters and patients. Physicians may not be aware of how patients experience their communication practices; peer assessment of communication skills is an important approach in identifying areas for improvement.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1370/afm.2241DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6037531PMC
July 2018

Test-retest stability of patient experience items derived from the national GP patient survey.

Springerplus 2016 7;5(1):1755. Epub 2016 Oct 7.

Primary Care Research Group, University of Exeter Medical School, St Lukes Campus, Exeter, EX1 2LU UK.

Purpose: The validity and reliability of various items on the GP Patient Survey (GPPS) survey have been reported, however stability of patient responses over time has not been tested. The purpose of this study was to determine the test-retest reliability of the core items from the GPPS.

Methods: Patients who had recently consulted participating GPs in five general practices across the South West England were sent a postal questionnaire comprising of 54 items concerning their experience of their consultation and the care they received from the GP practice. Patients returning the questionnaire within 3 weeks of mail-out were sent a second identical (retest) questionnaire. Stability of responses was assessed by raw agreement rates and Cohen's kappa (for categorical response items) and intraclass correlation coefficients and means (for ordinal response items).

Results: 348 of 597 Patients returned a retest questionnaire (58.3 % response rate). In comparison to the test phase, patients responding to the retest phase were older and more likely to have white British ethnicity. Raw agreement rates for the 33 categorical items ranged from 66 to 100 % (mean 88 %) while the kappa coefficients ranged from 0.00 to 1.00 (mean 0.53). Intraclass correlation coefficients for the 21 ordinal items averaged 0.67 (range 0.44-0.77).

Conclusions: Formal testing of items from the national GP patient survey examining patient experience in primary care highlighted their acceptable temporal stability several weeks following a GP consultation.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1186/s40064-016-3377-9DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5055510PMC
October 2016

Rating Communication in GP Consultations: The Association Between Ratings Made by Patients and Trained Clinical Raters.

Med Care Res Rev 2018 04 3;75(2):201-218. Epub 2016 Oct 3.

1 University of Cambridge School of Clinical Medicine, Cambridge, UK.

Patient evaluations of physician communication are widely used, but we know little about how these relate to professionally agreed norms of communication quality. We report an investigation into the association between patient assessments of communication quality and an observer-rated measure of communication competence. Consent was obtained to video record consultations with Family Practitioners in England, following which patients rated the physician's communication skills. A sample of consultation videos was subsequently evaluated by trained clinical raters using an instrument derived from the Calgary-Cambridge guide to the medical interview. Consultations scored highly for communication by clinical raters were also scored highly by patients. However, when clinical raters judged communication to be of lower quality, patient scores ranged from "poor" to "very good." Some patients may be inhibited from rating poor communication negatively. Patient evaluations can be useful for measuring relative performance of physicians' communication skills, but absolute scores should be interpreted with caution.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1177/1077558716671217DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5858640PMC
April 2018

Web-based textual analysis of free-text patient experience comments from a survey in primary care.

JMIR Med Inform 2015 May 6;3(2):e20. Epub 2015 May 6.

Primary Care, University of Exeter Medical School, University of Exeter, Exeter, United Kingdom.

Background: Open-ended questions eliciting free-text comments have been widely adopted in surveys of patient experience. Analysis of free text comments can provide deeper or new insight, identify areas for action, and initiate further investigation. Also, they may be a promising way to progress from documentation of patient experience to achieving quality improvement. The usual methods of analyzing free-text comments are known to be time and resource intensive. To efficiently deal with a large amount of free-text, new methods of rapidly summarizing and characterizing the text are being explored.

Objective: The aim of this study was to investigate the feasibility of using freely available Web-based text processing tools (text clouds, distinctive word extraction, key words in context) for extracting useful information from large amounts of free-text commentary about patient experience, as an alternative to more resource intensive analytic methods.

Methods: We collected free-text responses to a broad, open-ended question on patients' experience of primary care in a cross-sectional postal survey of patients recently consulting doctors in 25 English general practices. We encoded the responses to text files which were then uploaded to three Web-based textual processing tools. The tools we used were two text cloud creators: TagCrowd for unigrams, and Many Eyes for bigrams; and Voyant Tools, a Web-based reading tool that can extract distinctive words and perform Keyword in Context (KWIC) analysis. The association of patients' experience scores with the occurrence of certain words was tested with logistic regression analysis. KWIC analysis was also performed to gain insight into the use of a significant word.

Results: In total, 3426 free-text responses were received from 7721 patients (comment rate: 44.4%). The five most frequent words in the patients' comments were "doctor", "appointment", "surgery", "practice", and "time". The three most frequent two-word combinations were "reception staff", "excellent service", and "two weeks". The regression analysis showed that the occurrence of the word "excellent" in the comments was significantly associated with a better patient experience (OR=1.96, 95%CI=1.63-2.34), while "rude" was significantly associated with a worse experience (OR=0.53, 95%CI=0.46-0.60). The KWIC results revealed that 49 of the 78 (63%) occurrences of the word "rude" in the comments were related to receptionists and 17(22%) were related to doctors.

Conclusions: Web-based text processing tools can extract useful information from free-text comments and the output may serve as a springboard for further investigation. Text clouds, distinctive words extraction and KWIC analysis show promise in quick evaluation of unstructured patient feedback. The results are easily understandable, but may require further probing such as KWIC analysis to establish the context. Future research should explore whether more sophisticated methods of textual analysis (eg, sentiment analysis, natural language processing) could add additional levels of understanding.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.2196/medinform.3783DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4439523PMC
May 2015

Understanding high and low patient experience scores in primary care: analysis of patients' survey data for general practices and individual doctors.

BMJ 2014 Nov 11;349:g6034. Epub 2014 Nov 11.

Cambridge Centre for Health Services Research, Institute of Public Health, Forvie Site, University of Cambridge School of Clinical Medicine, Box 113 Cambridge Biomedical Campus, Cambridge CB2 0SR, UK.

Objectives: To determine the extent to which practice level scores mask variation in individual performance between doctors within a practice.

Design: Analysis of postal survey of patients' experience of face-to-face consultations with individual general practitioners in a stratified quota sample of primary care practices.

Setting: Twenty five English general practices, selected to include a range of practice scores on doctor-patient communication items in the English national GP Patient Survey.

Participants: 7721 of 15,172 patients (response rate 50.9%) who consulted with 105 general practitioners in 25 practices between October 2011 and June 2013.

Main Outcome Measure: Score on doctor-patient communication items from post-consultation surveys of patients for each participating general practitioner. The amount of variance in each of six outcomes that was attributable to the practices, to the doctors, and to the patients and other residual sources of variation was calculated using hierarchical linear models.

Results: After control for differences in patients' age, sex, ethnicity, and health status, the proportion of variance in communication scores that was due to differences between doctors (6.4%) was considerably more than that due to practices (1.8%). The findings also suggest that higher performing practices usually contain only higher performing doctors. However, lower performing practices may contain doctors with a wide range of communication scores.

Conclusions: Aggregating patients' ratings of doctors' communication skills at practice level can mask considerable variation in the performance of individual doctors, particularly in lower performing practices. Practice level surveys may be better used to "screen" for concerns about performance that require an individual level survey. Higher scoring practices are unlikely to include lower scoring doctors. However, lower scoring practices require further investigation at the level of the individual doctor to distinguish higher and lower scoring general practitioners.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4230029PMC
http://dx.doi.org/10.1136/bmj.g6034DOI Listing
November 2014

The feasibility of collecting information from people with Multiple Sclerosis for the UK MS Register via a web portal: characterising a cohort of people with MS.

BMC Med Inform Decis Mak 2012 Jul 18;12:73. Epub 2012 Jul 18.

College of Medicine, Swansea University, Swansea, Wales, UK.

Background: A UK Register of people with Multiple Sclerosis has been developed to address the need for an increased knowledge-base about MS. The Register is being populated via: a web-based portal; NHS neurology clinical systems; and administrative data sources. The data are de-identified and linked at the individual level. At the outset, it was not known whether people with MS would wish to participate in the UK MS Register by personally contributing their data to the Register via a web-based system. Therefore, the research aim of this work was to build an internet-mounted recruitment and consenting technology for people with Multiple Sclerosis, and to assess its feasibility as a questionnaire delivery platform to contribute data to the UK MS Register, by determining whether the information provided could be used to describe a cohort of people with MS.

Methods: The web portal was developed using VB.net and JQuery with a Microsoft SQL 2008 database. UK adults with MS can self-register and enter data about themselves by completing validated questionnaires. Descriptive statistics were used to characterise the respondents.

Results: The web portal was launched in May 2011, and in first three months 7,279 individuals registered on the portal. The ratio of men to women was 1:2.4 (n = 5,899), the mean self-reported age at first symptoms was 33.8 (SD 10.5) years, and at diagnosis 39.6 (SD 10.3) years (n = 4,401). The reported types of MS were: 15% primary progressive, 63% relapsing-remitting, 8% secondary progressive, and 14% unknown (n = 5,400). These characteristics are similar to those of the prevalent MS population. Employment rates, sickness/disability rates, ethnicity and educational qualifications were compared with the general UK population. Information about the respondents' experience of early symptoms and the process of diagnosis, plus living arrangements are also reported.

Conclusions: These initial findings from the MS Register portal demonstrate the feasibility of collecting data about people with MS via a web platform, and show that sufficient information can be gathered to characterise a cohort of people with MS. The innovative design of the UK MS register, bringing together three disparate sources of data, is creating a rich resource for research into this condition.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1186/1472-6947-12-73DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3444329PMC
July 2012

Use of live interactive webcasting for an international postgraduate module in ehealth: case study evaluation.

J Med Internet Res 2009 Nov 13;11(4):e46. Epub 2009 Nov 13.

University of Plymouth, Faculty of Health, Plymouth, UK.

Background: Producing "traditional" e-learning can be time consuming, and in a topic such as eHealth, it may have a short shelf-life. Students sometimes report feeling isolated and lacking in motivation. Synchronous methods can play an important part in any blended approach to learning.

Objective: The aim was to develop, deliver, and evaluate an international postgraduate module in eHealth using live interactive webcasting.

Methods: We developed a hybrid solution for live interactive webcasting using a scan converter, mixer, and digitizer, and video server to embed a presenter-controlled talking head or copy of the presenter's computer screen (normally a PowerPoint slide) in a student chat room. We recruited 16 students from six countries and ran weekly 2.5-hour live sessions for 10 weeks. The content included the use of computers by patients, patient access to records, different forms of e-learning for patients and professionals, research methods in eHealth, geographic information systems, and telehealth. All sessions were recorded-presentations as video files and the student interaction as text files. Students were sent an email questionnaire of mostly open questions seeking their views of this form of learning. Responses were collated and anonymized by a colleague who was not part of the teaching team.

Results: Sessions were generally very interactive, with most students participating actively in breakout or full-class discussions. In a typical 2.5-hour session, students posted about 50 messages each. Two students did not complete all sessions; one withdrew from the pressure of work after session 6, and one from illness after session 7. Fourteen of the 16 responded to the feedback questionnaire. Most students (12/14) found the module useful or very useful, and all would recommend the module to others. All liked the method of delivery, in particular the interactivity, the variety of students, and the "closeness" of the group. Most (11/14) felt "connected" with the other students on the course. Many students (11/14) had previous experience with asynchronous e-learning, two as teachers; 12/14 students suggested advantages of synchronous methods, mostly associated with the interaction and feedback from teachers and peers.

Conclusions: This model of synchronous e-learning based on interactive live webcasting was a successful method of delivering an international postgraduate module. Students found it engaging over a 10-week course. Although this is a small study, given that synchronous methods such as interactive webcasting are a much easier transition for lecturers used to face-to-face teaching than are asynchronous methods, they should be considered as part of the blend of e-learning methods. Further research and development is needed on interfaces and methods that are robust and accessible, on the most appropriate blend of synchronous and asynchronous work for different student groups, and on learning outcomes and effectiveness.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.2196/jmir.1225DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2802565PMC
November 2009

Geographic Information Systems for healthcare organizations: a primer for nursing professions.

Comput Inform Nurs 2009 Jan-Feb;27(1):50-6

School of Nursing and Community Studies, Faculty of Health and Social Work, University of Plymouth, Devon, UK.

The sharing of spatial information among members of the health sector can have vast strategic and operational benefits. Geographic Information Systems, or GIS, can be a key technology in optimally using this information. There are two types of applications under GIS: (1) studying health outcomes and epidemiology and (2) studying and informing healthcare delivery. With the advent of GIS that can be used over the Internet, a wider audience of decision makers and stakeholders now has the opportunity to use these technologies through something as simple as a Web browser. There is a small but growing number of published articles giving examples of using GIS for nursing practice and research. However, increased efforts are needed to make nurses, other health professionals, and health organizations aware of the possibilities of these information products for empowering their decision making. An incremental "capacity building" approach is proposed as the best way forward for sustainable and sustained nursing GIS development. The aims of this article are (1) to provide a brief nontechnical overview for readers not familiar with GIS, (2) to provide a framework for the adoption of GIS in health service organizations, and (3) to identify ways in which GIS can impact on the nursing management of patients.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1097/NCN.0b013e31818e4660DOI Listing
February 2009

Wikis, blogs and podcasts: a new generation of Web-based tools for virtual collaborative clinical practice and education.

BMC Med Educ 2006 Aug 15;6:41. Epub 2006 Aug 15.

Faculty of Health and Social Work, University of Plymouth, Drake Circus, Plymouth, Devon PL4 8AA, UK.

Background: We have witnessed a rapid increase in the use of Web-based 'collaborationware' in recent years. These Web 2.0 applications, particularly wikis, blogs and podcasts, have been increasingly adopted by many online health-related professional and educational services. Because of their ease of use and rapidity of deployment, they offer the opportunity for powerful information sharing and ease of collaboration. Wikis are Web sites that can be edited by anyone who has access to them. The word 'blog' is a contraction of 'Web Log' - an online Web journal that can offer a resource rich multimedia environment. Podcasts are repositories of audio and video materials that can be "pushed" to subscribers, even without user intervention. These audio and video files can be downloaded to portable media players that can be taken anywhere, providing the potential for "anytime, anywhere" learning experiences (mobile learning).

Discussion: Wikis, blogs and podcasts are all relatively easy to use, which partly accounts for their proliferation. The fact that there are many free and Open Source versions of these tools may also be responsible for their explosive growth. Thus it would be relatively easy to implement any or all within a Health Professions' Educational Environment. Paradoxically, some of their disadvantages also relate to their openness and ease of use. With virtually anybody able to alter, edit or otherwise contribute to the collaborative Web pages, it can be problematic to gauge the reliability and accuracy of such resources. While arguably, the very process of collaboration leads to a Darwinian type 'survival of the fittest' content within a Web page, the veracity of these resources can be assured through careful monitoring, moderation, and operation of the collaborationware in a closed and secure digital environment. Empirical research is still needed to build our pedagogic evidence base about the different aspects of these tools in the context of medical/health education.

Summary And Conclusion: If effectively deployed, wikis, blogs and podcasts could offer a way to enhance students', clinicians' and patients' learning experiences, and deepen levels of learners' engagement and collaboration within digital learning environments. Therefore, research should be conducted to determine the best ways to integrate these tools into existing e-Learning programmes for students, health professionals and patients, taking into account the different, but also overlapping, needs of these three audience classes and the opportunities of virtual collaboration between them. Of particular importance is research into novel integrative applications, to serve as the "glue" to bind the different forms of Web-based collaborationware synergistically in order to provide a coherent wholesome learning experience.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1186/1472-6920-6-41DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1564136PMC
August 2006

Linking primary care information systems and public health vertical programs in the Philippines: an open-source experience.

AMIA Annu Symp Proc 2005 :311-5

Medical Informatics Unit, University of the Philippines Manila College of Medicine, Manila, Philippines.

Community-based primary care information systems are one of the building blocks for national health information systems. In the Philippines, after the devolution of health care to local governments, we observed "health information system islands" connected to national vertical programs being implemented in devolved health units. These structures lead to a huge amount of "information work" in the transformation of health information at the community level. This paper describes work done to develop and implement the open-source Community Based Health Information Tracking System (CHITS) Project, which was implemented to address this information management problem and its outcomes. Several lessons learned from the field as well as software development strategies are highlighted in building community level information systems that link to national level health information systems.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1560490PMC
February 2007

Linking primary care information systems and public health information networks: lessons from the Philippines.

Stud Health Technol Inform 2005 ;116:955-60

Medical Informatics Unit, University of the Philippines, College of Medicine, Pedro Gil Street, Manila,

Community-based primary care information systems are one of the building blocks for national health information systems. In the Philippines, after the devolution of health care to local governments, we observed "health information system islands" connected to national vertical programs being implemented in devolved health units. These structures lead to a huge amount of "information work" in the transformation of health information at the community level. This paper describes work done to develop and implement the open-source Community Based Health Information Tracking System (CHITS) Project, which was implemented to address this information management problem and its outcomes. Several lessons learned from the field as well as software development strategies are highlighted in building community level information systems that link to national level health information systems.
View Article and Find Full Text PDF

Download full-text PDF

Source
April 2016

Critically Appraised Topics (CAT) peer-to-peer network.

AMIA Annu Symp Proc 2003 :806

Medical Education and Biomedical Informatics, University of Washington, Seattle, WA, USA.

A peer-to-peer network of Critically Appraised Topics or CATs would allow sharing of relevant clinical information regarding specific clinical problems among physicians. This proposed network would enable clinicians to develop and share CATs to other users within the network. This poster describes a proposed implementation of a peer-to-peer internet based sharing of critically appraised topics in the Philippines.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1480064PMC
December 2004