Usability evaluation of mobile computer terminals: a focus group study

Funding: U.S. department of transportation

My Role: graduate leader

Timeline: October 2019-Feb 2020

Research Method: In-depth interviews, content analysis, qualitative data analysis

Motivation

Crashes involving emergency vehicles including police vehicles, fire trucks, and ambulances are a national and international concern that cause significant injuries, fatalities and property damage. Among all the emergency responders, law enforcement officers (LEOs) are involved in a higher number of fatal crashes as compared to firefighters and emergency medical services workers. Motor vehicle crashes account for almost 40% of fatal work injuries for LEOs and have been mainly attributed to use of in-vehicle technologies and driver distraction. Police in-vehicle technology use and especially the MCT was the main cause of LEO vehicle crashes as suggested by the police crash fact reports.

Research objective

The objective of this study is to provide a comprehensive assessment of MCT usability issues in police vehicles.

Methods

Participants:

Fifteen police officers (14 males and one female) from different departments across the U.S. volunteered to participate in this study. Below shows the demographic information of the participants:

Data analysis:

Following the meeting, the session was fully transcribed based on the collected recordings and supplemented field notes. The transcript was then analyzed by all three members of the research team individually based on the directed content analysis approach, separating the responses provided by the participants on the basis of how they were related to the established usability principles. The identified usability issues were then categorized by their applicable usability principle based on Molich and Nielsen’s usability principles. The team members individually categorized the usability issues based on the nine categories of usability principles including customizability, naturalness, consistency, preventing errors, minimizing mental workload, efficient interaction, feedback, effective use of language, and effective information presentation. Krippendorff’s alpha (α) coefficient was calculated to ensure inter-rater reliability between the opinions and analyses of the three researchers.

The coefficient ranging from 0.61 to 0.80 and from 0.81 to 1.0 indicates substantial and perfect agreement, respectively (Krippendorff, 2018). The three researchers individually rated usability issues based on the nine categories of usability principles. The Krippendorff’s Alpha coefficient was calculated as 0.67 with 95% confidence level. Results indicated that the researchers had a substantial agreement in categorizing the usability issues.

Results

MCT design guidelines

Impact

Industrial impact:
The findings of this study can be useful for MCT manufacturers and police agencies in order to enhance future MCT interfaces and modify departmental policies or guidelines regarding the use of MCTs while driving.
Academic impact:
Shahini, F., Wozniak, D., & Zahabi, M. (2021). Usability Evaluation of Police Mobile Computer Terminals: A Focus Group Study. International Journal of Human–Computer Interaction, 37(15), 1478-1487.