Evaluative Research
Performance based usability testing and comparative evaluations provide quantitative rigor for making decisions during the medical product development lifecycle. Our testing capabilities include formal, lab-based testing of concepts, prototypes and existing medical products. We have developed relationships with research facilities across the United States and abroad for recruiting and conducting studies for medical product development on an on-demand basis. Medical product usability testing can measure many aspects of user-medical device interaction, including:
- Task performance accuracy, duration and speed
- Error rates and type
- Ergonomic fit and problems
These quantitative metrics can be combined with qualitative measures to assist with medical product development including:
- Perceived comfort and control
- Interface learn ability and ease-of-use
- Aesthetic preferences and feedback
The integration of objective and subjective inputs provide a valuable combination of inputs to drive medical product development and design decisions. For example, in a comparative study, two prototype medical instruments may have equivalent ergonomic fit, but different significantly in perceived comfort due to more subtle issues such as the center of gravity or even the visual appearance of the handle.
We also advocate the use of checklists during the medical product development process. The humble checklist has been receiving a good deal of attention in recent years, particularly in the context of medical error avoidance. The New England Journal of Medicine published A Surgical Safety Checklist to Reduce Morbidity and Mortality in a Global Population. The international study reported a nearly 50% decline in patient deaths when a combined written/verbal checklist was used in conjunction with surgical procedures. The authors diligently point out several possible sources of error that may have contributed to the results, including the Hawthorne effect, a "short-term improvement caused by observing worker performance."
But rather than seeing the Hawthorne effect as a source of error, we see it as a source of performance. That is, the checklist process serves as a continuing form of performance observation - by having to refer to an external reference, the surgical teams were consistently self-aware and performed better as a consequence.
Self-awareness is an important issue in successful medical product development as well. A major challenge in user research for product development is communicating recommendations or guidelines to a design and/or engineering team, with the goal of affecting the medical product development team's behavior towards a desired design outcome which can be evaluated during research processes both internally and among test users.
Design guidelines are typically communicated in a number of ways - reports, presentations, working sessions, collaborative discussions, etc. - but the checklist format can be an even more effective communication tool during the medical product development process. This is because of both the way checklists are created, and how they are used.
The checklist format requires that information is presented in specific, succinct and actionable terms. For example detailing the size of touch points, the labeling of a control, or the diameter of a handle. This forces the checklist creator (i.e. the researcher) to come to specific recommendations.
For the checklist readers on the medical product development team, the benefits of a structured list over a more vague presentation or lengthy report should be clear. But it’s also important to keep in mind the verbal component of checklist usage in the ultimate medical study with users. The check-and-balance approach of having open communication is probably as important, if not more, than the checklist itself.
Comparing the benefits of a checklist in the highly structured, rigorous surgical domain with the creative, diverse world of medical product development enables clear communication among medical product development team members and sets the stage for very effective generative testing with users through the process.