My Portfolio

Hardware Prototype Usability Testing

<< BACK

6 minute read

Reinvisioning ways to make LC-MS column installation easier to decrease chromatography errors.

The Challenge
When a scientist uses LC-MS (liquid chromatography - mass spectrometry) to separate chemicals in a sample, they need to install an LC column into the instrument. Yet physical challenges abound. For example, a column that's not installed correctly -- even by millimeters -- can result in leaks, bad chromatographic data, wasted money, and wasted time.

The project team wanted to try out a few different machine-shop prototypes that they hoped would make column installation much easier for the user -- so easy, in fact, that even a layperson could do it -- but without causing any leaks. So the project team designed four different prototypes, and they wanted users to try them out in a usability test.

The Team
I was the UX researcher on the project team, and I worked with industrial designers, mechanical engineers, and a project manager. The designers and engineers designed the prototypes using a CAD program, a 3D printer, and various hardware.

The Prototypes
The prototypes were printed on a 3D printer and assembled in the machine shop. Four different prototypes were designed, with the third and fourth being additional iterations of the second prototype.

Still from a column installation video on the Waters website.

Normally, installing a column into an LC instrument properly is a multi-step process that includes screwing parts together tightly and aligning them in the instrument (and hoping that all goes well with the chromatography run). It's all the more challenging with less experienced users.

While I am not able to show photos of the prototypes due to their proprietary nature, I can describe the designs.
​
With the prototypes under test, the user would simply either lay the column down in the assembly (for the horizontal version) or drop the columns into the slots (for the vertical version), then close and lock the top. The prototype design eliminated the need to screw parts together.

Research Questions
The project team wanted the research study to answer the following questions:

• ​What is the baseline user experience of installing a column? What is the difference in this experience between experts and novices?
• What is the user experience of the physical manipulation of putting the column into the device?
• How do users interact with the prototypes? What works? What could be improved?
• [For the expert users] What are the differences, if any, in chromatographic run results between participants?

UX Research Method: Moderated In-person Usability Testing
I conducted moderated in-person usability testing in the Waters Experience Design Center (XDC) with Waters scientists -- experts and novices -- to gather user impressions and feedback on use of the four prototypes, compared to use of an actual Waters LC product. For each participant, the order of prototypes was randomized.

For each usability task, the start and end times were recorded to ascertain whether there were significant differences between participants, and more importantly, to help quantify the difference between expert and novice users.

In addition, a Single Ease Question (SEQ) -- on a 7-point scale -- was asked after each task was completed.

After the SEQ, the participants were asked to explain their rating, and then offer up three words they would use to describe the system or prototype they just used. For each system/prototype, word clouds were created for each collection of three words to allow the project team to visualize at a glance the differences and similarities between user impressions.

At the very end of each usability session, the participant was then asked to explain what they would change in the prototype if they could only change one thing. Finally, they were asked to pick which system they preferred.

Observations from the usability tests were categorized, and then scored on two levels:

• Impact: the level of identified impact to the user, the business, or both.
• Frequency: duplicate observations were grouped together and given a frequency score based on the number of users affected.

The Bottom Line
This usability study showed that the 3D printed prototypes had promise from a UX and interaction perspective, and the decision was made to continue to the next iteration in industrial and mechanical design. However, while the prototype designs greatly improved the user experience of installing a column into an LC instrument, the concern about leaks (leaking of sample liquid from the column to the tubing and then to the instrument) was just as high as it is with current LC instrumentation across various brands. Therefore, the user experience is not the only item of concern -- the integrity of the connection between column and tubing would still need to be top of mind for future designs.