User Study Test Plan
Main Research Goal
How to design a low-tech assistive device to increase the everyday independence of someone with hemiplegia through additions in their home environment? From the above research goal a set of further defined questions were created. These are then used to create measurable data points to evaluate the readiness of the product to be put onto the market. Prototype testing can also be used to discover a new set of requirements that may have been overlooked, find and resolve unnoticed issues. User testing can get users excited about the product and uses less resources than to modify it after it has already been launched.
- What are the participant’s limitations?
- Defines a basis for comparison between a situation with and without the product.
- What are the needs of the participant?
- Defines the level of necessity of the product for the participant and how much is the possible impact of the product on the participant’s everyday life.
- Is the size and the weight of the product comfortable to use?
- Defines if the product size does not negatively impact the environment. The weight of the product should not put extra strain on the participant’s right shoulder (right shoulder tendon rupture)
- Is the design intuitive and easy to use?
- Can the participant understand and use the product effectively without external assistance?
- Is the product safe?
- Does the product reduce risk of injury to the user’s environment?
- What is the likelihood that the product will not be abandoned?
- Will the product be used in the long-term/will users see the added benefits of the product
- Does the product allow the user to complete their tasks effectively?
- Does the product achieve the desired tasks of the user without external help or additional steps.
Test Preparations
To prepare for testing, the fidelity of the product needs to be examined to establish appropriate methods. The product is later on in the development phase where main functionalities and requirements have been iterated, improved and detailed upon with the help of the co-designer. Due to this a high-fidelity prototype can be used in which most functions of the cutting board are implemented, the majority of the experience is realised and this can give greater insight into the adoption and longevity of the product. To save resources, cheaper and adaptable materials can be used for the prototype. Plastic would have been the final chosen material but can be made with wood that can be better modified. To perform usability testing with our co-designer, their permission would be required. This can be done with an informed consent form that has been clearly detailed and understood by the co-designer, the nature and purpose of the test as well as the handling of the data and possibility to withdraw their consent. The informed consent form would also be used to ask for permission to record video and audio during testing. The testing would be done in the home environment of the co-designer, this gives real-world testing which can give great insight into how well the product performs for the user. It will also allow for greater understanding of possible unforeseen situations and points of improvement. The time and date would have to suit both parties, the co-designer and the researchers. A small number of researchers would be present (preferably 2 – 4) with the permission of the co-designer to ensure their comfort.
Moderated Comparative Testing
The initial testing method would be moderated comparative testing via observations and an interview. The test will consist of two rounds of observations, the first round would be done before introducing the product to gain insight into the co-designer’s current cooking routine/method. It gains more detailed information into the user’s limitations and provides a basis to the level of independence of the co-designer. A moderator would give the co-designer a set list of general cooking tasks that would need to be completed. This gives a reference point or baseline to better gauge the level and nature of impact from the product. The tasks should be general enough not to guide the co-designer in a specific way of thinking/working. The second observation would be a repeat of the first but with the prototype of the cutting board. After each observation an interview is conducted. With a set of predetermined questions, these would be used to gather qualitative data on the product in comparison to the current environment. It will also gather data that may not have been observed or that is difficult to determine from observations, such as user satisfaction.
Below is a preliminary observation list and set of interview questions.
Observation List
Interview Questions
- On a scale of 1-5 (1 being the worst and 5 being the best) how would you rate the ease of use?
- On a scale of 1-5 (1 being the worst and 5 being the best) was the product comfortable to use?
- On a scale of 1-5 (1 being the worst and 5 being the best) was the product safe to use?
- On a scale of 1-5 (1 being the worst and 5 being the best) was the product intuitive enough to use?
- On a scale of 1-5 (1 being the worst and 5 being the best) how satisfied were you with the product?
- What are some issues that you have with the product, if there are any?
- On a scale of 1-5 (1 being the worst and 5 being the best) how willing are you to use the product in the long-term?
- If any, what are some ideas to improve the product?
Analysing the results
From the moderated comparative testing method the prototype can be further refined. This testing method can be repeated if deemed necessary by the researchers and co-designer. The results are divided into their different themes being ease/convenience, needs and limitations, intuition, comfort, satisfaction and functionality. From this thematic analysis within each theme an overall ranking can be done and possible points of improvement identified. The theme of needs and limitations can be used to better identify if the product addresses them and in what ways. Additionally, it can be used if the product offers its own set of new problems that may have been overlooked during the design phase. In analysing the results, using the observations, a comparison can be made to see if the product increases the user’s ease in completing cooking tasks. This can also be gauged by their time to complete a task as well as their response during the interview. To estimate if the product would be abandoned, the co-designer can be asked about their willingness to use the product in the long-term. Between each repeat of the test, the prototype is further refined with results taken from the previous test. The thematic analysis is also repeated between each test. The tests can also be used to compare prototypes and used to identify what were the most optimal ways to solve specific problems. If little to no improvement can be made and all related research questions have been positively answered then the product may be ready for its last phase of testing. The prototype would be expected to score higher in all aspects as compared to the baseline test.
Possible Improvements to the user evaluation
To achieve more accurate results the previously stated methods can be repeated with multiple participants. Due to the specificity of the target audience (people with hemiplegia) the majority of the participants should have similar conditions. Additionally, the prototype can be tested with other groups of people. This will then add another research question of “what are the possibilities of the product being appropriated by other target groups?” Majority of participants should fit a similar demographic with a few outliers to explore possibilities that the product could better fit into another target group. The number of participants is dependent on the amount of available resources. When using multiple participants the number of times the test can be done is significantly decreased. The high-fidelity prototype should be in its last stages of development. Ideally, the moderated comparative testing would be done once unless a significant number of issues have been discovered. The final research question, which was the likelihood of abandonment of the product, would be better answered over a period of time. Due to this process taking a lot more resources it would be done with the final prototype, which has all of the functional qualities. It is then given to the co-designer/participants to use over a duration of time stretching from either a couple of weeks to a couple of months. Periodically, during this testing period interviews are set up either in person or remotely between the researchers and participant/s. The interviews should be spaced out between each other and allow the participants to naturally use or disuse the prototype. The interviews are designed to be a standard list of questions used to gauge the level of use, satisfaction and to explore any unforeseen issues with the prototype. From this the results can be analysed to determine if the product would actually be used in the long-term by its intended audience and can also gauge what may be some concerns from consumers if the product were to be launched. The results from this test can be done similarly to the moderated comparative test using similar themes with the additional theme of abandonment. The use of the previous themes can be used to see how these change over time and if there are any necessary changes to be made to the design. Taking considerations from the involved stakeholders the product can be assumed to be ready for launch.