Cognitive Walkthrough

Generally, Cognitive walkthrough is a technique that focuses on exploratory learning. Its a very good method for exposing assumptions about users and learning that can help designers create better experiences.
Number of ParticipantsFacilitatorsCategoryDurationLevel of Difficulty
51-2Test Phase30minModerate

Description

It is a usability evaluation procedure for a user interface to test its exploratory learning features i.e whether any first time user can be comfortable understanding and using the interface without any formal training or background. (Shekhar& Marsden,2018)
Cognitive walkthrough is based on a theory that users often learn how to use a product through a process of exploration not through formal training course. (Polson & Lewis,1990)
Cognitive Walkthrough is more of a task specific approach to usability as compared to heuristic evaluation which is a more holistic usability inspection method. It is done at the testing stage of a product’s development Cycle. A user who is not very conversant with the product is given a list of task to carry out and questions are asked from the users’ point of view. CW is basically a task-specific approach to doing usability testing.

Materials

  • Laptop
  • Projector
  • Sticky Notes
  • Pens & Paper

Preparation

The Cognitive walkthrough workshop participants will be selected based on the following criteria:
The user should be someone who hasn’t used the tested application before. This is to ensure that the user has no prior knowledge of the Interface, which will go a long way in highlighting usability issues as the user tries to navigate around the web application.
The user may be someone with an Intermediate level of IT proficiency. It is an important criteria because it helps the designer to test how intuitively users can navigate around the application.

  • Develop a persona for a typical user
  • Define tasks the user should perform within the application (possible to have a task flow, e.g. Login —> open browser and launch the website —> enter user credentials).

Step-by-Step Instructions

  1. Participants are admonished to perform all the tasks in a chronological manner, no step should be skipped.
  2. The follow up questions must be answered based on your experience while interacting with the Interface of the software.
  3. While Interacting with the Interface, it is important to write down some of your observations with the Software this will be talked about in the discussion section.
  4. All ideas are welcomed during this workshop, please feel free to talk to any of the moderators about what you experience while using the software.

For each step in the task, the following questions should be answered with a YES or NO. In the case of a NO response, this answer will be substantiated by a credible failure story which indicates that a usability problem has been identified.

  • Will the User understand how to start the task?
  • Are the controls easy to locate and interact with?
  • Will the User associate the correct action with the effect trying to be achieved?
  • Was there feedback to indicate you completed (or did not complete) the task?

Remarks, Tips, Limitations

To highlight all usability issues on the system during the workshop, it is important for the facilitators to take note of the following:

  • For all tasks that the user answers ‘yes’ , this answer should be substantiated by a credible success story for that question. And in the event of a ‘No’, it should be substantiated with a credible failure story.
  • While the users are executing the task, the facilitators should also record side issues such as (spelling errors, as well as design changes .
  • The purpose of the tasks/questions is to focus on how the users goals interact with the visible state of the interface to induce the user to take action.


Limitations

  • Cognitive walkthrough is not suited for identifying problems experienced by experts
  • This method of evaluation is not too good for assessing the efficiency and satisfaction of an interface.
  • The method is laborious and slow , and coverage is often limited to a relatively small number of tasks.
  • The cognitive walkthrough has a narrower focus then other inspection methods such as heuristic evaluation.
Strengths: Great for anticipating usability problems before the design is developed. Requires few resources. Can identify multiple usability problems.

Weaknesses: Not suited for identifying problems experienced by trained users. Focuses only on specific tasks (limited method). Can not assess efficiency and satisfaction of an interface.

References

Rieman, J., Davies, S., Hair, D. C., Esemplare, M., Polson, P., & Lewis, C. (1991, March). An automated cognitive walkthrough. InProceedings of the SIGCHI conference on human factors in computing systems(pp. 427-428).
Mahatody, T., Sagar, M., & Kolski, C. (2007). Cognitive Walkthrough for HCI evaluation: basic concepts, evolutions and variants, research issues. InProceedings EAM 2007 European Annual Conference on Human-Decision Making and Manual Control, Technical University of Danemark, Lyngby.
Rieman, J., Franzke, M., & Redmiles, D. (1995, May). Usability evaluation with the cognitive walkthrough. InConference companion on Human factors in computing systems(pp. 387-388).
http://blogs.ubc.ca/cpsc544/files/2017/08/3-s2.0-B978012410391700004X-main.pdf
Shekhar, A., & Marsden, N. (2018, May). Cognitive Walkthrough of a learning management system with gendered personas. InProceedings of the 4th Conference on Gender & IT(pp. 191-198).
http://blogs.ubc.ca/cpsc544/files/2017/08/3-s2.0-B978012410391700004X-main.pdf
https://ext.eurocontrol.int/ehp/?q=node/1622#:~:text=Disadvantages%3A,problems%20experienced%20by%20trained%20users
https://medium.com/user-research/cognitive-walkthroughs-b84c4f0a14d4

Contributed by Charles Ajiboye.