Assignment 4: Heuristic evaluation
Paper prototypes for three versions of a home thermostat. From [Tohidi CHI2006].
Brief
Here's the strategy for this assignment
1. pick a concrete direction and
2. create two rapid interactive prototypes (Wireframes)
3. finally, the peer evaluation will introduce you to the concept of Heuristic Evaluation.
Assignment
Choose a Storyboard
Decision time. Set out your storyboards that you created in assignment #3. Take a deep breath.
Spend some time to reflect on the different ideas you've had. Make sure you discuss the strengths
and weaknesses of each design, and how well they achieve the goals set out by your point of view.
And then, decide on one of these ideas to prototype.
Create 2 Rapid, Interactive Prototypes
Make two rapid interactive prototypes that implement the idea you've decided on. (Can you see we
really value enumerating alternatives?) A rapid interactive prototype concretely shows all the
elements of a user interface, except that it's implemented with very high level tools (just like pen and
paper), as opposed to actual code. This is often called a ‘Wire Frame’, or ‘Wireframing.’
This time, your rapid interactive prototype will be done with Balsamiq Mockups. No Photoshop, no
Illustrator, no polished work, even if you are familiar with them! You will be sent an invitation to join
myBalsamiq.com. Once you receive it, click on the included link and create your account. Follow the
instructions in the blackboard video “Joining MyBalsamiq”. This video will get you started with
creating your prototype wireframe. For more in-depth training watch the “Balsamiq Tutorial” video
also on blackboard in the assignment #4 folder under Balsamiq Tutorials.
Again, rapid prototyping helps focus on the concepts, and saves you from wasting hours twiddling
pixels. If you haven't understood that yet, the goal is to be fast. Years after taking this course at
Stanford, students often come back and tell us that rapid prototyping (whether on paper or electronic
version) was their favorite part of the class because of its effectiveness for rapid ideation.
The prototypes you create should be interactive. That means that your user should be able to
navigate around it and get a feel for how the ‘real thing’ would work. This is not as complex as it
sounds! Balsamiq's built-in tools allow you to create "links" between screens and buttons to simulate
real functionality - back, next, home, etc - even though visually it is nothing more than a line drawing.
The prototypes should vary in interface, but offer the same basic functionality. For example, if you
were designing a mobile transit application, your two prototypes could display the bus times in two
very different ways. The prototypes should be complete enough to "run" a new user through each
task. (Note: the story here is very linear, but your process doesn't have to be. You can start making a
rapid prototype, then change your mind. And your prototype doesn't have to exactly align with your
storyboard.) Your prototype interface should enable people to navigate, recover from errors, and
change their mind.
(Although we are not going to do any paper prototyping in this assignment, being familiar with this
concept is interesting. For further information, see the book Paper Prototyping by Carolyn Snyder and
a few references).
As part of this assignment find someone to give you feedback about your completed and interactive
wireframe. This can be someone from your class or anyone else. Capture that feedback and have the
evaluator rate the application or website on a scale from 1 to 10 overall. One is the low-end and 10
would be perfect, indicating an application that accomplishes is rich in affordances with clear
navigation and obvious purpose. Then briefly describe in your submission what the evaluator liked
and/or disliked. In other words, why did they give the score they did? You do not need to act on any
evaluator recommendations at this time. This is just to get you familiar with the process of repeated
heuristic evaluations as your application and/or website develops. In later assignments you will be
required to capture much more evaluation information and show how it relates to the overall project
plan.
Student Examples
Prototype 1 Example
Prototype 2 Example
What’s this for? A UX agency perspective
by Mike Davison, Community TA and UX Project Manager
Wireframing is an important step in the design process. You may know how you want your application
to look and wish to skip ahead, - otherwise, you need to do the work twice, right? But this step is vital.
It allows you to decide on navigation, layout, menu options, application structure and much more,
making quick changes as you go.
Importantly, it allows you to TEST your plan with users before you spend many hours on visual
design. I recall a recent project that sprung some major surprises here; the user base disagreed
fundamentally with the ‘needs’ their corporate management had identified for them. It prompted a
drastic redesign...if the visual layer had been applied, we would have wasted tens of thousands more.
Think about it as if you were building a car. You do not start painting body panels and trimming the
interior before you decide where the engine is going to go!
Submit
Your write-up will contain:
•
•
•
Digital photos or scans of the storyboard you chose. Be sure to include detailed descriptions to
help in my understanding of exactly what you were trying to achieve.
A .PDF file your 2 rapid interactive prototypes wireframes from myBalsamiq.com. You can put
both your prototypes (mockups) inside one project.
Include in the comment section of the assignment area of blackboard an evaluator rating for
each of your interactive prototype wireframes. In a brief explanation describing what the
evaluator liked or disliked about the application and/or website.
Peer Evaluation
Common "I wish" Feedback
The following statements are common feedback given on this assignment. We call these statements
"I wish" feedback because they are a way to express things that you wish the assignment had
contained. You could think of these as common problems to avoid.
•
•
•
•
Prototype idea should try to address user needs in ways that are not similar to features of
commonly used services.
Prototype should relate to the user needs in the storyboard more.
Prototype should contain more of the interactions outlined in the storyboard.
Prototypes should diverge more.
Evaluation criteria & Grading rubric
The peer evaluation will introduce you to Heuristic Evaluations (HE). As you are grading your peers,
we will ask you to conduct heuristic evaluations (HEs) of the prototypes in order to complete the
prototyping phase of the project, providing you with the feedback you need to begin implementing.
Heuristic Evaluations
The heuristic evaluations will be a way to highlight usability issues in your rapid interactive prototypes.
Heuristic evaluations will follow the "How to Conduct a Heuristic Evaluation" and "Ten Usability
Heuristics" readings by Jakob Nielsen. Using Nielsen's ten heuristics, you as an evaluator will list as
many usability issues as possible. It is less important that the evaluators worry about exactly which
heuristic the usability issues occur in. The heuristics are there to help evaluators find a variety of
different types of usability issues. For each usability issue found the evaluator should capture a
description of the issue written as concisely and as specifically as possible.
Being an Expert Evaluator
As an evaluator, be thorough and write down all problems you can find. Don't try to be "nice" by not
reporting problems, everything you find will help your peer improve his interface. Since you are
evaluating two prototypes, try to make your feedback comparative. For example, if a prototype had a
big problem that you also found in the other prototype, make a note of it. If one prototype had a
problem that, for some interesting reason or another, was solved or not an issue in the other
prototype, make a note of that as well. You don't have to compare both prototypes in every sentence
in your evaluation (and it's even fine if most of your feedback isn't comparative), but you should
highlight enough similarities and differences between the two prototypes so that the peer receiving
your feedback will understand the advantages and drawbacks and each design.
Your feedback should help your peer understand and decide which features of which designs they
should implement in the coming weeks. Use Nielsen's heuristics as a guide and specify what
heuristic(s) each problem is related to (but if you come across problems that don't fall neatly into
particular heuristics, note them down also!). In addition, write the problem down with a severity rating
as in Nielsen's Severity Ratings for Usability Problems.
Go over both of your peer's prototypes. When grading your peers, write a heuristic evaluation of
the two prototypes. This comprises a bulleted list of usability issues you found, along with their
severity, for each prototype. Include comparative feedback between the two prototypes.
Evaluation criteria & Grading rubric
Category
Unsatisfactory
Bare minimum
Satisfactory effort
& performance
Above & Beyond
Prototype #1:
Clarity
No prototype or
completely
irrelevant prototype.
Many elements
have no defined
purpose, and it can
be difficult to know
how to use many
parts of the
prototype.
Some elements
have no defined
purpose, and it can
be difficult to know
how to use certain
parts of the
prototype.
All elements have a
clearly defined
purpose and it is
easy to know how
to use the prototype.
A programmer
could use the
prototype to create
a functional
application with a
defined flow.
Prototype #1:
Completeness
No prototype or
completely
irrelevant prototype.
1: The prototype
incomplete, as it is
not interactive or
interactions are
broken.
The prototype is
mostly complete,
although there are
some functions that
are not yet
interactive.
The prototype is
complete, with all
functions
interactive.
Prototype #2:
Clarity
No prototype or
completely
irrelevant prototype.
Many elements
have no defined
purpose, and it can
be difficult to know
how to use many
parts of the
prototype.
Some elements
have no defined
purpose, and it can
be difficult to know
how to use certain
parts of the
prototype.
All elements have a
clearly defined
purpose and it is
easy to know how
to use the prototype.
A programmer
could use the
prototype to create
a functional
application with a
defined flow.
Prototype #2:
Completeness
0: No prototype or
completely
irrelevant prototype.
The prototype
incomplete, as it is
not interactive or
interactions are
broken.
The prototype is
mostly complete,
although there are
some functions that
are not yet
interactive.
The prototype is
complete, with all
functions
interactive.
Prototype
divergence
No prototypes to
compare.
The prototypes do
not vary in interface
in any significant
The prototypes
somewhat vary in
interface, or they
The prototypes
explore clearly
different interfaces
way.
Outside the Box
+1. Up to 5% of submissions.
vary in interface but
address different
ideas.
addressing the same
idea.
Wonderfully creative choice of observation setting,
insights, and/or ideas.
Purchase answer to see full
attachment