Evaluation

• Controlled Experiments involving users
• Natural settings involving users
• Any setting not involving users

Expert reviews

• Design Experts
• Visualization Experts
• Usability Experts
• Domain Experts

Types of expert reviews

• Heuristic evaluation (golden rules)
• Guidelines review
• Consistency inspection
• Cognitive walkthrough
• Metaphors of human thinking
• Formal usability inspection (courtroom style)
• Accesibility inspection

8 golden rules of design

• Strive for consistency
• Cater for universal usability
• Offer informative feedback
• Design dialogs to yield closure
• Prevent errors
• Permit easy reversal of actions
• Support internal locus of control
• Reduce short-term memory load

Controlled Experiments

• Experiments in the lab
• Controlled confounding variables
• Measure one or more quantitative variables
• • Usability testing
  • Living labs

What to measure?

• Time to learn
• Speed of performance
• Rate of errors
• Retention over time
• Subject satisfaction

Usability Testing

Natural settings involving users

• Observation
• Interviews
• Logging

Triangulation

Different researchers observe the same effect

Interviews

• Unstructured
• Structured
• Semi-structured
• Focus group
• Telephone/Online interviews

Questionnaire

Like interviews but without the researcher present

Likert Scale

What do you think?

• Strongly disagree
• disagree
• ok
• agree
• Strongly agree

More about likert scales

• Can be 3, 5, 7 or more responses
• Continuos or Discrete
• Middle response is the balance

Observation

• User's setting
• Can be direct or indirect

Direct Observation in the field

Ethnography

Direct Observation in controlled environments

• Think aloud techniques

Direct Observation: tracking users

• Diaries
• Interaction Logs and web analytics

MILCS

• Multi-dimensional
• In-depth
• Long-term
• sase studies

Focus groups

One researcher, many attendees

Prototyping

• Low vs high fidelity?
• read data
• build scenarios, tell a story

Quatitative evaluation