Evaluation
John Alexis Guerra Gómez| ja.guerrag[at]uniandes.edu.co| @duto_guerra
Jose Tiberio Hernández | jhernand[at]uniandes.edu.co
Universidad de los Andes
http://johnguerra.co/lectures/visualAnalytics_fall2017/10_HCI_Evaluation/
Based onslides from Tamara Munzner
Evaluation
- Controlled Experiments involving users
- Natural settings involving users
- Any setting not involving users
Expert reviews
- Design Experts
- Visualization Experts
- Usability Experts
- Domain Experts
Types of expert reviews
- Heuristic evaluation (golden rules)
- Guidelines review
- Consistency inspection
- Cognitive walkthrough
- Metaphors of human thinking
- Formal usability inspection (courtroom style)
- Accesibility inspection
8 golden rules of design
- Strive for consistency
- Cater for universal usability
- Offer informative feedback
- Design dialogs to yield closure
- Prevent errors
- Permit easy reversal of actions
- Support internal locus of control
- Reduce short-term memory load
Controlled Experiments
- Experiments in the lab
- Controlled confounding variables
- Measure one or more quantitative variables
-
- Usability testing
- Living labs
What to measure?
- Time to learn
- Speed of performance
- Rate of errors
- Retention over time
- Subject satisfaction
Usability Testing
Natural settings involving users
- Observation
- Interviews
- Logging
Triangulation
Different researchers observe the same effect
Interviews
- Unstructured
- Structured
- Semi-structured
- Focus group
- Telephone/Online interviews
Questionnaire
Like interviews but without the researcher present
Likert Scale
What do you think?
- Strongly disagree
- disagree
- ok
- agree
- Strongly agree
More about likert scales
- Can be 3, 5, 7 or more responses
- Continuos or Discrete
- Middle response is the balance
Observation
- User's setting
- Can be direct or indirect
Direct Observation in the field
Ethnography
Direct Observation in controlled environments
- Think aloud techniques
Direct Observation: tracking users
- Diaries
- Interaction Logs and web analytics
MILCS
- Multi-dimensional
- In-depth
- Long-term
- sase studies
Focus groups
One researcher, many attendees
Prototyping
- Low vs high fidelity?
- read data
- build scenarios, tell a story
Quatitative evaluation
http://yatani.jp/teaching/doku.php?id=hcistats:anovaReferences
- Shneiderman, B. and Plaisant, C., 1987. Designing the user interface: Strategies for effective human-computer interaction
- Rogers, Y., Sharp, H., Preece, J. and Tepper, M., 2007. Interaction design: beyond human-computer interaction.
- Martin, D.W., 2007. Doing psychology experiments. Cengage Learning.
Evaluation
John Alexis Guerra Gómez| ja.guerrag[at]uniandes.edu.co| @duto_guerraJose Tiberio Hernández | jhernand[at]uniandes.edu.coUniversidad de los Andes
http://johnguerra.co/lectures/visualAnalytics_fall2017/10_HCI_Evaluation/
Based onslides from Tamara Munzner