What is the course about?

This course presents a wide variety of methodologies for evaluating technological interfaces with users. It is aimed at providing engineering students with the necessary knowledge on working with users, statistical techniques for analyzing the information gathered from the users, and how to interpret the results from the user studies.

Logistics

When and where?

Earth & Marine Sciences B214, Tuesdays and Thursdays 12:00-1:45

Who to contact?

Sri Kurniawan: Instructor

Assessments

  • Take home exam: 30%, due Nov 17, 2015

This exam requires you to perform data snooping on the quantitative and qualitative data from a survey that we conducted. The survey and the csv file  will be released on an appropriate date. Just like in real life, the data is messy, some people probably filled in the survey more than once (there was a draw for two $100 Amazon gift cards for those who completed the survey), and we probably did not phrase the questions appropriately. The deliverable is a paper (no page limit, please use the ACM template) that has the following sections:

  1. Introduction: why you chose to investigate this particular case (e.g., why are you comparing data from older vs. younger people, or men vs. women, etc?).
  2. Data analysis: quantitative and qualitative (at least 3 variables each) - you can do whatever you wish, correlating them, grouping them based on one or more variable, etc. Let's assume any numerical (or those that can be converted to numerical) are scale and only need parametric statistics. I also need you to do content analysis on a subset of the data (coding all will take a while). For the quantitative analysis, I also need to know which data you discarded and why (not row by row, explain at a high level, e.g., you threw away all participants above 65 years old of age because there were only 2 of them). For the qualitative analysis, explain the sampling strategy, e.g., every 5 rows. 
  3. Discussion and conclusion. For discussion, I expect to see an analysis of your findings, relating qualitative and quantitative data (e.g., "Women on prepaid scheme spent significantly less than men on similar scheme because they fear for brain cancer (70% women in this sample stated this fear compared to 40% men)." I also would like you to critique the original survey (the phrasing of the question, the options, the order of questions, or anything else you feel should be different). For the conclusion, I would like you to summarize your findings, and if relevant, the take home messages for relevant researchers or industry. 
  • Student-led class presentations: 20%. One group of students (most likely of 2-3 students) will be assigned to learn more about a certain topic and find papers relevant to that topic. The group may be asked to synthesize the chapters in one of the textbooks, suggested book and those papers; when appropriate collect/analyze some data as a proof of concept of the topic; and present the synthesis/data analysis in class. This presentation should be presented in lecture format, around 30 minutes long, followed by 15 minutes of class discussion. The ppt or pdf file needs to be sent to me the day before the presentation so that I can upload it to the class website.
  • Student-led class discussions: 10%. Another group of students will be asked to read the papers that accompany that week's presentation. This group will present a summary of the two studies and then will critique the papers for "flaws" in terms of user evaluation (e.g., the experimental methods, the sampling approach, the data analysis used). The rest of the class is then expected to start the discussion/debate - you have 20 minutes for this exercise, it is up to the group how to use this time slot. The ppt or pdf file needs to be sent to me the day before the presentation so that I can upload it to the class website.
  • Individual project: 40%.
    For this project I need you to recruit an appropriate number of users to evaluate a technological interface of your own choosing. You need to collect both qualitative and quantitative data, analyze the data, interpret them and provide conclusions in terms of improvements to the technological interface to make it more user-friendly. 10% of this grade is for the presentations during the final exam slot, and 30% is for the report. 
    As a reference point, this is the sort of data gathering and analysis I am expecting you to do in your project: A paper on Older people and mobile phones: A multi-method investigation. The deliverables for the final exam are:
    1. A report in the format resembling the example paper above.
    2. 10 minutes presentation covering: motivation (why did you choose that particular interface), methodology (participants, procedures, measures), results, discussion and conclusion.
    3. 5 minutes Q&A.

Policies and accommodation

  • All students enrolling in this class are advised that Academic Integrity will be strictly enforced. Academic misconduct includes but is not limited to cheating, fabrication, plagiarism, or facilitating academic dishonesty or as further specified in campus regulations.
  • If you qualify for classroom accommodations because of a disability, please submit your Accommodation Authorization from the Disability Resource Center (DRC) to me in a timely manner, preferably within the first two weeks of the quarter. Contact DRC at 459-2089 V, 459-4806 TTY.

Readings/resources (subject to change)

  1. Textbook: Cairns, P. & Cox, A.L. (2008). Research Methods for Human-Computer Interaction. Preface is here. Chapter 1. Chapter 2. Replacement of Chapter 3. Chapter 4. Chapter 7.
  2. Textbook: Graziano, A. M., & Raulin, M. L., 8th ed. Research methods: A process of inquiry.
  3. Lazar, J., Feng, J. & Hochheiser, H. (2010). Research Methods in HCI.
  4. Usability Evaluation Methods
  5. Emerging research methods for understanding mobile technology use. In Proceedings of OZCHI, pp. 1-10.
  6. Various papers assigned in class.
  7. IRB.