Usability Testing Report 20190506
Hanna Alatalo
Jarkko Kuivanen
Elisa Nauha
Jere Ojala
Kimmo Urtamo
Version 1.0.0
Public
5.6.2019
University of Jyväskylä
Faculty of Information Technology
Info
Title of Document: Usability Testing Report 20190506
File:
Abstract: The functionalities developed into virtual learning environment called TIM by the TIMCAN project will replace already existing service called ICAnDoiT. ICAnDoiT is an application used by The Centre for Applied Language Studies (CALS) for the purpose of studying adaptive feedback. The adaptive feedback is feedback that becomes gradually more detailed with every wrong response. The project team will develop a dropdown menu plugin different from the original ICAnDoiT, a drag-and-drop plugin, adaptive feedback plugin and functionality for exporting the test results in CSV file.
The document reports the results and improvement suggestions of the test session performed on 6.5.2019.
Keywords: Usability, usability testing, learning enviroment, project, testing, testing report, report template, TIM.
Change History
| Versio | Date | Changes | Author |
|---|---|---|---|
| 0.0.1 | 6.5.2019 | The document was created and tables of the report was filled. | JK |
| 0.0.2 | 13.5.2019 | The document was finished for internal check. | JK |
| 0.1.0 | 14.5.2019 | Typos were fixed. | JK |
| 0.2.0 | 20.5.2019 | Edited document according to suggestions. | JK |
| 0.2.1 | 21.5.2019 | Edited document according to suggestions. | JK |
| 1.0.0 | 5.6.2019 | The document was approved by the customer. | KU |
1. Tested Functionalities, Goals and Methods of Test Session
The tested functionality was creation of the task with dropdown question items. Usability testing focused on the discoverability, ease of use and learnability of the tested functionality. The goal of usability testing was to find shortcomings, errors and problems related to the tested functionality.
2. Suitability and Knowledge of the Test Subject and Observer
The test subject and the observer met the requirements specified in the test plan. The test subject was a person appointed by the customer. Based on the background information interview (results in Table 1), test subject knew the concept of adaptive feedback. He had no previous experience of ICAnDoiT or TIM learning environment.
During the test session the test subject expressed his thoughts aloud as he was advised.
| Question | Answer |
|---|---|
| The role of test subject: | researcher |
| The test subject knows the concept of adaptive feedback: | yes |
| The test subject has used ICAnDoiT application: | no |
| The test subject has participated test in ICAnDoiT application: | no |
| The test subject has created test in ICAnDoiT application before: | no |
| The test subject could guide novice user to create a test in ICAnDoiT application: | no |
| The test subject has used TIM learning environment before: | no |
| The test subject has read TIM documents and know how to navigate in a TIM folder system: | no |
| The test subject has created TIM documents: | no |
| The test subject could guide novice user to create a TIM document: | no |
3. Identification Information of the Test Session
The test environment and identification information of test session are represented in the following table.
| Parameter | Identification information |
|---|---|
| The date of test session: | 6.5.2019 |
| The starting time of test session: | 14:09 |
| The ending time of test session: | 15:00 |
| The observer of test session: | Jarkko Kuivanen |
| The subject of test session: | Ghulam Khushik |
| The assisting persons: | Elisa Nauha |
| The test plan and version: | Test Plan for Usability Testing in the TIMCAN project version 0.1.2 |
| The place of test session: | Quiet office (Ag C518.2) |
| The software and version: | TIMCAN branch of TIM |
| The operating system and version: | Windows 10 Home 10.0.17134 |
| The web browser and version: | Firefox 66.0.3 |
| The hardware setup: | Laptop with mouse for assignments and desktop computer with mouse for manuals. |
| The test server (address): | https://timdevs01-2.it.jyu.fi |
| The functionalities to be tested: | Creating task with dropdown items |
| The test data used for testing: | None |
4. Summary and Recommendations of Test Session
| Summary data | Number |
|---|---|
| The number of functionalities tested | 1 |
| The number of assignments in total | 2 |
| The number of completed assignments | 1 |
| The number of unfinished assignments | 1 |
| The number of observations in total | 30 |
| The number of positive observations | 1 |
| The number of negative observations | 27 |
| The number of critical observations | 0 |
| The number of neutral observations | 0 |
| The number of suggestions | 2 |
5. Summary of Observations of the Test Session
The test session consisted of the assignments defined in the usability test plan [1].
The section consist of observations which were recorded during the test session, analysis of the observations and suggested improvements based on analysis and observations.
Each recorded observation is ranked as one of the following:
- Critical: If a bug or nonintentional functionality is discovered.
- Negative: If the test subject complains or struggles with the assignment or a part of it.
- Positive: If the test subject advances exceptionally fast or they praise the functionality.
- Suggestion: If the observer or the test subject discovers an improvement that could make the application better.
- Neutral: If the observation does not fill the definition of a critical, negative or positive observation and is not a suggestion.
5.1 Creating a Task with Dropdown Items
During testing of the functionality the test subject was assisted and he used existing manuals. Assignment 1.1 was not be completed. The biggest problem of the test session was that the test subject didn´t have any previous experience about TIM learning environment. Because of that he was assisted constantly and use of TIM was difficult. According to the test subject the manuals were too long and he suggested video instructions for the application. However after editing one question item block he could edit others too. Editing feedback block was shocking for the test subject and he thought it was programming. The markdown was criticized.
The proposed improvements and their status are presented in the following table:
The recorded observations in assignments are shown in the following tables:
Assignment 1.1 Create the following task containing an instruction block, four question item blocks and a feedback block.
Assignment 1.2 Answer the questions asked by the observer.
References
[1] Hanna Alatalo, Jarkko Kuivanen, Elisa Nauha, Jere Ojala and Kimmo Urtamo, Test Plan for Usability Testing in the TIMCAN Project.
https://tim.jyu.fi/view/kurssit/tie/proj/2019/timcan/dokumentit/testaus/Test-plan-for-usability-testing-in-TIMCAN-project, University of Jyväskylä, Faculty of Information Technology, 2019.
These are the current permissions for this document; please modify if needed. You can always modify these permissions from the manage page.