Preliminary Functional Testing Report of TIMCAN project

Hanna Alatalo
Jarkko Kuivanen
Elisa Nauha
Jere Ojala
Kimmo Urtamo

Version 0.3.0
Public
28.5.2019

University of Jyväskylä
Faculty of Information Technology
Jyväskylä

1. Test Session Identifications

The identifying information of the test session are presented in the following table:

Tester(s)	Elisa Nauha, Jere Ojala
Date	17.4.2019
Test plan version	TIMCAN Project Functional Testing Plan Version 0.0.15
Test cases	1.1-4.7
Test data	testaustesti1

2. Testing Environment

Application and version: TIM version 17.4.2019 of TIMCAN branch
Operating System: Windows 10 Enterprise version 1809
Hardware setup: Dell OptiPlex Desktop PC with a keyboard, a computer mouse and a computer screen.
Browser: Google Chrome Version 73.0.3683.86
Testing Server: https://timdevs01-2.it.jyu.fi

3. Test Cases for Building a Test

The test session results for the test cases for building a test are presented in the following table.

Case ID	Task and input	Conclusion	Details
1.1	Instruction page	OK
1.2	Task 1	OK
1.3	Task 2	OK
1.4	End page of test	OK

4. Test Cases for Student Answering the Test

The test session results for the test cases for answering the test as a student are presented in the following table.

Case ID	Task and input	Conclusion	Details
2.1	Beginning a test	OK
2.2	Correct streak	Note	OK button doesn't work. Only the link works.
2.3	Final feedback level	Note	Same as 2.2. Also previously answered question appeared with previous answers on.
2.4	Not choosing options	Note	Same as 2.2. The second task is drag & drop, instead of dropdown. The test ends after second task and doesn't continue.

5. Test Cases for Report Exporting

The test session results for the test cases for exporting the test results are presented in the following table.

Case ID	Task and input	Conclusion	Details
3.1	Exporting with default	Note	Unclear whether contains only one task.
3.2	Cancel	OK
3.3	Whole Test, Only username	Note	Unclear whether contains whole test.
3.4 A	nonymous username O	K
3.5	Tab-delimiters	Note	The image in the test plan is faulty.
3.6	Semicolon-delimiters	OK
3.7	Only one visible user	Note	The instruction on inserting 1 into the text field is unclear.
3.8	Two visible users	OK
3.9	Empty visible users list	Error	The results of Testuser 1 are visible in the report.
3.10	Only selected user	OK
3.11	Empty visible users list with selected user	Error	The results of Testuser 2 are visible in the report.

6. Building Task with Errors

The test session results for the test cases for error handling in the test are presented in the following table.

Case ID	Task and input	Conclusion	Details
4.1	Differing number of feedback levels	Note	Markdown has been altered since the test plan was compiled.
4.2	No link to next task	Error	Error did not appear.
4.3	No instruction plugin	Error	Error did not appear.
4.4	No right answer defined	Note	Error message is slightly different.
4.5	Wrong names for plugins	OK
4.6	No default feedback	Note	Removed wrong part of the markdown. Instructions could be clearer.
4.7	No feedback	Error	YAML errors also appears.

7. Summary

Executed test cases	26
Unexecuted test cases	0
Total test cases	26
OK conclusions	11
Note conclusions	10
Error conclusions	5
Total conclusions	26
Test session conclusion	Rejected

8. Recommended Procedures

The problems in the testing plan should be corrected before next test session.

The following errors in the application should be fixed:

• OK button at the end of the task should either work as a link to the next task or the end of test, or it should be removed.
• On drag-and-drop questions a reappearing question still has answers from the previous time it appeared. They should be reset.
• When acquiring a report of the answers of visible users or selected users from a filtered empty user list, the report should have been empty. However, it contained the answers of a test user. This should be fixed.
• Warnings about building tasks with errors need to be checked or added in.