# Tests¶

Within a Category you can create the actual tests. There are several different tests, which offers enough flexibility to easily set up completely new tests or execute any already existing testing tools.

Our philosophy is that even automatic assessment using AutoTest should be feedback oriented: AutoTests should always give as much feedback to the student as possible. This is already enforced by linking the AutoTest and Rubric Categories. Another way to maximize feedback to students is by naming your tests and inputs in a meaningful way, as this is the first feedback students will receive from the tests.

Note

Use weights per test (or per input of IO test) to differentiate between the importance of tests.

## IO Test¶

The IO Test is the most common test to create. This is a test in which you specify an input to a program and an expected output. If the students’ program returns the same output as expected, the test passes and they achieve points.

A (student) program can be specified to be run in the IO tests, then, you can specify all input and output combinations you want to check for this program.

Multiple options allow you to more flexibly compare the output of the program:

• Case insensitive: ignore case in comparing the expected output and actual output.

• Ignore trailing whitespace: trim all trailing whitespacing from the start and end.

• Ignore all whitespace: all whitespace is ignored when comparing the student output with the expected output (including newlines).

• Substring: the expected output should be a substring of the actual output.

• Regex: the expected output is a Python3 Regex which should match with the actual output.

Note

The options Ignore all whitespace and regex cannot be activated together.

Note

The option substring is required when using the regex option.

Use the button in the bottom left corner to create a new input/output combination.

### Use cases¶

As the name suggests, IO Tests are used to check the output of a program given an input. IO Tests are very useful to check the functionality of simple and advanced programs. From checking for Hello, World! to checking if a sudoku solver is correctly implemented or even complex mathematical programs. Additionally, IO Tests can be used to check the correct handling of incorrect input.

Note

### Use cases¶

Unit Tests are useful if you want to run many tests but condense their output into a single entry in the test category. They are also useful when you already have a unit testing setup, and do not want to translate them all into individual AutoTest tests.

### Compatibility scripts¶

For most unit testing frameworks it is necessary to install additional software and to figure out how to make the framework output its results to the correct location. We provide wrapper scripts for a growing list of frameworks at our AutoTest Examples GitHub repository that handle all of this for you. Is your preferred testing framework not listed or does a script not work as you need it to? Please send an email to support@codegrade.com so we can discuss the possibilities!