This tutorial is in “BETA” mode, please shoot me an email if you notice anything wrong with it! You know where to reach me ;)
Test your Python code with pytest
Testing your Python code is a fairly easy task, and this step-by-step guide will get you started on how to use it in your algo homeworks. It will not go in-depth, but will give you everything you need to properly and cleanly test your code.
Before we start
One thing to note: the format used for your submission file is… not
optimal. Because it has a dot in it, it is impossible to import it like any
other module in Python using the good old
from <module> import <stuff>.
Moreover, the dynamic import recommended on Nathalie Bouquet’s website would
make it impossible to use autocompletion in your IDE. I recommend that you keep
the basic name of
login_<homework>.py as given to you and only modify it when
you are ready to submit. That way, you can import everything in your test file
from login_<homework> import *.
Another thing to remember is that YOU MUST NEVER SUBMIT TESTS. Keep your tests in a separate file!
It might sound dumb, but make sure you install Python! We will need to install a
few things through
pip, which is Python’s package manager.
SHARING UNIT TESTS WITH FELLOW STUDENTS MAY BE CONSIDERED TO BE CHEATING AND IS SEVERELY PUNISHED AT EPITA. DO NOT SHARE YOUR TESTS WITH YOUR FRIENDS, AS THAT COULD EARN YOU A MAGNIFISCENT CHEATING FLAG (the famous Flag Triche).
I highly recommend that you use a full IDE for this. If you enjoy using Rider, consider using PyCharm from JetBrains, who are the guys behind Rider. The free Community edition is fine, but you also have access to the Professional version with your EPITA email address1. This guide assumes that you are using PyCharm, but most of what is written here can be done with anything.
Here is a tip that applies to both PyCharm and Rider (and also to any IDE from JetBrains): you can “search anything” by simply double pressing the shift key (right or left), and using the window that appears to type in what you want to search: files, actions, settings… It also shows the various shortcuts associated with each action, which is great for learning how to use JetBrains IDEs very efficiently.
When launching PyCharm, open it directly at the root directory of your homework
(the one in which the
login_homework.py file is), and point it to the
interpreter you have on your system if it asks for it.
If PyCharm complains with PEP8 warnings, know that this means that your code does not comply with the general Python coding style. Respecting the coding style is essential for a language like Python which is based on indentation and not curly brackets. You can find out more about PEP8 here.
Two other essential keyboard shortcuts to know when using the editor:
- Alt+Enter: When you are on an error or a warning, opens a small popup where you can select what you want to do to fix the problem
- Ctrl+Space: Opens up the autocompletion panel, which allows you to quickly fill the line.
Why use Unit Tests?
There are a few very good reasons for using unit tests. For those who do not know, a “unit test” is a small test that usually only tests a single function.
- It makes sure that your code works, which is nice
- It ensures that no regressions happen in your code. A regression is when something that worked is now broken. This is invaluable when you want to efficiently check when and if your code breaks after you’ve made some optimizations.
- It gives you a direct approximation of how much has been done on your task. Writing your tests before writing your code can be an efficient way of making reliable code. Writing tests before actually coding things can even be the way you develop your entire project: this is known as TDD (Test-Driven Development)
- Creating unit tests before even starting to code your function gives you a better understanding of what said function is supposed to do. It basically forces you to look at the example. Trust me, it is very easy to skip an example when looking through the general specifications.
- Click a button, test everything at once. It’s that simple. Automate all the things!
- Testing your code is not that different from printing out the results and checking them manually. Here, instead of printing the results, you directly compare them with what you expected.
Side note: Making good unit tests
A unit test must be:
- Small: a true unit test must only test one unit of code at most2. A unit
of code can be
- A specific use case
- A specific edge-case that your code should handle.
- Repeatable: it can be ran multiple times.
- Stateless: it must be able to run no matter what the configurations and environment are3
You must think of your unit-tests not as specific tests of what your code does
but of what your code should do. Do note that there are also multiple valid
return values for certain functions: you will have to have more complex
assertions than just simple equality tests (
result == expected) in these
Why pytest specifically?
Algo homeworks generally do not require huge amounts of testing or complex
interactions between tests.
pytest is extremely simple and
straight-to-the-point, and does not require any sort of configuration in order
to be used, which is perfect for us.
pytest can also be extended with plugins,
so, if something is missing in the basic
pytest, a plugin probably exists.
Another massive advantage of
pytest is that it only uses thing you already
know, except for the
assert keyword which will be explained later on.
Step 0: Installation
Install Python if you have not done so already. For this
guide, you will need to install
pip, Python’s package
manager. Use the following in your favorite terminal, console or whatever:
pip install pytest
Run this and you should be good to go. You might need to use
sudo to install
everything on UNIX systems (Linux, Mac OS X), but I am not entirely sure.
Why not use a requirements.txt?
If you are asking this question, (and know how to deal with this kind of issue)
feel free to use a
requirements.txt file and configure PyCharm accordingly.
I just felt like it was not necessary for this specific
pytest is good to have for any python file in my
opinion. This is a guide for beginners for whom
are not really important.
Step 1: My first unit test
Creating a unit test is extremely simple. Say I have a function called “add” in my main file “myfile.py”, which takes two arguments and returns their sum. I could simply write a few unit tests for it like so in a file named “test_myfile.py”:
# The following line imports all of the function from myfile.py from myfile import * def test_add_positive(): assert add(5, 6) == 11 assert add(10, 99) == 109 assert add(0, 1) == 1 assert add(1, 0) == 1 def test_add_negative(): assert add(-5, -10) == -15 assert add(-99, -100) == -199 assert add(0, -1) == -1 assert add(-1, 0) == -1 def test_add_mixed(): assert add(1, -1) == 0 assert add(-1, 1) == 0 assert add(-15, 10) == -5 assert add(10, -5) == 5
This is a regular Python file, meaning you can write intermediate functions or
do more crazy Python things. Test functions are identified by their
Nothing here is really new for you, apart from the
assert keyword. In Python,
assert keyword does nothing if the condition you give it is true, and
crashes if the condition is false. It is a way of saying “I’m sure that this
condition is true, so make sure that it indeed is”. This is basically the entire
point of unit tests, and
pytest smartly extracts information from the
resulting assert error to show exactly what went wrong.
Each function will show up as a separate test. You can see that each assert
statement corresponds to a separate test of the
add function. And no, you do
not have to be as thorough as I was there, but you really should, because that’s
the whole point of unit-testing.
You may notice that, although we have no information on how the function was implemented, we can still create a lot of test cases, including trickier patterns like adding two negative numbers, mixing and matching negative and positive numbers, or even cases where we should get 0. If you are ever lost while writing test-cases, always think about what could be some nasty case that I wouldn’t want to have to deal with. Talk with fellow students about which cases they think are complicated, write tests for these cases, and check if your code is able to handle these test cases properly. Get creative!
Step 1.1: Using examples as unit tests
The subject will often feature examples of the result a function should give. Say you have the following in your subject:
>>> my_amazing_function("Abcde", 123, BinTree("all your bintrees are belong to us", None, None)) [("kb.zoroark.guru", 50), ("is alright", 25)]
We can directly re-use them as unit tests!
def test_amazing_function(): result = my_amazing_function("Abcde", 123, BinTree("all your bintrees are belong to us", None, None)) assert result == [("kb.zoroark.guru", 50), ("is alright", 25)]
It is a good idea to separate the function call from the comparison to the
expected result. This way, if your code fails miserably with an exception (that
is, it actually fully crashes because of the function you’re testing instead of
just failing the assertion), you know that it is caused by the function and
== issues, because it will indicate the line with the call to
my_amazing_function. This will also help you isolate cases where something was
None somewhere. Failed on the
assert? Might be a
None problem. Failed on
the function call? Something went wrong inside your code.
Step 2: It’s time to test!
You have a few options here, but it mostly comes down to whether you are using PyCharm or not.
Step 2A: Using PyCharm
You will only have to do step 2A.1 once to tell PyCharm how to test your code.
Step 2A.1: Creating a configuration
Click the “Add Configuration” button in the top-right corner (or the existing configuration’s name and then Edit Configurations). Click the + in the top left corner, pick “Python Tests” and then “pytest”. Give it a nice name, and put the name of your test file in the text field right below “Target”. When you are done, click OK, and you can now run the test suite.
Step 2A.2: Running the tests
You can now run your test suite like any other run configuration, and you should see something like this pop up.
On the left, you can see the file that was just tested (
with the output from
pytest on the right. Clicking on each individual test
will display what was printed during the test, and details about the error (if
any). If the test passed successfully and nothing in particular was printed, you
will only see a dot.
Make sure that the “Show passed test” box is enabled (the tick on the top left of the results panel).
What if one of our tests fails?
Alright, I clearly don’t know how to create a simple
add function there.
Clicking on each failing test will give you detailed information on what failed
in the assert statement.
pytest even presses F for you, how nice!
If the assert did not fail but an exception happened,
pytest will also give
you full insight on which function failed. Of course, a malformed test file will
crash the entire testing sequence immediately and you will only get an error
message. If the original file you are testing has a syntax error somewhere
inside of it, that will also crash the testing sequence immediately.
A few tips: right-clicking on a test allows you to relaunch it separately, and you have a button on the left of the tests panel to relaunch only failed tests.
Step 2B: Using any shell
I am using PowerShell in the following example, but you can use anything.
pytest is done with the following command:
The results are fairly similar to what can be seen in PyCharm, so I am just leaving the screenshots as examples.
You can also just run the following command, and
pytest will figure out where
the tests are on its own:
… yup, that was pretty simple to type. And now that I have recovered my
missing braincells and fixed my function,
pytest shows us that, indeed, the
unit tests work, hurray!
Congrats! You now know how to write unit tests for your homework project.
Need more out of your unit tests? I suggest you create very long ones which
would be used to benchmark your algorithm. Use them in conjunction with
to get detailed profiling results to determine which functions get the most
calls. If enough people are interested, I might make a more detailed tutorial,
but it all boils down to these few lines:
# at the beginning of the file import cProfile # in one of your benchmarking unit tests prof = cProfile.Profile() prof.enable() # the code you want to profile prof.disable() prof.print_stats("calls")
You will get a very detailed output with the number of calls for every function
used between the
disable calls, as well as the time spent in each
function. This should be your starting point for optimizing both the number
of function calls and the time spent in each function. Good luck!
One particularly convenient feature of the paid version of PyCharm is code coverage right inside the IDE, but you can use coverage tools without that. Overall though, PyCharm is quite nice because it gives you a cozy environment where you don’t have to worry too much about configuring your stuff. ↩
This is the best case scenario: you can of course put multiple test cases (i.e. multiple
asserts) in the same unit test for peak laziness. Don’t worry, I’ve been there too ;) ↩
That’s not what “stateless” really means but that’s all you need to remember for now: unit tests are isolated from one another. ↩