HowToWriteGoodTests

This is a set of guidelines for anyone that wants to write maintainable automated test cases.

Most common mistakes

  • Write tests thinking about problems, not about your test passing. When your test fails, does it print enough information for anyone to know what is wrong without looking at your code? If the answer is No, you are doing something wrong.

  • Writing code but not asserting any conditions. What makes test code useful is the fact that there are expected results. So just by running some code you are not really testing anything, unless the right checks are in place.
  • Writing test cases with the expectation that they will always pass. Wrong, your test cases will fail, and when they do you want to have the right logging in place to be able to retrace what went wrong without having to spend 3 hours fiddling with code and trying to remember what you were thinking when you added that assert there.

  • Giving up too early in the code. When something fails, your test code needs to be able to continue running and finish cleanly. Just because something fails it does not mean you can just do an exit and forget about the rest. There is nothing more frustrating that knowing there is a problem but not knowing what would have happened if the rest of your test cases had run. Well, you'd be in a better place now.
  • Overcomplicating things. Be concise, write just the necessary amount of code, no more no less. When you see yourself copy and pasting stuff around, just factor it out.
  • Unreadable/Ugly code. Just because you are writing test code doesn't mean you can afford to be sloppy. Test code is as expensive to maintain as production code and badly written/difficult to maintain code leads to a lot of engineering hours wasted. Be considerate with your fellow developers/testers, spend the right amount of time developing your tests so that they don't cause trouble down the line.

Good Practices Checklist

  • Test cases should be independent from each other. If you have many tests that need a particular set up, then do that at the beginning of them all, and clear at the end. But each test case should deal with its own mess without upsetting the others.
  • Log information so that diagnosing problems becomes trivial. Add a comment before every assert or assert block: "Verifying this condition", so that when something fails, you know exactly what it was.
  • When you see yourself copying code over and over again, just factor it out. Make a function. Use it everywhere. This will make changes less painful in the long whenever someone suddenly decides to change the production code at the other end. As a tester, you'll have to modify your code every time a developer decides to change the behaviour of his, so code for it, be prepared.
  • Do not abort or exit from your code randomly when something fails. Assert for the test to fail and continue running. If there are tests down the line depending on that particular condition to be true, then you've done something wrong.
  • Give readable and sound names to classes, test cases, variables, methods, etc, so that your code is easy to follow. If you are iterating through a list of files, call each file file, and the list file_list, do not make up exotic names that you will have forgotten the next time you look at the code.
  • Be consistent. If you use one kind of nomenclature, stick to it. If you are new in the team, find out what people are using and write the same way, do not reinvent the wheel.
  • Format things properly, aid readability. Make your code pretty and tidy, spaces in the right places (i.e. following commas and dots, in between operators). Pay as much attention to your code as to the messages you are printing for the generations to come. Don't write to yourself, be aware that your text is going to be read by other people.
  • Remove trailing spaces from your files, or use an editor that helps you avoid them. It is just ugly to have them there.
  • Send your tests for review, seek for other opinions on how to best approach problems. And whenever asked, send feedback, don't just answer yes like you cannot be bothered helping a colleague make a test better.
  • Finally, be proud of your tests, because you are helping make Ubuntu better.

QATeam/AutomatedTesting/HowToWriteGoodTests (last edited 2013-06-04 18:39:43 by adsl-74-179-115-246)