Test Cases are dead, long live Checklists!*

By: Ömürden Cengiz @hipolabs

For testers, documenting our tests is critical for a healthy testing process, so it pains me to see how boring writing test cases can be. We’ve recently changed our approach to test documentation, and so I’ll be talking about an alternative method that shortens the time it takes to write our tests down.

In classical approaches, test cases are overcrowded with words; mostly of information about the project that everyone already knows. What test cases don’t have, is filtered information that allows readers to focus on the core functionality. To that effect, I believe Checklists are poised to take the throne from Test Cases, especially among startups and client-servicing development shops (where projects move much faster and with higher turnover than the rest of the industry). So let me make my statement:

Test cases are dead, long live checklists!*

*: only for suitable teams and projects

Wait, what’s that fine print?

This is an obvious disclaimer, but I have to cover all use cases (ahem). The checklist approach is not suitable for all kinds of teams and projects. First of all, teams that use checklists should be small (1–3 people). Larger teams will be better served by detailed test cases that cover all user actions of test scenarios. Since it’s very hard for everybody in those teams to be on the same page at all times, the extra information that test cases provide would be beneficial.

It is also ideal for everybody in the team to know every part of the project (not just a section) when using checklists. Larger, more comprehensive apps are very hard to track. If testers are only responsible for a fraction of the app, the extra documentation of test cases would help with crossovers and handoffs.

Lastly, teams employing checklists should be in a very collaborative environment, live editing , changing, and sharing their checklists, for things to stay updated. Test teams who are not spending most of their work week in the same room might prefer test cases to get rid of ambiguity that can be caused by miscommunication.

So, why use Checklists?

Simple: Creating test cases is extremely time-consuming. Much like code, its requirements can change with every sprint, rendering your test cases obsolete, and warranting an update. All of this takes more time than testers usually have in agile development structures.

Plus, customers care about the app, not its test documentation. All testers have to do, in essence, is to just prove that the app works the way it should. With processes like agile development and constantly evolving projects, testers might need a more concise way to make sure things work.

What are checklists, Exactly?

Checklists are simplified versions of test cases, which aim to cover all user actions that are possible within a given functionality. Checklists provide a shorthand way of writing the functionality, putting the core action at the top. It’s thus much more easy to write a checklist item than a test case.

Here are two test cases written in a traditional way:

1. Login with existing user

Steps :

• Open application.
• Type username of the existing user
• Type password of the existing user
• Tap “Login”

Expected results:

• User sees the login success message
• User sees the items on feed

2. Register as new user

Steps :

• Open application.
• Type username of the new user
• Type password of the new user
• Type password again
• Tap “Register”

Expected results:

• User sees registration success message
• User sees the welcome animation
• User sees the items on feed

Developers, testers (and even clients) know these features intuitively. While this process standardises the format of all cases, it definitely does not make them easier to read.

The checklist approach claims that we just need to document what to test and the expected result of the test. Nothing else. Here’s how you can write this as a checklist

Login/Register Flow

• Login with existing user — no welcome animation, just feed
• Register as new user — welcome animation, then feed.

Done

Editing checklist items by testers or developers (or even clients) is easier than test cases if the feature is changed. Items in the checklist are much more user friendly, understandable and editable. Also, one of the most important advantages of checklists is that it’s easy to integrate with test automation frameworks like Robot that have a keyword-driven approach

What we do

At Hipo, we have a small QA team, and projects move pretty fast, making test case writing redundant. We’ve reverted to checklists, which take around 50% of the time that full test cases would take.

Checklist can be created anywhere, be it on a notepad, Excel, or an online tool. Some online tools out there make a tester’s life a lot easier by providing some custom elements that help with tracking and reporting.

We’re using a great online tool called Testpad. Here is what we like about it:

• Creating and editing checklists are easy. You feel like you’re working on a regular spreadsheet.
• You can create test cycles and fill in custom details about each cycle. This is not mandatory, but a great tool for tracking your test cycles over multiple projects.
• Checklist items can be marked passed/failed/blocked like test cases.
• Failed items can store links to the ticketing system leading developers directly to the ticket or task related to solving the issue. We link failed items with the Codebase environment, and check them again after tickets are solved.
• It’s really easy to share test cycles with team members and external users with a link.

Why I wrote this

I come across a lot of small development teams that feel intimidated by the “Real QA” standards of testing. The standardisation of the process is great for one-project teams that stay with the project for a long time (as has always been the case until recently). However the new, smaller, client-servicing development teams who have to juggle multiple projects with a relatively fast turnover rate, are almost never comfortable adhering to those standards. I wanted to share my experiences trying out new things, and I hope this helps some people understand that QA, too, needs to adapt to its environment.