Positive vs Negative Test Cases: Which is Which?-EN

One of the topics we cover in our Writing Test Cases for Software Requirements class is the concept of positive versus negative test cases. It sounds like a simple concept, right? Interestingly enough, it’s easy to get turned around and confused.

What’s the difference?

Positive test cases ensure that users can perform appropriate actions when using valid data.

Negative test cases are performed to try to “break” the software by performing invalid (or unacceptable) actions, or by using invalid data.

Let’s take a straightforward example. Given the following requirement:

“The system shall allow a new class to be added to the schedule.”

The positive test case would ensure that a new class (with all valid data) can be added to the schedule. Negative test cases would try to “break” the software by testing things like:

• Creating a new class when no rooms are available
• Creating a new class on a date in the past
• Creating a new class with an invalid course title
• Etc

The distinction between positive and negative is pretty clear for this requirement.

What’s the problem then?

It gets hard when requirements are written to describe what the software should do when somebody tries to break it. I term these “negative” requirements. Let’s say you have a requirement that says:

“The system should prevent an unauthorized user from accessing the system.”

This is the point where some people say, “OK, the positive test is trying to have an unauthorized user log in and making sure that the system doesn’t let you do that.”

Actually, no. Trying to log in as an unauthorized user would still be a negative test. Why? Remember our definition of positive and negative test cases:

• Positive tests: Acceptable actions, valid data
• Negative tests: Unacceptable actions, invalid data

Regardless of how the requirement is written, attempting to access the system as an unauthorized user is an unacceptable action. Since it’s an unacceptable action, it’s a negative test.

Is your head hurting? What if I re-wrote the requirement to say:

“The system will only allow access to authorized users.”

Now, does it make sense that trying to log in as an unauthorized user is a negative test? It’s sort of like going back to high school English class when your teacher talked to you about double negatives. Or math class where you had to figure out what to do when you multiplied negatives and positives.

Some ways to think about it

Since most of us have no desire to go back to high school math or English, let’s look at some other ways to get this straight. Some analysis artifacts can be really helpful.  As an example, consider the following user story:

“As an instructor, I want to be able to schedule vacation time so that I don’t get scheduled to teach when I’m not available.”

Acceptance criteria:

• Should only be able to schedule vacation time in the future
• Can only schedule vacation time for myself
• Can’t take more than my annual allotment
• Don’t allow more than two consecutive weeks of vacation to be scheduled

The story is typically written to describe the valid action that the user wants to take, and positive tests can be derived from it. Acceptance criteria often describe invalid actions or data, and negative tests can be derived from them.

Likewise, consider a use case for adding an item to a customer’s shopping cart in a mobile app for a grocery store. The primary (happy) path leads us naturally to a positive test case – successfully adding an item to the cart. The alternate paths often identify negative test cases. This would include things like trying to add an item that’s out of stock or can’t be sold through the app, or exceeding the quantity allowed for a sale item. Also, if you’ve written misuse cases or abuse cases, those also can often be used to identify negative test cases.

Flowcharts can also help. The main flow through a flow chart typically leads us to the positive test case. All the branches from our decision points can be examined to try to identify negative test cases.

If you need more help with flow charts and process symbols, read my The Unofficial Guide to Process Flow Chart Symbols article.

Who cares?

Good question! It’s essential to remember that the most important objective of testing is to find defects. And the most likely way to find defects is by taking unexpected actions or using invalid data. In other words…the most likely way to find defects is by doing negative tests. This means it’s really important to be able to distinguish between the two so that you use your (typically very limited) testing time wisely.

Think positive!

If you’re stumped, remember to think positive! If it’s a good thing, something you should be able to do, it’s generally a positive test. If it’s a bad thing, something you shouldn’t be able to do, it’s generally a negative test.

Happy testing,

Kathy