Designing Effective Test Cases & the PM’s Role in Quality Assurance

Designing test cases that are most likely to uncover errors is a critical activity that should start during the Discovery phase—before development even begins. By specifying these tests upfront, you add measurable quality requirements to your project: clear-cut acceptance criteria that define when the software is good enough to ship.

The testing process includes three broad categories:

• White-box tests: Technical tests of the software’s components and architecture, based on internal knowledge of the code.
• Requirements-based tests: Black-box tests that check if the software works as advertised against the documented requirements.
• System tests: Tests that validate whether the software performs its functions “well enough,” including non-functional requirements like performance and security.

Your actual role as a Product Manager in testing depends on the organization. Some companies have dedicated Quality Assurance (QA) teams; in others, PMs are deeply involved in specifying and running tests. Many people with a PM title spend most of their time on testing-related activities.

Most of your testing focus will be on requirements-based testing since it directly relates to the Product Requirements Document (PRD) and user expectations. If your organization lacks dedicated usability testers, you might also help specify or run usability tests. You don’t need to be a technical expert on white-box or system tests, but you should understand enough to ask the right questions and interpret results.

Testing is about finding bugs before customers do

The honest truth about testing is that it is a process of proving that the software does not do what it is supposed to. This negative phrasing is intentional. Every test that finds a bug has proven its worth because the bug will not be released to the end-user.

The term “quality assurance” sometimes suggests a broader scope than just running tests—for example, validating use-case descriptions with stakeholders is also a form of testing.

What exactly is a bug? A bug is any variance between what the system is supposed to do and what it actually does. Some bugs come from programming errors, but many are introduced earlier—through inaccurate, ambiguous, or missing requirements. It is your responsibility as a PM to eliminate as many of these as possible.

The discovery phase is your testing foundation

The Discovery phase consumes most of a PM’s time—it is when you discover and document the system requirements that become the contract between business and developers.

By the end of Initiation, you will have identified system use cases in the PRD and baselined that version to create a stable reference. During Discovery, you review and update these use cases as you gather more information.

Your next step is to investigate and document each use case thoroughly, using templates and UML diagrams as guidance. This documentation becomes a key input for designing test cases.

Deriving test cases from use cases and requirements

Ivar Jacobson, one of the founders of Object-Oriented (OO) methods, advises that you derive test cases from system use cases as follows:

• Test scenarios that cover the basic flow of each use case
• Test scenarios that cover alternate and exception flows
• Tests of line-item requirements in the PRD, traced to use cases
• Tests of features in user documentation, traced to use cases

Use-case scenario testing is a powerful approach to requirements-based testing because the narrative style of use cases closely resembles testing scripts. The organization of use cases into end-to-end business use cases and user-goal system use cases aligns naturally with test organization.

However, use cases alone are not enough. You need more detail to design effective tests, such as:

• Graphical User Interface (GUI) screens
• Structural models providing validation rules
• Documentation of business rules (sometimes stored in business rules engines)

These artifacts complement the use cases but are not part of the use-case documentation itself.

Use-case flows to test scenarios

Your system use cases include flows chosen to cover all important scenarios:

• Basic flow: The normal, expected path through the use case
• Alternate flows: Less common variations or optional paths
• Exception flows: Error conditions or unrecoverable failures

Derive test scenarios from these flows. At minimum, ensure the basic flow and each alternate and exception flow is covered at least once. You may also design tests that combine certain alternate flows to uncover bugs caused by their interaction.

For example, a stock-trading site might have alternate flows for “non-standard lot size” and “partial order fill.” You would want test scenarios that exercise each individually and one that exercises both together to see if the system handles combined conditions correctly.

If you need to cover all combinations of input conditions, use decision tables (covered below) to systematically generate test cases.

During test execution, verify that the sequence of events matches the use-case description. A common approach is to use a test template where:

• Steps beginning with “The user…” go into the Action/Data column
• Steps beginning with “The system…” go into the Expected Result/Response column

Decision tables reveal complex input combinations

When input conditions affecting a use case are interrelated, testing each input separately is insufficient. You must test all combinations of input conditions.

An input condition is any factor that impacts the system’s response. For example, on a web retail site, input conditions might include:

• Item on sale
• Customer discount eligibility
• Fast delivery method selected

Some input conditions appear as alternate flows in use cases; others might be documented in decision tables appended to use cases or requirements.

From a testing perspective, each column in a decision table identifies a test scenario. But the table alone is not a full test script—you must complete test templates for each scenario, specifying the expected sequence of actions.

Decision tables do not specify exact data values—for example, how many Peace Committee members to test for, or what invalid data to use. This is where boundary-value analysis comes in.

Boundary-value analysis targets error-prone inputs

Boundary-value analysis helps you select test data most likely to reveal bugs by focusing on “edges” where system behavior changes.

It covers both:

• Positive tests: Valid input values that should succeed
• Negative tests: Invalid input values that should trigger error handling

Summary of boundary-value analysis rules:

• If an input must lie within a specific range (e.g., 2–10 parties), test the boundaries (2 and 10) and just outside them (1 and 11).
• If multiple valid ranges exist, test the boundaries of each range and just outside the smallest and largest.
• If an input must be one of a set of values, test at least one valid option and one invalid option.
• If different valid options are treated differently by the system, test each valid option separately.
• Combine positive tests in a single run when possible, but never combine invalid tests.

Watch for any boundaries not covered by these rules, such as sorting order, empty inputs, or zero/negative numbers.

White-box testing requires technical expertise

White-box testing involves knowledge of the internal workings of the system—code structure, logic paths, and data flows.

Developers carry out white-box tests, but as a PM, you support this by:

• Specifying the required level of white-box testing before accepting the software
• Reviewing test reports or automated evidence that white-box tests were performed

Limitations make complete white-box testing impossible. For full coverage, all possible execution paths must be tested—a number that grows exponentially with code complexity.

For example, a function with 20 statements repeated 20 times and nested conditionals could require about 10^14 tests for full coverage.

Unit testing sequencing: big bang vs incremental

Software is developed in units—subroutines, functions, or classes. Testing these units and their integration is called unit testing.

Two main approaches to sequencing unit tests:

• Big bang: Test each unit in isolation, then integrate and test all units together. Easy to manage but makes diagnosing integration issues difficult.
• Incremental: Add and test units one by one, testing both individual units and their integration. Easier to diagnose issues.

Incremental testing can be:

• Top-down: Starting from high-level modules toward low-level ones, requiring stubs for missing lower units.
• Bottom-up: Starting from low-level units upward, requiring drivers but no stubs.

In OO environments, units are objects collaborating at the same level, so testing is directed by system use cases and iterations. Each iteration develops and tests only the classes and operations needed for scheduled use cases.

System tests validate non-functional requirements

After black-box tests, system tests verify whether the product meets non-functional or service-level requirements (SLRs), such as:

• Maximum acceptable response time
• Security standards
• Throughput capacity
• Reliability and recovery

Glenford Myers defines system testing as showing that the product is inconsistent with its original objectives.

Common system tests include:

• Regression testing: Ensures new code does not break existing functionality. The level depends on risk and may be governed by a problem review board.
• Volume testing: Checks system behavior with large data volumes.
• Stress testing: Subjects the system to heavy loads in a short time.
• Usability testing: Evaluates user-friendliness and interface design.
• Security testing: Attempts to find vulnerabilities.
• Performance testing: Measures response time, CPU time, throughput, and storage.
• Configuration testing: Validates behavior across hardware/software setups.
• Compatibility/conversion testing: Ensures new software works with or replaces old systems correctly.
• Reliability testing: Checks if reliability objectives are met (e.g., mean time to failure).
• Recovery testing: Tests recovery procedures after failures.

Usability testing questions

• Is the UI appropriate for users’ education level?
• Are system messages clear and corrective?
• Are error messages helpful with “ways out”?
• Are UI elements consistent?
• Are redundant checks on key inputs in place?
• Are system options useful, not cluttering?
• Does the system confirm important actions?
• Does the system workflow support natural business flow?

User acceptance testing (UAT) closes the loop

UAT is the final testing phase before users sign off. It is often performed by users themselves or by the PM with users observing.

Two approaches:

• Formal UAT: Users and developers sign an agreement outlining test terms. Successful completion means acceptance.
• Informal UAT: Users experiment freely without strict procedures. More subjective but can reveal unexpected use cases.

Beta, parallel, and installation testing

• Beta testing: Distributes the product to a wide group of users to uncover remaining errors before production release.
• Parallel testing: Runs new and old systems concurrently to minimize risk.
• Installation testing: Verifies the correctness of the installation process itself.

The PM’s role in testing

The actual job is to support testing by:

• Designing requirements-based test cases early, during Discovery
• Understanding different testing types enough to set expectations and interpret results
• Collaborating with QA and developers on test planning and scheduling
• Ensuring acceptance criteria in the PRD are measurable and testable
• Participating in usability and acceptance testing when needed

Testing is not just a checkbox activity. It is your best opportunity to find errors before customers do. The earlier you start and the more thorough your test design, the higher the quality of your product.

Test yourself: Prioritizing tests on a fintech product

Where to go next

• Learn how to write clear, testable requirements: Writing Effective PRDs
• Master user research techniques to inform test design: User Research Methods
• Understand the product development lifecycle in practice: Agile and Waterfall Fundamentals
• Get hands-on with usability testing: Usability Testing for PMs
• Deepen your knowledge of quality assurance: QA Fundamentals for Product Managers