How a SaaS Team Added AI-Assisted Testing Without Replacing Existing Automation

A B2B SaaS company operating a multi-tenant platform relied on frequent deployments to deliver new customer workflows, permissions, billing features, and administrative capabilities.

Its engineering teams already had a mature CI/CD process supported by Selenium and Playwright automation.

The challenge was keeping release-critical test coverage aligned with the speed of development. New Jira stories and acceptance criteria often reached staging before QA had finished preparing the corresponding smoke tests.

To close that gap, the company introduced CoTester into its release workflow. Here’s how the rollout worked and what changed for the QA team.

The Challenge: Release Complexity Was Outgrowing the Smoke Suite

The SaaS platform consisted of more than 40 independently deployed services, with several staging deployments taking place each day.

Release validation had to account for more than whether a feature worked in its primary path. Tenant configuration, account state, user permissions, billing rules, and feature dependencies could all change the expected result.

A billing update, for example, might behave differently for an administrator, a read-only user, or an account already carrying a pending plan change. Similar variations existed across provisioning, authentication, and tenant-management workflows.

QA engineers had to interpret those conditions from Jira stories and acceptance criteria, convert them into test scenarios, define the necessary assertions, and prepare the automation before a change could receive a release-ready signal.

That preparation became the bottleneck. The automation estate itself was not the problem. The team needed a faster way to move newly approved requirements into reviewable tests while preserving its release controls.

Also Read: Selenium Tutorial: Complete Automation Testing Guide

The CoTester Approach: From Jira Requirement to Approved Release Test

The rollout began with release-critical workflows covering authentication, account administration, user permissions, billing configuration, and tenant settings.

Rather than applying CoTester across the full regression estate immediately, the team used it where requirements changed frequently and new smoke coverage had to be prepared quickly.

Generate the first test from product context

Approved Jira stories and acceptance criteria provided the starting context.

CoTester converted that information into executable test scenarios with ordered steps and expected outcomes. This gave QA a working test to review instead of requiring an engineer to construct every path manually.

For scenarios where additional application context was required, the team could also use the relevant staged web flow to guide test creation.

Review the workflow before execution

QA engineers reviewed generated scenarios in CoTester’s step-by-step editor. They could add or remove steps, adjust validations, and extend a generated happy path with role-based, negative, or edge-case conditions.

This was particularly important for the company’s multi-tenant application, where the same feature could produce different outcomes depending on permissions, account configuration, or billing state. Only reviewed scenarios were added to the release gate.

Trigger approved tests through CI/CD

After a successful staging deployment, a custom pipeline hook triggered the selected TestGrid tests. The CoTester scenarios ran alongside the company’s Selenium and Playwright jobs rather than requiring those suites to be converted or migrated.

Engineering continued to own the promotion decision. TestGrid supplied the execution result and supporting evidence; the release team decided how that result affected the deployment.

Use execution evidence for review

Each run produced step-level results, screenshots, and execution logs. When a scenario failed, QA could inspect where the workflow diverged from the expected result and pass that evidence to engineering without manually reconstructing the complete flow.

Also Read: Generate Test Cases from JIRA Stories with CoTester

How the Workflow Played Out on a Billing Change

In the fourth week after rollout, a Jira story reached staging that allowed tenant administrators to switch a workspace from monthly to annual billing.

The acceptance criteria specified that only users with billing-administration privileges could make the change. The workflow also had to update the selected plan and display the revised invoice preview before confirmation.

CoTester generated the initial scenario covering the administrator path, plan selection, confirmation step, and updated invoice preview. During review, a QA engineer added a negative scenario for a user with read-only billing access.

That test exposed a permission defect on its first execution.

The read-only user could open the plan-change screen even though the acceptance criteria restricted the action to administrators. The run captured the failed step, screenshot, and execution logs, giving the development team the context required to trace the incorrect permission check.

The defect was fixed and the updated workflow cleared the release gate the same day. Under the previous process, comparable negative scenarios often entered the smoke suite one or two releases after the feature they were intended to validate.

Also Read: CoTester 2.0 – The First True Enterprise-Grade AI Agent for Software Testing

The Impact: QA Moved Closer to the Pace of Development

The company compared the eight weeks before the CoTester rollout with the eight weeks that followed. Median release-gate feedback time fell from 48 minutes to 19 minutes.

Faster preparation of new tests meant more of the required smoke coverage was ready when a deployment reached staging. Average effort to prepare a new release-critical test fell from roughly 2.3 hours to 30 minutes, including QA review.

Engineers spent less time assembling routine steps and more time checking business rules, permissions, negative paths, and expected outcomes.

The approved smoke suite also grew from 32 to 51 scenarios. The added coverage focused on new and changing workflows while the company continued to use its established Selenium and Playwright automation for mature application areas.

The practical change was visible before execution began: QA entered the release gate with more of the required coverage already prepared and reviewed.

“Before the rollout, we could have a feature ready in staging while QA was still writing the release-critical tests around it. The biggest change is that our engineers now review coverage much earlier. By the time a change reaches the release gate, we are validating the edge cases instead of still creating the basic smoke test.” – Director of Quality Engineering, B2B SaaS Company

Close the Gap Between Requirements and Release-Ready Tests

If new requirements are reaching staging faster than your QA team can prepare the corresponding tests, CoTester can help you move from approved product context to reviewed, executable coverage earlier in the release cycle.

Use CoTester to generate and refine tests from Jira stories, requirements, and application context, then run approved scenarios within your existing delivery process. Book a CoTester demo today.