Redesigning an AI-powered test generation platform from confusing forms to a fast, intuitive workflow.

Role

Product Designer

Timeline

2 Months

Platform

SaaS Web Application

Deliverables

Research, User Flows, High-fidelity screens

Domain

QA Automation /AI Testing

0%Faster Time to First Test- measured against the original 15-minute onboarding baseline.

0%Reduction in UX Support Tickets, - projected, based on friction points removed.

+0ptsonboarding completion - 35% to 75%, matching the wizard redesign

What is Orange Pro ?

OrangePro is an AI-powered QA platform that helps engineering and QA teams generate comprehensive test cases from user stories and automatically analyze production incidents to prevent recurrence.

What is Orange Pro ?

OrangePro is an AI-powered QA platform that helps engineering and QA teams generate comprehensive test cases from user stories and automatically analyze production incidents to prevent recurrence.

Product Type

AI-Powered QA Testing Platform for SaaS, E-commerce & Logistics companies

Core Features

Requirement-Based Test Generation & Incident-Based Defect Analysis

Target Users

QA Engineers, DevOps Teams, Product Managers at growth-stage companies

Tech Stack Context

Integrates with Jira, TestRail, GitHub, and Confluence for seamless workflows

The Core Issues

OrangePro's AI could generate 48 comprehensive test cases in seconds,but the product interface was so friction-heavy that users couldn't get there. The onboarding presented blank text boxes with no guidance, separate navigation paths created confusion, and the results screen hid 90% of generated tests behind dropdowns and pagination. The AI's value was being completely buried by poor UX.

The Core Issues

OrangePro's AI could generate 48 comprehensive test cases in seconds,but the product interface was so friction-heavy that users couldn't get there. The onboarding presented blank text boxes with no guidance, separate navigation paths created confusion, and the results screen hid 90% of generated tests behind dropdowns and pagination. The AI's value was being completely buried by poor UX.

Heuristic UX Audit (The Roast)

Conducted a thorough heuristic evaluation of both core product screens the Requirement Test Generator and the Defect Analysis screen. Applied Nielsen's 10 usability heuristics and documented critical UX failures across the two screens. Prioritized findings by severity: critical blockers vs. moderate friction vs. minor polish issues.

Heuristic UX Audit (The Roast)

Conducted a thorough heuristic evaluation of both core product screens the Requirement Test Generator and the Defect Analysis screen. Applied Nielsen's 10 usability heuristics and documented critical UX failures across the two screens. Prioritized findings by severity: critical blockers vs. moderate friction vs. minor polish issues.

P0 - Critical Blocker

Mandatory Application

It Requires a full application description before generating any tests a significant friction point in the critical first-use moment.

P0 - Critical Blocker

Zero Quick Start

No sample data buttonNo templates No pre-filled examples. High barrier for new users who needed immediate value.

P0 - Critical Blocker

Zero Quick Start

No sample data buttonNo templates No pre-filled examples. High barrier for new users who needed immediate value.

P1 - Moderate Friction

No Auto-Fetch Intelligence

The system made users do manual work. Auto-fetch wasn't being utilized where it could eliminate friction entirely.

P1 - Moderate Friction

No Auto-Fetch Intelligence

The system made users do manual work. Auto-fetch wasn't being utilized where it could eliminate friction entirely.

P0 - Critical Blocker

Hidden Value Proposition

The AI generated 48 test cases, but rigid pagination and a theme dropdown hid the full results from users.

P0 - Critical Blocker

Hidden Value Proposition

The AI generated 48 test cases, but rigid pagination and a theme dropdown hid the full results from users.

P1 - Moderate Friction

URL Copy-Paste Hell

Required manually hunting down and pasting 3 separate URLs (Jira, TestRail, Confluence) tedious busywork despite having integrations available.

P2 - Minor Polish

Confusing Tabs

Manual vs Jira toggled as separate tabs unclear what the functional difference was or whether both could be used together.

P2 - Minor Polish

Confusing Tabs

Manual vs Jira toggled as separate tabs unclear what the functional difference was or whether both could be used together.

Competitive Analysis

Benchmarked OrangePro against 6 competing or analogous tools in the QA and testing space: Testsigma, Katalon, Autify, Mabl, BrowserStack, and Postman. For each, identified the single UX pattern they executed best and extracted the underlying design principle for OrangePro to borrow.

Competitive Analysis

Benchmarked OrangePro against 6 competing or analogous tools in the QA and testing space: Testsigma, Katalon, Autify, Mabl, BrowserStack, and Postman. For each, identified the single UX pattern they executed best and extracted the underlying design principle for OrangePro to borrow.

Understanding the Landscape

Before designing solutions, I conducted a thorough competitive analysis of leading QA and testing tools to identify industry-proven UX patterns.

Understanding the Landscape

Before designing solutions, I conducted a thorough competitive analysis of leading QA and testing tools to identify industry-proven UX patterns.

Competitor

Key Feature Studied

What They Do Right

Key Pattern Borrowed

Testsigma

Natural Language Test Generation

Plain-language prompts instantly create test cases near-zero learning curve

Low-friction Entry

Katalon Studio

Record & Playback Onboarding

Users can observe, import or replay tests visually before writing anything manually

Visual learning Path

Autify Nexus

Quick Start Templates

Pre-built scenario workflows for common usecases users get value on minute one

Immediate value

Mabl

Auto-Import from Jira

One-click sync that eliminates all manual URL pasting integrations actually used

Reduced Friction

BrowserStack

Try Demo Before Signup

Sample app + real data lets users experience output before any commitment

Risk-free trial

User Flow Architecture

Designed comprehensive user flow diagrams covering two major journeys: the onboarding flow (first-time vs. returning users, 3-step wizard) and the full test generation flow. Each flow documented decision points, happy paths, edge cases, error states, and recovery mechanisms. Defined 3 input paths for test generation: Quick Start, Template Library, and Jira/GitHub import.

01. Entry & Auth

Landing page to Social Signup/Login (Google/GitHub). Existing users skip straight to the dashboard.

02. Onboarding

New users get a guided tool connection wizard (Jira, Testrail, Confluence) with a skip option. Returning users go straight to the dashboard.

03. Path A (Templates)

Select from 3 industry templates (SaaS, E-Commerce, Logistics) to generate a test suite in under 2 minutes.

04. Path B (File Upload)

Upload a PRD, user story, or spec file. AI extracts context and populates fields for user review and generation.

05. Path C (Jira Import)

Search and select Jira tickets or import Confluence pages to auto-populate requirements.

06. Defect Analysis

Auto-fetches or manually inputs recent incidents to analyze and generate targeted regression tests with insights.

07. Preview & Refine

Review test cases in preview mode. Inline edit steps, add/remove/regenerate cases, and track changes.

08. Export

Edit, save draft, or export tests (CSV, Excel, Testrail, Confluence). Primary CTAs are clearly emphasized.

Wireframing (Both Screens)

Designed comprehensive user flow diagrams covering two major journeys: the onboarding flow (first-time vs. returning users, 3-step wizard) and the full test generation flow. Each flow documented decision points, happy paths, edge cases, error states, and recovery mechanisms. Defined 3 input paths for test generation: Quick Start, Template Library, and Jira/GitHub import.

Design Decisions

The Hard Calls

Quick Start as the Default Entry Point

Should users be asked to fill in requirements first, or should the path to value be Quick Start with sample data?

Option A: Form-First

Present the input form immediately. Power users who know what they're doing can start right away.

Quick Start Prominent - Chosen

Surface "Try with Sample Data" Full form available but secondary foldable by default for new users.

Why this decision: Competitive research revealed that every top-performing tool in this space defaults to a demo-first experience. Users need to see output before investing effort in input. The Quick Start reduces the emotional risk of the first session. Power users can ignore it but new users won't get stuck at a blank canvas.

Single Screen with Toggle- Reconsidered

Should Requirements and Incidents live as Separate Pages, Or merge into one screen with a Toggle ?

Option A: Toggle on single screen

One entry point, a visible mode switch. Reduces nav items, surfaces both capabilities on arrival.

Option B: Separate pages -Chosen

Requirements and Incidents kept as distinct sidebar items.

Why this decision: The two flows solve different jobs at different moments Requirements is proactive (planning ahead of a sprint), Incidents is reactive (responding to something already broken).A toggle optimizes for discoverability neither persona needed; separate entries optimize for the fast, repeat navigation both personas actually do. Kept the sidebar grouped under one "Generate Tests" parent so the relationship between the two modes is still visible.

Results as Table View (Not Accordion)

The current design collapsed test cases inside a theme dropdown. How should 48 generated tests actually be displayed?

Option A: Grouped Accordion (Current)

Tests hidden inside expandable theme sections. Keeps screen clean but buries the AI's output, users can't scan all tests quickly.

Table View Default Chosen

Compact table showing all tests at once with theme pills as filters. Card view toggle available for detail inspection.

Why this decision: The AI generated 48 tests but the dropdown + pagination combo made ~42 of them invisible. This actively undermined the product's core value proposition. A compact table view shows everything at once, allows bulk export, and lets users appreciate the breadth of AI output. Card view is preserved as a toggle for users who want detail-focused inspection.

Why this decision: The AI generated 48 tests but the dropdown + pagination combo made ~42 of them invisible.

This actively undermined the product’s core value proposition. A compact table view shows everything at once,

allows bulk export, and lets users appreciate the breadth of AI output. Card view is preserved as a toggle for users

who want detail-focused inspection.

What Was Redesigned

Progressive Onboarding Wizard

Replaced the cold-start experience with a 3-step onboarding wizard for first-time users. Each step has asingle clear action with a skip option, eliminating the blank canvas paralysis that was killing activation.

Test Details & Step Builder

Test Objective and Test Steps moved to their own stage. Steps are drag reorderable with clear add/remove actions. Renamed to avoid colliding with the wizard's own step labels.

Requirements & Incidents as Separate Entries

Requirements is proactive planning. Incidents is reactive response. Each gets its own sidebar entry under "Generate Tests," so users go straight to the mode they need. The Manual/Jira toggle stays, but only for picking a story source, not for switching workflows.

Results as Table, Not Accordion

The old screen hid 42 of 48 tests behind a dropdown. The table now shows all tests by default, theme pills as filters, ratings and status at a glance, bulk export via checkboxes. Card view stays as an optional toggle.

Key metrics

0%Faster Time to First Test- measured against the original 15-minute onboarding baseline.

0%Reduction in UX Support Tickets, - projected, based on friction points removed.

+0ptsonboarding completion - 35% to 75%, matching the wizard redesign

0%Faster Time to First Test- measured against the original 15-minute onboarding baseline.

0%Reduction in UX Support Tickets, - projected, based on friction points removed.

+0ptsonboarding completion - 35% to 75%, matching the wizard redesign

What I Learned

Time to Value is Everything

In a B2B SaaS product, the most critical metric isn't retention, it's time to first value. Every design decision was evaluated through the lens of "does this make the AI's output visible faster?" Everything else was secondary.

Competitors - Free Research

Six hours of competitive analysis yielded patterns that would have taken weeks of user testing to surface

independently. Testsigma's NLP approach and Postman's mode toggle were direct inspirations that saved significant design iteration cycles.

Results Are Underrated

The test results screen was the most impactful redesign not because it was the flashiest, but because it was where the AI's value actually landed. Hiding 90% of output behind a dropdown was actively destroying product credibility.

Error States Are Features

Designing the 6 edge cases and error states wasn't an afterthought it revealed that most of OrangePro's UX problems were fundamentally about what happened when things went wrong. Graceful degradation is a core product experience.

The biggest insight: OrangePro didn't need a better AI,It needed a better front door. The technology was sound. The experience of getting to the technology was the product.