2025

How Your Team Can Achieve Sustainable Test Growth: Balancing Speed, Cost, and Quality in the AI Era

May 7, 2025 by Aishling Warde

The promise of AI-driven development is undeniable – faster code, quicker releases, and unprecedented innovation. But here’s the catch: AI isn’t perfect, and the code it generates could be riddled with hidden flaws. In fact, within three years, over a third of all code will be AI-generated, and much of it may introduce more bugs into production than ever before.

Digital transformation isn’t just a buzzword anymore – it’s a $3.9 trillion race to stay competitive, with 85% of organizations adopting cloud-first strategies. As release cycles accelerate and budgets tighten, how do you ensure quality doesn’t fall by the wayside?

For years, the rule of thumb has been “pick two – speed, cost, or quality.” Now, that luxury is gone. In this blog, we’ll dive into the growing pressure to balance all three, and why outdated testing processes could make or break your transformation efforts.

Testing Bottlenecks in the Era of Digital Transformation

Despite advancements in test automation, testing remains one of the biggest bottlenecks to digital transformation. Surprisingly, 80% of tests are still conducted manually across the industry. While automation promises greater efficiency, many test automation projects are started but never completed, and their ROI often falls short.

We recently surveyed SmartBear customers who do not use automation tools. The three most common barriers to automation adoption were:

Lack of Time – Teams prioritize releasing the next version, leaving little time to develop automated tests. Automation efforts consistently lag, typically falling two sprints behind development.
Lack of Expertise – Automation tools often require technical skills that teams may not possess. Record-and-playback solutions have failed to meet expectations, leading many teams to abandon automation altogether.
Tool Overload – With hundreds of automation tools available, selecting the right one is overwhelming. Many teams revert to manual testing simply because it’s easier than navigating the complex tool landscape.

These challenges create friction and prevent teams from scaling their testing processes, slowing down release cycles and increasing the risk of bugs in production.

The High Cost of Delayed Bug Detection

The cost of bugs discovered in production far exceeds those caught earlier in development. A striking example is the recent CrowdStrike issue, which resulted in $5.4 billion in losses due to widespread system failures. The actual fix took only an hour and a half, but the repercussions were far-reaching.

On a broader scale, the numbers are staggering. Each year, 100 billion lines of code are added to software systems, with an estimated 25 bugs per thousand lines. This results in roughly 2.5 billion bugs leaking into production annually. The cost to fix these issues post-release is exponentially higher than addressing them during development.

Strategies for Sustainable Test Growth

To address these challenges, organizations must adopt a sustainable approach to testing – one that pushes defect detection earlier in the process (shift left) while improving monitoring and feedback in production environments (shift right).

Shift Left – Catching Bugs Early

The earlier a bug is found, the cheaper it is to fix. Shift left practices encourage testing earlier in the development lifecycle, reducing the risk of costly production issues. However, developers cannot be expected to take on all testing responsibilities. While developers are doing more testing than ever, end-to-end and UI testing require specialized skills. Overburdening developers with testing tasks detracts from their primary focus – writing application code.

Shift Right – Monitoring Production for Faster Feedback

By extending testing into production, teams can monitor for errors, track performance, and gather valuable insights to refine pre-production testing. Effective shift-right strategies rely on robust production monitoring systems that capture issues in real time and relay information back to development teams. This feedback loop ensures continuous improvement, reducing the cost and complexity of addressing bugs discovered in the field.

Tying It All Together

Combining these strategies creates a continuous quality loop that not only reduces the number of bugs slipping into production but also significantly lowers the cost of fixing them. By catching defects earlier and refining tests through production insights, businesses can avoid the ballooning costs associated with late-stage bug fixes. This holistic approach improves release velocity, enhances software reliability, and ultimately delivers a higher return on investment (ROI) by preventing revenue loss caused by critical failures.

Sustainable test growth isn’t just about preventing issues – it’s about driving long-term savings and maximizing the value of every development hour spent.

The SmartBear Approach to Testing

At SmartBear, we understand the delicate balance between speed, cost, and quality. Our holistic testing strategy focuses on continuous quality at every stage of development. By leveraging SmartBear API Hub, Test Hub, and Insight Hub, teams gain end-to-end visibility across the software development lifecycle, ensuring they can build, test, and release with confidence.

The Test Hub allows teams to manage, automate, and execute a variety of tests – from functional and UI tests to API and load tests – all within a single platform. This centralized approach streamlines workflows and reduces the overhead associated with managing multiple testing tools.

AI-Powered Enhancements for Modern Testing

SmartBear’s roadmap is filled with AI-driven features designed to accelerate test growth and simplify automation. Some of the latest innovations include:

Natural Language-Based UI Test Automation – Convert manual tests into automated scripts for web and mobile apps using simple natural language prompts, reducing the need for technical expertise.
Test Case Generation from Requirements – Instantly generate manual test cases directly from user stories and requirements, speeding up test creation and ensuring coverage aligns with business needs.
Test Data Generation – Create synthetic test data on demand through contextual prompts, eliminating the delays associated with test environment setup.
Visual Testing – Detect visual defects across web applications at scale, ensuring consistent performance across browsers and devices.
Contract Test Generation – Produce contract tests directly from OpenAPI specs, client code, or HTTP request/response pairs, ensuring robust API coverage.

By embedding AI throughout the testing process, SmartBear empowers teams to automate faster, identify defects earlier, and minimize production risks without overburdening development teams. These AI-driven capabilities are already delivering tangible results for organizations:

“Previously, locator-based plug-ins required painful updates as programs evolved. Zephyr Scale’s AI automation eliminates that issue, interpreting commands like ‘click on magnifying glass,’ cutting regression time from 90 to 20 minutes, improving consistency, increasing coverage, and saving time and money.” — Test Analyst at a Leading Automotive Services Provider
“Adopting no-code automation cut our manual regression time by about 60%, allowing QA to focus on complex scenarios. Non-technical team members now create tests aligned with business goals, increasing coverage, enhancing collaboration, reducing post-release defects, and fostering greater ownership.” — Quality Assurance Analyst at a Global Software Company

Future-Proofing Software Quality in the AI Era

As AI continues to reshape the software development landscape, organizations stand at a critical crossroads. The potential for faster development is undeniable, but without the right testing strategies in place, the influx of AI-generated code could unravel hard-won gains. Sustainable test growth isn’t just a technical goal – it’s a business necessity for navigating the complexities of digital transformation.

Shifting left to catch bugs early, embedding robust production monitoring, and integrating AI-driven automation can help businesses break free from the outdated “pick two” mentality. The organizations that succeed in balancing speed, cost, and quality will lead the next wave of innovation. Those that don’t risk falling behind grappling with costly production bugs, delayed releases, and customer dissatisfaction.

SmartBear Hubs provide the framework to streamline testing across the entire development lifecycle, enabling teams to release with confidence, minimize risk, and scale at the pace digital transformation demands. But the time to act is now.

If you’re ready to stop firefighting production issues and start building a proactive, AI-empowered testing strategy, SmartBear can help. Get in touch today and discover how our end-to-end solutions can future-proof your development pipeline and deliver sustainable test growth.

Author

Prashant Mohan

Prashant Mohan is a VP of Product Management at SmartBear. He is responsible for driving the vision and strategy of products that help developers and testers deliver quality applications at scale. Prashant is an engineer with a business degree, and has worked across several industries including B2B tech, Fintech and HealthIT.

SmartBear were Gold Sponsors in EuroSTAR 2025. Join us at EuroSTAR Conference in Oslo 15-18 June 2026.

Taking Your First Steps with GenAI in Quality Engineering

April 21, 2025 by Aishling Warde

Generative AI (GenAI) is increasingly being recognized for its potential to enhance quality engineering by generating content from existing information to achieve known outcomes. However, determining where to begin can be challenging.

Key Activities for GenAI Implementation

Reviewing Requirements/User Stories – Use GenAI to analyze and refine requirements, ensuring clarity and completeness.
Generating Test Cases/Scenarios – GenAI can quickly generate diverse scenarios, reducing the manual effort involved.
Generating Test Scripts – Generate test scripts, or agile session sheets, that can be used for manual testing.
Generating Automation Code – Focus on small functions rather than entire frameworks to incrementally enhance your automation suite.
Generating Bug Reports – GenAI can help standardize bug reports, making them more useful for developers and save you time during the execution process.

Choosing the Right Starting Point

When choosing where to start, identify the activity that causes the most pain or time loss and poses the least risk if the outcome isn’t perfect. This strategic choice will free up time to invest in other areas of the lifecycle.

Start Small and Scale Gradually

Remember, the key to successful GenAI implementation is to start small. Master one activity before scaling to others. By doing so, you can gradually build confidence and expertise, ultimately enhancing your quality engineering processes with GenAI.

Which GenAI tool to use?

There are many free or low cost GenAI tools that are available today. Here are some tools you can consider:

ChatGPT
Gemini
Claude
Microsoft Copilot

In most cases you can use the free versions of these tools, but be aware that there may be limits on transaction rates, and your data may be used for training. Speaking of which…

Security & Privacy

Data security & privacy is a critical concern for GenAI usage. Some vendors offer paid versions that provide greater data security and privacy features.

Evaluate the data privacy policies associated services to ensure they align with your organizational requirements.

NEVER Put Sensitive or PII Information Into GenAI Tools!

Quick Start guide to Prompt Engineering

So you’ve identified the activity you want to automate, you’ve selected the GenAI tool of preference…Now to write your first prompt!

Use a robust prompt engineering pattern like R.I.S.E. (Role, Input, Steps, Expected Output) to help guide GenAI to produce the desired results. The following is an example prompt for creating high-level test scenarios:

# Role: You are a software tester
 
# Input: I will provide a requirement
 
# Steps: I want you to generate high-level test scenarios. You should ensure that you generate positive and negative test cases, using equivalence partitioning and boundary analysis.
 
# Expected Output: The response should be in a table with the following headings: 
- TS-ID: A unique test scenario identifier starting with "TS-" 
- Description: A description of the test scenario 
- Type: A value of "Positive" or "Negative" that indicates if the test case is a positive or negative scenario. 
- Expected Outcome: The expected outcome from the test scenario

R.I.S.E is just one pattern, so try experimenting with different ones to see what works best for your input & output.

Experiment and Have Fun!

Author

Jarrod Plant

Jarrod Plant is a seasoned professional with 20 years of experience in software testing consulting, providing him with a diverse range of skills and knowledge in various industries, tools, and corporate cultures. He has a technical background, with experience in both Automation & Performance testing, that is balanced by his experience in customer solutions driving a value-focused delivery.

Driven by his passion for the potential of Artificial Intelligence, Jarrod currently serves as the product owner of Planit’s Quality Engineering-centric Generative AI platform. This role provided him with invaluable firsthand experience in building, managing, and testing a generative AI platform. He believes that we are at the precipice of an evolution in software delivery and testing, brought on by the power and accessibility of AI.

Planit were Exhibitors at EuroSTAR 2025. Join us at EuroSTAR Conference in Oslo 15-18 June 2026.

Unlock the Fun with Passport Around the EXPO at EuroSTAR Conference!

April 15, 2025 by Aishling Warde

At the EuroSTAR Conference EXPO, we’re all about creating engaging, interactive experiences for our delegates, it’s a fun and rewarding challenge for attendees, and a fantastic way for exhibitors to connect with more visitors.

What is Passport Around the EXPO?

The Passport Around the EXPO is an exciting delegate challenge designed to get attendees actively exploring the EXPO floor. Every delegate will receive a ‘passport’ in their conference bag displaying each opted-in exhibitor logo. and must visit partner stands to have their ‘passport’ card stamped.

This initiative serves as a great icebreaker and provides an incentive for delegates to stop by and engage with your booth, while also offering a fun and memorable experience at the conference. It’s a simple, effective way to increase foot traffic to your stand and raise awareness about your company, product, or service.

Why Gamification Works at Booths

Gamification — using game-like elements in non-game contexts — has proven to be an effective strategy in boosting booth engagement. Here are some key statistics showing why this approach works:

85% of attendees are more likely to remember a brand that incorporates a gamified experience at an event.
78% of exhibitors report that gamification increases foot traffic to their booths.
70% of booth visitors are more engaged when interactive activities, like games or challenges, are involved.

Gamified experiences can lead to increased booth engagement by 30% or more, compared to traditional static displays

By participating in the Passport Around the EXPO, your booth becomes part of an engaging experience, increasing the likelihood of attendees stopping by, interacting with your team, and learning about your offerings.

How Does It Work?

Step 1: Participants receive a passport card upon arrival in their EuroSTAR Conference Swag Bag.

Step 2: Delegates visit partner stands throughout the EXPO and get their passport stamped.

Step 3: The challenge is completed once delegates collect all stamps.

The more stamps delegates collect, the more they’ll immerse themselves in the excitement and fun of the conference—and increase their chances of winning the very first ticket to next year’s EuroSTAR Conference!

Why Should You Participate?

Passport Around the EXPO isn’t just a way to engage delegates; it’s also a fantastic opportunity for your booth to stand out. By participating, you’ll be:

Maximise Brand Exposure: Participating in the Passport Around the Expo puts your logo directly into the hands of every conference attendee. It’s a high-impact way to boost brand visibility and ensure your company is top-of-mind as delegates navigate the EXPO Hall
Building Connections: It offers a great reason for delegates to stop and chat with you, providing an opening for meaningful conversations and relationship-building.
Fun & Interactive: It makes your booth more interactive, turning a simple visit into an engaging experience that delegates will remember.

How Can You Get Involved?

The best part? Participation is completely free and optional. All you need to do is sign up, and we’ll take care of the rest — including providing the Passport cards and stamps. There’s no need for you to supply a prize; EuroSTAR has that covered. All you need to do is be ready to stamp passports and connect with attendees who stop by your stand!

This initiative is a fantastic way to bring a bit of excitement and fun to your EuroSTAR experience, while also boosting your visibility and creating new connections.

We Can’t Wait to See You!

So, if you’re exhibiting at the EuroSTAR Conference, don’t miss out on the Passport Around the EXPO initiative! It’s a fun and easy way to engage with delegates and make the most of your time at one of the largest and most prestigious testing conferences in Europe.

We look forward to seeing you at the conference!

Author

Clare Burke

EXPO Team, EuroSTAR Conferences

With years of experience and a passion for all things EuroSTAR, Clare has been a driving force behind the success of our EXPO. She’s the wizard behind the EXPO scenes, connecting with exhibitors, soaking up the latest trends, and forging relationships that make the EuroSTAR EXPO a vibrant hub of knowledge and innovation.

t: +353 91 416 001
e: clare@eurostarconferences.com

Agentic testing for the enterprise: Ushering in a new era of software testing

April 14, 2025 by Aishling Warde

In today’s fast-paced world of software development, the need for effective testing has never been more important. Conventional approaches to testing are often challenged by rapid release cycles and complicated integrations, leading to slow delivery, high costs, and low software quality. Agentic testing is an innovative way to address these challenges. This transformative approach to software testing empowers testers with advanced AI capabilities, allowing them to automate much broader, more non-deterministic, and non-linear efforts in testing. AI agents take on the tedious and time-consuming tasks, providing a level of productivity that is not possible with conventional testing methods.

So how can you get started with agentic testing? We’re bringing agentic testing to life with the launch of UiPath Test Cloud, now generally available. UiPath Test Cloud—the next evolution of UiPath Test Suite—equips software testing teams with a fully featured testing solution that accelerates and streamlines testing for over 190 applications and technologies, including SAP®, Salesforce, ServiceNow, Workday, Oracle, and EPIC. Let’s take a closer look.

Comprehensive testing capabilities for the enterprise

Test Cloud is an environment where software testers feel at home. It’s your solution for bringing agentic testing to life—augmenting you with AI agents across the entire testing lifecycle. Zooming in, Test Cloud is a fully featured platform designed to serve all your testing needs. Whether it’s functional or performance testing, Test Cloud empowers you with open, flexible, and responsible AI across every stage, from test design and test automation, to test execution and test management. And it’s built for scale—with everything you need to handle the largest and most complex testing projects. It helps you design smarter tests with capabilities like change impact analysis and test gap analysis, ensuring a risk-based, data-driven approach to testing. It gives you the flexibility to automate tests the way you want, whether it’s low-code or coded user interface (UI) and API automation, across platforms. Plus, with continuous integration and continuous delivery (CI/CD) integrations and distributed test execution, Test Cloud seamlessly fits into your ecosystem while accelerating your testing to keep up with rapid development cycles. And when it comes to test management, Test Cloud has you covered with 50+ application lifecycle management (ALM) integrations, as well as a rich set of test data management and reporting capabilities.

Unlock built-in and customizable AI for the entire testing lifecycle with UiPath Autopilot™ for Testers

What makes agentic testing truly agentic? AI agents. With UiPath Autopilot for Testers, our first-party AI agent available in Test Cloud, you’re equipped with built-in, customizable AI that accelerates every phase of the testing lifecycle.

Leverage Autopilot to enhance the test design phase through capabilities such as:

Quality-checking requirements
Generating tests for requirements
Generating tests for SAP transactions
Identifying tests requiring updates
Detecting obsolete tests

Then, use Autopilot to take your test automation to the next level through capabilities such as:

Generating low-code test automation
Generating coded user interface (UI) and API automation
Generating synthetic test data
Performing fuzzy verifications
Generating expressions
Refactoring coded test automation
Fixing validation errors in test automation
Self-healing test automation

And enhance test management with Autopilot capabilities such as:

Generating test insights reports
Importing manual test cases
Searching projects in natural language

Any type of tester—from a developer tester to a technical tester to a business tester—can use Autopilot to build resilient automations more quickly, unlock new use cases, and improve accuracy and time to value. Organizations are already yielding tangible benefits from this versatility and efficiency, as showcased by Cisco’s experience with Autopilot in accelerating their testing processes.

“At Cisco, our GenAI testing roadmap centers on leveraging UiPath LLM capabilities throughout the entire testing lifecycle, from creating user stories to generating test cases to reporting, while ensuring seamless integration with code repositories,” said Rajesh Gopinath, Senior Leader, Software Engineering at Cisco. “With the power of Autopilot, we’re equipped to eliminate manual testing by 50%, reduce the tools used in our testing framework, and reduce dependency on production data for testing.”

Build your own AI agents tailored specifically to your unique testing needs with Agent Builder

Now, let’s meet the toolkit for building AI agents tailored to your testing needs: UiPath Agent Builder. Leverage a prebuilt agent from the Agent Catalog, or build your own agent using the following components:

Prompts: define natural language prompts with goals, roles, variables, and constraints
Context: use active and long-term memory to inform the plan with context grounding
Tools: define UI/API automations and/or other agents that are invoked based on a prompt
Escalations: asks people for guidance with UiPath Action Center or UiPath Apps
Evaluations: ensure the agent meets your desired objectives and behaves reliably in various scenarios

Looking for inspiration to jumpstart your first attempt at building an agent? Here are some recommendations for agents that you can build to help accelerate your testing:

Data Retriever: helps find test data for exploratory testing sessions in databases
Bug Consolidator: identifies distinct bugs behind failed test cases after nightly test runs
Compliance Checker: finds test cases that do not adhere to best practice
Stability Inspector: identifies flaky tests, repeatedly failed tests, and false positives

These are just a few agents that augment your expertise throughout the testing lifecycle. Join the Agent Builder waitlist to be the first in line to try your hand at building one.

Open, flexible, and responsible

Beyond AI agents, what does Test Cloud offer that helps you engage in agentic testing?

With UiPath Test Cloud, you can harness the power of an open and flexible architecture that seamlessly integrates with your existing tools, including connections with your CI/CD pipelines, ALM tools, and version control systems, as well as webhooks that keep you informed in real time. This flexibility ensures that Test Cloud adapts to your unique enterprise needs.

When it comes to responsible AI, you benefit from the UiPath AI Trust Layer, which provides you with explainable AI, bias detection, and robust privacy protection. You can confidently meet regulatory requirements and internal governance standards thanks to comprehensive auditability features. By embracing the open architecture and responsible AI capabilities of Test Cloud, you’re not just streamlining your testing process–you’re future-proofing your software quality with intelligent, efficient, and trustworthy technology that grows with your team’s needs.

Resilient end-to-end automation

With UiPath Test Cloud, you can unlock the power of resilient end-to-end automation that will enhance your testing processes. Experience seamless automation capabilities for any UI or API, giving you unparalleled flexibility in your testing approach. Whether you’re working with home-grown web and mobile applications or complex enterprise systems like SAP, Oracle, Workday, Salesforce, and ServiceNow, you can engage in automated testing that covers all aspects of your software ecosystem. By leveraging powerful end-to-end automation, you’ll not only improve the efficiency of your testing processes but also gain greater confidence in the quality and reliability of your software releases. Customers like Dart Container, Quidelortho, Orange Spain, and Cushman and Wakefield have achieved 90% automation rates, 30-40% cost reduction, 6X faster release speeds, and other significant benefits through using UiPath automated testing capabilities.

Production-grade architecture and governance

You and your team may face the challenge of maintaining a secure, scalable, and compliant testing infrastructure that can keep up with your agile development processes. With Test Cloud, you’re equipped with a production-grade architecture and robust governance features that will transform your agentic testing experience.

Benefit from Veracode certification, ensuring your testing environment meets the highest security standards and giving you peace of mind. Comprehensive auditing capabilities provides you with detailed insights into all testing activities, enabling you to maintain full transparency and easily demonstrate compliance. You also have granular role management features, allowing you to precisely control access and permissions, ensuring that the right people have the right level of access at all times. With centralized credential management, you can streamline security processes and reduce the risk of unauthorized access, making it easier than ever to manage and protect sensitive testing data.

Powered by the UiPath Platform

When you choose UiPath Test Cloud, you’re not just getting a standalone testing solution–you’re tapping into the power of the entire UiPath Platform™. This opens up a world of possibilities for streamlining your testing processes and boosting your overall automation efforts. You’ll benefit from shared and reusable components across teams, allowing you to leverage expertise and reduce duplication of effort. EDF Renewables, for example, achieved 75% component reuse by leveraging testing capabilities within the UiPath Platform. With access to the UiPath Marketplace, you’ll have a wealth of prebuilt solutions at your fingertips, accelerating your testing initiatives. Access to snippets and libraries empowers you to create modular, reusable code that can be easily shared and maintained across your organization. Plus, you can leverage centralized object repositories, which simplify test maintenance and improve consistency across your automation projects. Additionally, the robust asset management capabilities ensure that you can efficiently organize, version, and deploy your automation assets enterprise-wide, maximizing the value of your organization’s investment in the UiPath Platform™.

The benefits of agentic testing with UiPath Test Cloud

No matter your role or ranking at your organization, you can start reaping the benefits of Test Cloud for agentic testing right away. As a CIO, you’ll experience increased efficiency, reduced costs, and better resource utilization, ultimately leading to faster time-to-market and enterprise-wide automation. Testing team leads will benefit from improved consistency and reliability, increased productivity, and better defect detection, while standardizing testing processes and achieving unprecedented scalability. For testers, Test Cloud offers increased accuracy and efficiency, enhanced test coverage, and faster feedback loops, resulting in higher job satisfaction. The tangible benefits are clear: based on an in-depth study conducted by IDC, customers using UiPath for testing have achieved $4M average annual savings per customer, 529% three-year return on investment, and 6 months payback on investment.

With agentic testing powered by Test Cloud, all roles will enjoy accelerated test cycles, deeper test coverage, and reduced risk, all while realizing significant cost savings and resource optimization. This comprehensive and adaptive testing approach will empower your organization to deliver high-quality software faster than ever before, accelerating your time to value and giving you a competitive edge in today’s fast-paced software landscape. This vision of AI-augmented testing is not just theoretical; forward-thinking organizations like State Street are already anticipating how it will transform their testing processes.

The future of agentic testing

Test Cloud isn’t just built for the testing you know today—it’s built for where testing is going. With Test Cloud, you’re not just keeping up with increasing testing demands—you’re staying ahead. Get started with UiPath Test Cloud by signing up for the trial today.

Author

Sophie Gustafson

Product Marketing Manager, Test Cloud, UiPath

The Evolution of AI in Software Testing: From Machine Learning to Agentic AI

April 9, 2025 by Aishling Warde

Everywhere you turn, someone is talking about AI — AI this, AI that. No wonder some people roll their eyes at the mention of artificial intelligence. For some, it’s all smoke and mirrors, just a glorified spreadsheet rather than a technological breakthrough capable of real cognitive reasoning.

And just when you think you’ve caught up, something new appears. First, we had simple machine learning and AI, then came Generative AI, and now Agentic AI is all the rage. If you feel like you’re constantly playing catch-up, you’re not alone.

But whether you love it or loathe it, AI isn’t going anywhere. In fact, some tools are now designed to think, create, and learn—just like Keysight’s Eggplant Intelligence.

The Thinking, Creating, and Learning Framework

This framework simplifies AI by breaking it into three key functions:

Thinking involves decision-making and adaptability, much like Agentic AI, which enables AI to make choices based on real-time data.
Creating is tied to generative AI capabilities, allowing AI to generate test cases and user scenarios autonomously.
Learning follows the principles of traditional machine learning, pioneered by Alan Turing in 1950, and enables AI to improve over time based on historical data.

Figure 1: Eggplant Intelligence supports the entire Quality Engineering Lifecycle

So, what’s the real difference between these AI types? How do they impact software testing? And does anyone actually care? The short answer: there are plenty of differences, they have a huge impact, and yes, you should care.

Before we unravel these questions, let’s take a trip down memory lane to understand how we got here.

The Birth of AI in Software Testing – Keysight Eggplant’s Heritage

Back in 1947, Alan Turing gave a lecture that introduced the idea of a machine’s ability to exhibit intelligent behaviour and learn just like a human. Since then, ‘machine learning’ and artificial intelligence has evolved considerably, and in 2018, Keysight Eggplant integrated such tools into its Digital Automation Intelligence (DAI) platform, which is now known as Eggplant Test. This was groundbreaking then and remains so today, enabling automated software testing to:

Identify all user journeys – Machine learning algorithms analyze applications and uncover every possible user journey to generate test cases automatically, improving test coverage and reducing manual effort.
Prioritize test cases – By learning from historical test runs and code changes, the system can pinpoint high-risk areas and prioritize testing where it matters most, optimizing testing time and resources.
Detect anomalies – AI can track normal system behavior, spot deviations, and flag potential defects early in the development cycle.
Adapt test scripts – Automated scripts dynamically adjust to application changes, minimizing maintenance and improving long-term test stability.

This goes beyond simple test automation. Imagine changing your payment gateway on an eCommerce site—Eggplant can auto-generate new test cases to reflect the update without requiring hours of script rewrites. That’s the power of intelligent automation.

But AI in software testing isn’t just about running test cases. Keysight Eggplant Test has also led the way in image-based testing, optical character recognition (OCR), and computer vision—critical for automating graphical user interface (GUI) testing in complex, secure environments.

Generative AI – Automating Test Creation

Next up: Generative AI, the “Creating” part of the framework. This subset of AI revolves around understanding and generating human-like language through natural language processing (NLP), including large language models (LLMs).

Generative AI can be used to automate test cases, reducing manual effort while improving accuracy. But Keysight is taking it a step further—our Gen AI capabilities are in development to generate test case frameworks directly from software requirements documentation, allowing testers to refine them rather than start from scratch once launched.

Security is also a major priority, which is why when Eggplant Test with Gen AI is launched it will operate using secure, offline, technology-agnostic LLMs. Unlike cloud-based solutions, our models will be deployed on-premises, ensuring complete control over sensitive data and compliance with strict security regulations.

Cloud-based AI testing tools that use ChatGPT pose risks, such as “shadow prompting,” where unchecked user inputs generate unreliable outputs. While techniques like prompt engineering can mitigate this, on-premises AI solutions eliminate the risk altogether.

Agentic AI – The Next Evolution

Now, we arrive at Agentic AI, the “Thinking” part of our framework. This evolution introduces intelligent agents that can autonomously design, execute, and optimize test cases. Using chain of thought, a technique that stacks multiple commands to perform complex tasks, these agents perform intricate testing, ensuring all possible user interactions and edge cases are covered.

Another breakthrough is computer use agents (CUA) such as large action models (LAMs), which automate browser-based processes by interacting with web applications just like human testers. This is crucial for end-to-end web testing across various devices and browsers.

And then there’s large vision models (LLaVA), which enhance technologies like traditional computer vision to interpret and validate visual data, verifying UI elements and graphical components in applications.

Sound familiar? It should. Eggplant Intelligence already integrates elements of AI, Gen AI, and Agentic AI into a single platform. Our system optimizes test coverage, automates interactions across digital environments, and executes tests just as a human would, all while remaining offline and compliant with AI governance laws in the UK, EU, and US.

AI Testing Compliance – The Keysight Advantage

Many testing tools rely on cloud-based AI architectures, making them non-compliant with the EU AI Act and other regulatory frameworks. Cloud-based solutions often fail to meet the strict security demands of regulated industries, leaving organizations exposed to potential privacy violations.

For industries like aerospace, defense, and healthcare—where data security is non-negotiable—cloud-based AI testing tools are simply not an option. Storing customer or intellectual property data outside a secure firewall can lead to legal consequences and hefty fines.

This is why Keysight Eggplant is the only AI-powered testing solution that prioritizes security, transparency, and governance. Our on-premises approach ensures that all sensitive data remains secure, meeting even the most stringent compliance requirements.

And let’s be clear—using cloud-based AI for test script generation or test reports is not only risky but illegal in many jurisdictions. GDPR and other data protection laws prohibit storing customer data outside of an organization’s firewall, making cloud AI tools a liability for compliance-conscious businesses.

The Future of AI in Software Testing

AI in testing isn’t just about keeping up with the latest buzzwords. It’s about making smart, future-proof choices that balance innovation with security, scalability, and compliance.

Keysight Eggplant has been pioneering AI-driven testing since 2017, long before many of today’s players entered the field. As AI evolves, we continue to push boundaries, ensuring our platform remains at the cutting edge of secure, offline AI testing.

So, if you’re serious about automated software testing and need a future-proof, AI-driven platform that doesn’t compromise security, compliance, or flexibility—it’s time to take a closer look at Keysight Eggplant.

Header image is a photo by Mauro Sbicego on Unsplash.

Author

Mike Wager

Product Marketing Manager at Keysight Technologies

Keysight were Gold Partners in EuroSTAR 2025. Join us at EuroSTAR Conference in Oslo 15-18 June 2026.

Thought you couldn’t automate that? Think again.

March 5, 2025 by Aishling Warde

As testers looking to increase automated coverage, we often encounter parts of the user journey – such as when an email notification, SMS alert, or multi-factor authentication workflow is involved – that on the surface appear difficult or outright impossible to automate. This leads to reliance on manual testing, which can be time-consuming and prone to human error, affecting the speed of your testing process or likelihood of regression.

However, with the right tools, even complex test scenarios can be automated effectively, improving test coverage and accelerating release cycles. Let’s dive into some common challenges and how they can be tackled with automation.

Transactional emails or SMS alerts

If you’re responsible for testing a modern customer experience, for example a hotel booking system, the emails or SMS messages sent by that system are as critical to the user experience as the screens with which a user interacts.

Whilst it’s possible to pull together a solution reliant on a public email service like Gmail, or self-host and maintain an open-source solution. You’ve got better things to be spending your time on! Most now solve this by integrating a managed service like Mailosaur to access email testing functionality, and international phone numbers for SMS, to capture and perform assertions against every message a product sends.

Mailosaur facilitated easy email testing, streamlined QA automation … and provided crucial support for evergreen projects.” – Manager of Software Quality Engineering at Domino’s.

Authentication workflows

Whether you’re responsible for a fintech application, healthcare portal, or a secure ecommerce platform, you’ll almost certainly be responsible for securing access to key areas of your product. That means the introduction of anything from email verification on signup, password reset workflows, and the increasingly essential two-step verification processes.

Sure, if you’re a ‘one-man band’ you can use your own phone to set up 2FA/MFA at least, but not only does that not work for automation, it also falls apart as soon as you’re working at any kind of scale – whether that’s more QAs around you, or a product that supports sending SMS messages in more than one country.

Things have moved on, and there are now solutions that not only give businesses access to phone numbers internationally, but there are also solutions that can mimic apps like LastPass and Google Authenticator to test “auth app” functionality. Here’s a snippet showing how this is done using our platform:

// Get a one-time password (OTP) from an SMS 
const sms = await mailosaur.messages.get(inboxId, { 
  sentTo: ‘123456789’ // phone number 
}); 
 
console.log(sms.text.codes[0].value); // e.g. “654321” 
  
// Get a one-time password (OTP) from a ‘security device’ 
const currentOtp = await mailosaur.devices.otp(sharedSecret); 
console.log(currentOtp.code);  // e.g. “123456”

Email rendering bugs and blind spots

The way an email looks to your customer can differ wildly, depending on which email client they are using. While it might look perfect to you on Gmail or the latest edition of Outlook on Windows, there are a ton of email clients outside your organization.
With Mailosaur’s email preview functionality, you can instantly see how your email will look for any recipient, no matter what email client they use, so there are no nasty surprises.

Come and see us at EuroSTAR 2025!

We required a solution that could provide multiple servers, allow us to generate test email addresses, and support extensive email testing. Additionally, the ability to automate and perform high volumes of email testing daily was critical to our operations’’ – Sr. Director, Quality Engineering at Malwarebytes.

Automating traditionally difficult test scenarios is easier than you think, with the right tools, those “impossible” tests become straightforward and we’re here to show you how at EuroSTAR this June.

Stop by the Mailosaur stand for live demos, practical advice, and a chance to chat with our team about simplifying your toughest tests.