• Skip to main content
Year End Offer: Save 30% + Teams Save Up to an Extra 25%!

EuroSTAR Conference

Europe's Largest Quality Engineering Conference

  • Programme
    • Programme Committee
    • 2025 Programme
    • 2025 Speakers
    • Community Hub
    • Awards
    • Social Events
    • Volunteer
  • Attend
    • Location
    • Highlights
    • Get Approval
    • Why Attend
    • Bring your Team
    • Testimonials
    • 2025 Photos
  • Sponsor
    • Sponsor Opportunities
    • Sponsor Testimonials
  • About
    • About Us
    • Our Timeline
    • FAQ
    • Blog
    • Organisations
    • Contact Us
  • Book Now

Uncategorized

Efficient Software Testing in 2023: Trends, AI Collaboration and Tools

May 31, 2023 by Lauren Payne

Thanks to JetBrains Aqua for providing us with this blog post.

In the rapidly evolving field of software development, efficient software testing has emerged as a critical component in the quality assurance process. As we navigate through 2023, several prominent trends are shaping the landscape of software testing, with artificial intelligence (AI) taking center stage. We’ll delve into the current state of software testing, focusing on the latest trends, the increasing collaboration with AI, and the most innovative tools.

Test Automation Trends

Being aware of QA trends is critical. By staying up to date on the latest developments and practices in quality assurance, professionals can adapt their approaches to meet evolving industry standards. Based on the World Quality Report by Capgemini & Sogeti, and The State of Testing by PractiTest, popular QA trends currently include:

  • Test Automation: Increasing adoption for efficient and comprehensive testing.
  • Shift-Left and Shift-Right Testing: Early testing and testing in production environments for improved quality.
  • Agile and DevOps Practices: Integrating testing in Agile workflows and embracing DevOps principles.
  • AI and Machine Learning: Utilizing AI/ML for intelligent test automation and predictive analytics.
  • Continuous Testing: Seamless and comprehensive testing throughout the software delivery process.
  • Cloud-Based Testing: Leveraging cloud computing for scalable and cost-effective testing environments.
  • Robotic Process Automation (RPA): Automating repetitive testing tasks and processes to enhance efficiency and accuracy.

QA and AI Collaboration

It’s no secret that AI is transforming our lives, and ChatGPT’s collaboration can automate a substantial portion of QA routines. We’ve compiled a list of helpful prompts to streamline your testing process and save time.

Test Case Generation

Here are some prompts to assist in generating test cases using AI:

  • “Generate test cases for {function_name} considering all possible input scenarios.”
  • “Create a set of boundary test cases for {module_name} to validate edge cases.”
  • “Design test cases to verify the integration of {component_A} and {component_B}.”
  • “Construct test cases for {feature_name} to validate its response under different conditions.”
  • “Produce test cases to assess the performance of {API_name} with varying loads.”
  • “Develop test cases to check the error handling and exceptions in {class_name}.”

Feel free to modify these prompts to better suit your specific testing requirements.

Example
We asked for a test case to be generated for a registration process with specific fields: First Name, Last Name, Address, and City.

AI provided a test case named “User Registration” for the scenario where a user attempts to register with valid inputs for the required fields. The test case includes preconditions, test steps, test data, and the expected result.

Test Code Generation

In the same way, you can create automated tests for web pages and their test scenarios.

To enhance the relevance of the generated code, it is important to leverage your expertise in test automation. We recommend studying the tutorial and using appropriate tools, such as JetBrains Aqua, to write your tests that provide tangible examples of automatically generating UI tests for web pages.

Progressive Tools

Using advanced tools for test automation is essential because they enhance efficiency by streamlining the testing process and providing features like test code generation and code insights. These tools also promote scalability, allowing for the management and execution of many tests as complex software systems grow.

UI Test Automation

To efficiently explore a web page and identify available locators:

  • Open the desired page.
  • iInteract with the web elements by clicking on them.
  • Add the generated code to your Page Object.

This approach allows for a systematic and effective way of discovering and incorporating locators into your test automation framework.

Code Insights

To efficiently search for available locators based on substrings or attributes, you can leverage autocompletion functionality provided by the JetBrains Aqua IDE or plugin.

In cases where you don’t remember the location to which a locator leads, you can navigate seamlessly between the web element and the corresponding source code. This allows you to quickly locate and understand the context of the locator, making it easier to maintain and modify your test automation scripts. This flexibility facilitates efficient troubleshooting and enhances the overall development experience.

Test Case As A Code

The Test Case As A Code approach is valuable for integrating manual testing and test automation. Creating test cases alongside the code enables close collaboration between manual testers and automation engineers. New test cases can be easily attached to their corresponding automation tests and removed once automated. Synchronization between manual and automated tests to ensure consistency and accuracy is a challenge that does not need to be addressed. Additionally, leveraging version control systems (VCS) offers additional benefits such as versioning, collaboration, and traceability, enhancing the overall test development process.

Stay Tuned

The industry’s rapid development is exciting, and we are proud to be a part of this growth. We have created JetBrains Aqua, an IDE specifically designed for test automation. With Aqua, we aim to provide a cutting-edge solution that empowers testers and QA professionals. Stay tuned for more updates as we continue to innovate and contribute to the dynamic test automation field!

Author

Alexandra Psheborovskaya, QA Lead and Product Manager at JetBrains

Alexandra works as a SDET and a Product Manager on the Aqua team at JetBrains. She shares her knowledge with others by mentoring QA colleagues, such as in Women In Tech programs, supporting women in testing as a Women Techmakers Ambassador, hosting a quality podcast, and speaking at professional conferences.

JetBrains is an EXPO Platinum partner at EuroSTAR 2023, join us in Antwerp

Filed Under: Software Testing, Uncategorized Tagged With: 2023, EuroSTAR Conference

Check out the EuroSTAR 2023 EXPO (Including Prizes!)

May 22, 2023 by Fiona Nic Dhonnacha

Are you ready for EuroSTAR 2023? The conference kicks off in just a few weeks, with 4 awesome days of testing, learning, and connecting with the global testing community.

The EXPO hall is at the heart of EuroSTAR, and draws some of the biggest companies in the world! Meet one-on-one with pioneering organizations, explore the latest solutions and tools in testing, see live demos, and find new ways to solve your testing problems. Bring all your questions too!

It’s also a great way to get yourself out there and meet new people during the session breaks. Network with your fellow attendees – and of course, visit all the EXPO booths throughout the conference to find out how you can get your hands on some of the incredible prizes below!

A big thank you to our Platinum Partners Jetbrains, Karate Labs, LambdaTest, SauceLabs and Xray; and our gold partners, along with all our brilliant exhibitors. Check out all our 2023 partners.

Get Tickets
Check out the EuroSTAR 2023 EXPO to be in with a chance to win this super cool Batman helmet from LambdaTest.

I’m Batman! Scoop this very cool Batman Lego helmet from LambdaTest, who will also demonstrate how your businesses can drastically reduce time to market, through faster test execution: ensuring quality releases and accelerated digital transformation.

Sauce Labs provides the world’s largest cloud-based platform for the automated testing of web and mobile applications – and they have a chance for you to win a €100 Amazon voucher if you stop by their booth and enter the draw!

Say hello to the team at the Xray booth and be in with a chance to win Sony airpods. You can also hear more about their extensible test management platform – their REST API and out-of-the-box integrations make it easy to build a CI/CD pipeline.

Check out the EuroSTAR 2023 EXPO to be in with a chance to win these Beats headphones from HCL.

HCL Software‘s aim is to drive ultimate customer success; and it’s the cloud-native solution factory for enterprise software – powering millions of apps at more than 20,000 organizations. Enter their draw to win a set of Beats headphones.

Check out the EuroSTAR 2023 EXPO to be in with a chance to win this Lego hut from Leapwork.

Stop by the Leapwork team’s booth to be in with a chance to win this incredible Lego hut! You’ll also learn more about their entirely visual no-code system that everyone can understand, to empower your digital transformation.

Tea and coffee drinkers should stop by Mabl‘s booth, who are offering a gorgeous subscription to Sips By. You can also discuss their test automation platform built for CI/CD, and see how it can work for your business.

Enter the draw at the OpenText booth and you could scoop this slick gaming keyboard. It’s their 11th year partnering with us and they’re looking forward to chatting to you, and demonstrating their innovative solutions to help your business thrive.

The team at SmartBear are ready and waiting to show you the tools to help streamline your software process while seamlessly working with all the tools you already use. Plus, it’s your chance to win a portable laptop monitor.

Sogeti‘s quality engineering & testing practice help empower you to deliver the positive experience your customers expect. Enter their competition and you might be walking away with a VR headset and controller.

It’s TestRail‘s third year joining us at EuroSTAR, and they’re on hand to help you speed up testing, improve product quality and ship releases faster. Stop by their booth and you might win a Star Wars Lego set.

Chat to Testsigma to learn more about their open source no-code test automation tool, designed to reduce your test creation and maintenance time. The team are also offering a lucky competition winner an Apple iPad!

Tricentis are joining us for their 12th year – we’re thrilled to have them on board again. Stop by to hear about their AI-based, automation platform – and enter their booth competition to win some Apple airpods.

ACCELQ is proven to speed up automation development by 3 times and bring down the maintenance by 70% – have a chat with the team to see what they can do for you. Plus, they have a Google Home up for grabs.

You could win a super cool Nintendo Switch if you stop by the b.ignited booth. The team are excited to meet everyone and share how they produce no-nonsense software, automated by testing wizards!

Take some time to find out more about PractiTest‘s test management solution, which incorporates all testing elements in one central platform, to empower your organisation. Enter their competition and you could win this incredible drone.

*Photos are representations only. All prize details will be provided by each partner and these photos do not indicate the exact make or model of what is being provided.

That’s just a taster of what to expect – there are lots more prizes to be won, including:

  • iPad from Applitools
  • Lego Antwerp set from CTG
  • The Atari Flashback console from Curiosity Software
  • €100 Amazon voucher from Hexaware
  • Airpods from Inflectra
  • A Lego rocket set from PII Guard

It’s going to be an unforgettable 4 days, so grab your ticket now and we’ll see you at Antwerp Zoo.

book tickets

Filed Under: Uncategorized

EuroSTAR Testing Excellence Award

May 2, 2023 by Fiona Nic Dhonnacha

The 2023 EuroSTAR Testing Excellence Award is open for nominations!

Take a moment to consider who you would like to see recognized for their outstanding dedication and contributions to the world of software testing. It might be your colleague, your mentor, team-leader, or an author whose knowledge sharing has helped you in your career.

Community recognition

The Testing Excellence Award is the highest honour awarded by our community each year. It highlights outstanding achievements by the exceptional people in our community: those who help us achieve more, be more, and aim for more by continuously striving for excellence in their work.

Who can nominate?

Anyone can make a nomination for the EuroSTAR Testing Excellence Award, but you will need input from others in the community to demonstrate your nominee’s contributions to the overall software testing community.

nominate now

Who to nominate?

Think of your colleagues, friends, managers or mentors. Who through their passion, learning and willingness to share, significantly helps to move our industry and community forward?

They are usually driving innovation, research ways of advancing testing as a craft (and openly share their research!), and/or perhaps they are eternally supportive of others in the industry and help others grow.

They may have:

  • Increased public awareness of the importance of software quality & testing
  • Contributed to the improvement of software development and testing processes
  • Advanced the publication of research findings in the field of software quality and testing
  • Or it’s likely they will have promoted a love of further education and lifelong learning in software quality and testing

What to include?

  • About the Nominee – who they are, what they do, some biographical info
  • Highlights of Contributions to the wider Testing Community
  • Why they deserve this Award
  • Testimonials from members of the Community
  • If they have LinkedIn recommendations, you can also add these
EuroSTAR Testing Excellence Award past winners

Celebrating testing legends

The existing winners of the EuroSTAR Testing Excellence Award are some of the most influential members of the global testing community, from AnneMarie Charrett to Martin Pol. Each winner has made a difference to our community.

Commemorating achievement

Every year, we present the winner with a pdf of their nomination as a keepsake so the more you can include, the more they’ll see how much they are appreciated within the community. Here are some of the lovely testimonials given to Award winners in the past:

  • “a well-known expert who is recognized as a renowned thought leader in global Software Testing Community.”
  • “part of the role of a tester is to make people think – think about how something could be tested, think about the risks they may face, think about how to communicate that information and this year’s winner must he thanked for their continued efforts in making thinkers out of all of us.”
  • “has always been a supportive and challenging friend in testing. I know that I am far from alone in being able to say that she has helped many testers around the world to become better.”
  • “Nobody knows more practical things about testing, test management, test strategy – all things testing, than her”.
  • “There are many, many people in the testing community whom I respect and from whom I can learn. However, there are only a handful I regard as truly guiding influences.”

When to submit

Entries must be submitted by 21st May, so start your nomination today. If you have any questions at all, please get in touch.

start nomination

Filed Under: Uncategorized

Why Do Testers Need CI/CD Systems?

April 19, 2023 by Lauren Payne

Thanks to JetBrains for providing us with this blog post.

This post was originally published on the JetBrains Qodana Blog.

Competency in the TestOps field is now just as much an essential requirement for QA engineers as the ability to write automated tests. This is because of the ongoing development of CI/CD tools and the increasing number of QA engineers who work with pipelines (or the sequence of stages in the CI/CD pipeline) and implement their own.

So why is CI/CD such an excellent tool for quality control? Let’s find out!

Running Tests Automatically

Automated tests haven’t been run locally in what feels like ages. These days, CI/CD pipelines run tests automatically as one of their primary functions.

Pipeline configuration can be assigned to DevOps. But then we will be a long way from making use of the CI/CD tool’s second function: quality control, or more precisely, “quality gates”.

Quality Control Using Quality Gates

But what are quality gates? Let’s say the product code is like a castle. Every day, developers write new code – which could weaken the foundations of our castle or even poke holes in it if we are really unlucky. The purpose of a QA engineer is to test each feature and reduce the likelihood of bugs finding their way into product code. Lack of automation in the QA process could cause QA engineers to lose sleep, since there is nobody to watch over all the various metrics – especially at dangerous times, like Friday evenings when everyone wants to leave work and is hurrying to finish everything. An ill-fated merge at that moment can cause plenty of unwanted problems down the line.

This problem can be solved by building-in quality checks.

Each check deals with a different important metric. If the code doesn’t pass a check, the gates close, and the feature is not allowed to enter. A feature will only be merged into the product when it has passed all the checks and potential bugs have been fixed.

What Quality Checks can be Included in the CI/CD Pipeline?

We need to put together a list of checks to ensure that the process is as automated as possible. They can be sequenced in a “fail first” order. A feature must pass all the checks to get through the pipeline successfully. The initial checks ensure the app is capable of working: build, code style check, and static analysis.

“Build” speaks for itself: if the app fails to build, the feature does not progress. It is important to incorporate a code style check into your CI/CD pipeline to ensure the code meets unified requirements, as doing so allows you to avoid wasting time on this kind of bug during code reviews.

Static analysis is an essential tool for judging code quality. It can point out a vast number of critical errors that lead to bugs and decrease the number of routine and repetitive tasks for the QA team. Afterwards, developers should fix the detected issues and hand the code over for the testing stage.

We then continue with stage-two checks: unit tests with coverage analysis and coverage quality control, as well as integration and systems tests. Next, we review detailed reports of the results to make sure nothing was missed. At this stage, we may also perform a range of non-functional tests to check performance, convenience, security, and screenshot tests.

When developing a pipeline, we need to pay attention to 2 competing requirements:

  1. The pipeline must guarantee the best possible feature quality in light of your needs.
  2. Time spent running the pipeline should not slow down your workflow. It should generally take no more than 20 minutes.

Examples of Tools to Incorporate in Quality Checks

Code Style Highlighting

A code style is a set of rules that should be followed in every line of code in a project, from alignment rules to rules like “never use global variables”.

You might be wondering what style has to do with testers. The answer is a lot. A style check provides several benefits for QA experts, not to mention the rest of the team:

  1. A unified style helps developers work with the code and gives them more time to implement new features and fix bugs.
  2. A unified style allows you to dispense with manual code checks and use a CI/CD tool to run the checks instead.

Large companies usually have their own style guides that can be used as examples. For instance, Airbnb has a JavaScript style guide, and Google maintains several guides. You can even write your own, should you wish.

The choice of tools for code checking depends on the language. You can find a suitable tool on GitHub or find out which tools other teams use. Linters use bodies of rules and highlight code that fails to abide by them. Some examples include ktlint for Kotlin or checkstyle for Java.

Static Code Analysis

Static code analysis is a method of debugging by examining source code without executing a program. There are many different static code analyzers on the market.

We’ll now look at a platform we’re developing ourselves – Qodana. The significant advantage of this code analyzer is that it includes a number of inspections that are available in JetBrains development environments when writing code.
Many of you probably use an IDE-driven approach, where the IDE helps you write code and points out bugs such as suboptimal code usage, NullPointerExceptions, and duplicates.

But unfortunately, you can never be sure all the critical problems found by the IDE were fixed before the commit. However, you can ensure that the issues will be addressed by incorporating Qodana into your CI/CD pipeline.

Qodana, the latest addition to the family of products from JetBrains, is a cutting-edge static analysis platform designed to help developers and QA engineers improve their code quality, making it more efficient, maintainable, and bug-free. Its static analysis engine is the only solution on the market that brings native JetBrains IDE code inspections to any CI/CD pipeline. The platform provides an overview of project quality and lets you set quality targets, track progress, and automate routine tasks like code reviews.

Interactive inspection report in the Qodana code quality platform.

If you can’t fix everything at once, you can select critical problems, add them to the baseline, and gradually work your way through the technical debt. This allows you to avoid slowing down the development process while keeping the problems that have been found under control.

The updated baseline in the Qodana code quality platform.

Test Coverage

Test coverage is a metric that helps you understand how well your code has been covered by your tests (generally unit tests).

Here, you need to define the minimum coverage percentage you want to support. The code won’t be able to go live until it has been covered sufficiently by the tests. The minimum percentage is established empirically, but you should remember that even 100% coverage may not completely save your code from bugs. According to this article from Atlassian, 80% is a good figure to aim for.

Different coverage analyzers are available for other languages, such as Jacoco for Java, Istanbul for JavaScript, or Coverage.py for Python. You can build all these analyzers into your CI/CD pipeline and track the metrics with ease.

Shaping the Release Process

In addition to automatically running tests and ensuring particular code quality requirements are satisfied, the CI/CD tool lets testers organize the release process.

The release process can be complex and depend on many different manual actions. It is often a completely manual process: the artifact is created by a developer, then passed to the testers for checks, and finally comes to the person who knows how to roll it out for the go-live. Once again, there are a lot of potential choke points here. For instance, one of those people could fall ill or go on vacation.

An effective release process will look different for each team, but it will generally include the following steps:

  1. Each change in the Git branch triggers a build of the app.
  2. The build undergoes quality checks and does not become part of the main branch until it passes all the checks successfully.
  3. A release candidate is taken from the release branch or the main branch: this fixes the version and guarantees that nothing will go live unless it has been tested and has not been changed afterwards. This helps with tracking releases and all the changes they include. In addition, storing artifacts of the stable version makes it possible to revert to them quickly in the event of an unsuccessful release.
  4. The release candidate is tested and undergoes final checks.
  5. The release candidate goes live. This may be either a manual or automated pipeline launch, if the release candidate passed all the checks at the preceding stage. The choice between an automatic release process and a manual one will depend on how frequent and important the releases are, as well as the preferences among team members and the convenience of the rollout.

Any CI/CD system allows you to set up this type of process, which should be convenient for the whole team, including the testing team.

Given the factors outlined above, we believe following these basic rules will help ensure an easy and efficient release process:

  • Artifacts must be ready for download and testing, ideally stored in one place.
  • As many checks and tests as possible must be automated.
  • All complex operations with builds should be as automated as possible.
  • All builds that will go live should be recorded and remain available for a certain period after release. This will help if you need to investigate errors in the production version, reproduce bugs, or just track the history.

We would also like to remind you that if quality metrics are not controlled automatically and are not actionable, they are useless, as there’s no way to guarantee that these metrics will be adhered to.

Implement pipelines, automate processes, and use static code analysis!

Your Qodana team

Author

Alexandra Psheborovskaya, QA Lead and Product Manager at JetBrains

JetBrains is a global software company that creates professional software development tools and advanced collaboration solutions trusted by more than 12.8 million users from 220 counties and territories. Since 2000, JetBrains has built a catalog of 34 products, including PyCharm, IntelliJ IDEA, ReSharper, PhpStorm, WebStorm, Rider, YouTrack, Kotlin, and Space, a new integrated team environment.

Qodana is the code quality platform from JetBrains. It provides a project overview and lets developers and QA engineers set up quality gates, enforce project-wide and company-wide coding guidelines, better plan refactoring projects, and perform holistic license audits. Qodana’s static analysis engine enriches CI/CD pipelines with all of the smart features of JetBrains IDEs, supports 60+ languages and technologies, and allows analysis of unlimited lines of code.

JetBrains is an EXPO Platinum partner at EuroSTAR 2023, join us in Antwerp

Filed Under: DevOps, Test Automation, Uncategorized Tagged With: 2023, EuroSTAR Conference, Test Automation

EuroSTAR 2023 Track Talks

April 11, 2023 by Fiona Nic Dhonnacha

EuroSTAR 2023 is just a few weeks away, and we’re gearing up for an incredible 4 days of learning, community, and connections. Get your conference ticket now for an all-access pass to an inclusive, friendly, and memorable experience at Europe’s best software testing event. Think career-defining learnings, building new skills, having chats with a community of testers similar to you, and helping others to learn and grow.

EuroSTAR is bringing together the world’s leading experts in software testing – 66 of them, in fact. They are coming to Antwerp Zoo armed with ideas, insights, innovation, experience and their passion, to share it all with you.

With 65+ sessions all about software testing, you’re spoilt for choice when it comes to attending talks. We’ve got inspiring keynotes, in-depth tutorials and interactive track talks. The EuroSTAR tracks talks is your chance to get insights and ask questions at the 15 minute Q&A.

Want a little taster of what to expect? Here are just 20 things you will learn at this year’s track sessions.

get tickets
  1. See why tools are just helpers, not the main actors in our work – and how minimilism builds high quality products.
  2. Get a practical introduction on how you can introduce several different techniques to visualize and clarify workflows and user needs in a practical and easy, documented manner.
  3. Learn how you can automate some of the 10 Usability Heuristics, and how a tool could support testers to do manual UX testing.
  4.  Enhance your critical thinking skills and find ‘true knowledge’”’ about software based on the principles of Nyaya Shashtra.
  5. See how browser automation works behind the scenes, and get to know the WebDriver BiDi project and its current progress.
  6. Find out what a personal knowledge management system is, and why it’s essential for any career in the tech industry.
  7. Learn why you need to start with ‘why’ for quality standards awareness and improvements.
  8. Get a 10-step recipe in exploratory unit testing of a program created using computer-assisted software authorship.
  9. Use the Quality Delta concept to avoid doing unneeded work and map out the right questions to ask.
  10. Success stings if it comes at the expense of the hive – see why working well together requires dismantling competitive and hierarchical structures
  11. Change is both constant and unpredictable – learn how to adapt to change by creating a learning culture.
  12. Make yourself future proof and learn 4 key skills needed in the 21st century.
  13. Revisit how you work, communicate, test and build software in a remote first working world.
  14. Avoid unnecessary release disasters by recognising symptoms and taking appropriate action.
  15. Learn how to delegate meaningful tasks, how to communicate with trust and empowerment, and how to make sure you’re not micromanaging.
  16. Rock on! See what it takes to be a rounded rock star tester, and what skills to target to enhance your career.
  17.  Learn about test gap analysis, a novel approach that combines static and dynamic analyses to discover untested changes.
  18. Take important steps to improve the health of your team, and create an environment where you can serve both internal and external customers effectively.
  19. See what it takes to make the transition from Senior Tester, Test Lead, or Tester, into a fully-fledged Test Manager Role.
  20. Learn the ten commandments of automation, and ways to start driving automation to more collaborative action.

Whatever your testing goals and challenges, get the answers you’re looking for, along with lots of new perspectives, at EuroSTAR.

get tickets

Filed Under: Uncategorized

ChatGPT for Automated Testing: From Conversation to Code

April 5, 2023 by Lauren Payne

Thanks to SauceLabs for providing us with this blog post.

You’ve probably heard the buzz about what ChatGPT can accomplish but what testing opportunities are there for developers?

You’ve probably heard of ChatGPT, the most recent advance in Natural Language Processing that is taking the internet by storm with over one million users in just five days!

Released only a few weeks ago, the free AI chatbot’s overnight popularity is due to its ability to answer and explain concepts plainly, or perform complex tasks such as writing an article or telling a joke. For web and mobile app developers, there are broad implications for automated testing, since ChatGPT can write test cases in a wide range of frameworks and languages. Let’s explore the possibilities.

What is ChatGPT?

Created by OpenAI, ChatGPT is a large language model that was fine-tuned by supervised and reinforcement learning on large datasets. Using algorithms, ChatGPT analyzes the data to find patterns that help it understand natural language and how words are used in context. It is a dialogue-based model, which means that it is designed for a back and forth chat-like interaction. You can ask ChatGPT to say whatever you want that the system doesn’t consider offensive.

ChatGPT can generate creative and on-topic responses, often providing details on why it generates a specific response. It can also remember its previous responses to have a (mostly) coherent conversation. ChatGPT has been asked to generate everything from answers to questions on soil physics to writing folk songs about beer!

ChatGPT and Automated Testing

One of the most interesting features of ChatGPT to those of us in the software space is that it can generate properly formatted and relevant code based on a simple natural language request. It can generate code in many languages and can employ numerous built-in packages across those languages. So, the natural question is, can ChatGPT be used to generate code for automated testing?

The answer at this point is, “Yes, sort of.” ChatGPT can write Selenium in multiple languages, as demonstrated by Sauce Labs’ own Nikolay Advolodkin in his video, ChatGPT Can Code Better Automation Than Me.

But being able to write code that looks accurate is just the beginning. In an ideal world, one would provide ChatGPT with a description of the test to generate, it would know all the details about the version of the website one is testing, and it would generate perfect, executable code that needs no modifications. ChatGPT currently can’t do that. What it can do is still quite impressive though.

Let’s start with a use case for ChatGPT, not to completely replace testing engineers, but instead as a new low-coding method.

What is Low-Code Testing?

Low-code development allows people to write code with little to no coding experience, using drag-and-drop development platforms or plain English. Low-code testing solutions lower barriers to writing test code by making it easier to scale development organizations. Since tests can be written without a technical skillset, low-code is simpler for teams to write test automation code, reducing test debt.

ChatGPT can be viewed as a powerful low-code tool for writing test cases. ChatGPT accepts natural language as input, so users can write in their natural cadence and still be understood, unlike template-based models which often rely on particular language structures or key phrases. As we will show, ChatGPT does a remarkable job of generating test automation scripts, classes, and functions.

A Language for Low-Code: Cucumber

While ChatGPT can generate many languages and libraries, its sweet spot is its ability to generate Cucumber code. Cucumber is a testing framework that uses behavior-driven development. Scenarios are written in plain English, with some key terms like “Given,” “When,” and “Then” in a feature file. Then, in step definitions, these natural language phrases are associated with code.

Test maintenance is one of Cucumber’s benefits since Cucumber scenarios combine the natural language intent of a test with the automation code which implements it. This lets testers who aren’t as familiar with test code understand the linkage between test intent (in natural language) and test code. This leans into ChatGPT’s strength of generating natural language with associated source code.

ChatGPT and Cucumber Test Case Example

Now let’s see what ChatGPT can do.

As you can see in the following example, ChatGPT’s ability to generate code is quite impressive. With a simple prompt, it can generate the scenario and the step definitions used by Cucumber all at once. It even knows the different components Cucumber needs to run without being explicitly told.

Cucumber Example

The prompt is vague about what to test on the website, but ChatGPT still generates a script to test one of the main features of the Google website, its search functionality. In this example, it even correctly identifies the name of the search bar for Google, “q.”

A General and Manageable Script

While this is a good example of ChatGPT’s ability to write code in general, not all websites are as ubiquitous as Google. Its accuracy in predicting the identifier “q” does not indicate it will always be correct. And, from testing ChatGPT, we know that if it doesn’t know an accurate element locator, it will make one up.

Going through the generated code and finding and replacing all element locators is tedious, but the problem can be made much easier by separating the code about the page itself from the test cases. We do this by using a page object model. Using a page object model also allows us to update the code if element locators or the structure of the app changes between tests, improving the maintainability of the testing script.

In the following example using Cucumber in Python, we generate a generic test for a login page of a website and instruct ChatGPT to use a page object model and class variables for the element locators.

More specific ChatGPT cucumber python example

ChatGPT correctly generated the aptly named LoginPage object which represents the element locators as class variables and has methods for the common functionalities on login pages, such as entering one’s username. The step definitions, at the bottom of the example, use the methods in the LoginPage to interact with the website.

Now, using the page model object and the step definitions, it seems as though a user could either automate (with ChatGPT) or manually write scenarios to test the website and have working test code! But, will it actually work as-is?

Using ChatGPT to Change and Fix Code

If you look closely you can see all the input values for the test, such as the URL, username, and password, are all hard-coded into the step implementation. It is unlikely that the website you are testing will be “http://www.example.com/login” and that a working username and password combination will be “username” and “password” respectively. And these values do not need to be hard-coded, Cucumber can take variables from the feature file in the scenarios.

But what if you don’t know how this is formatted or don’t want to spend the time to update all the code? Well, you can ask ChatGPT to fix it for you.

ChatGPT code change request

By requesting ChatGPT to update the previously generated code, we can fix the problem. The updated step implementations read in the values that we requested instead of hard coding the most likely incorrect values. That is one of the remarkable features of ChatGPT: the conversational nature of the model allows you to tell the system exactly what you want to change in the code, and it is quite good at listening to and executing your requests.

ChatGPT and Sauce Labs

ChatGPT can generate test scripts that are compatible with Sauce Labs. Since running your script on Sauce Labs requires completely updating how the test is launched using Selenium, having ChatGPT write an accurate startup method could be vital to a user without much coding experience. By just adding “and run the test on Sauce Labs” to the prompt we can generate a script that includes an accurate method to start the test.

Start ChatGPT Test on Sauce Labs

The third code block includes the code to start testing using Sauce Labs. ChatGPT uses the correct URL, passes the necessary capabilities, and uses the correct driver method to start the test. While it arbitrarily decides which platform, browserName, and version to use, these are easily updated either manually or by telling ChatGPT to update these features. It is that easy to run a test generated by ChatGPT on Sauce Labs!

The Downsides of ChatGPT for Automated Testing

While ChatGPT has a lot of potential to be a low-code solution for automated testing, it still has issues. Users need some understanding of the app under test and the coding language and packages being used when generating code since the system often needs to be told to correct issues. ChatGPT does not run the code itself and therefore has no way to know if the code generated is truly runnable. One recurring issue is that the Xpaths or IDs need to be manually updated to accurately locate the correct element, as ChatGPT not only does not know these identifiers, it will fill in random identifiers so that the output code is as complete as possible.

Also, we found the model does not know what methods have been deprecated. It repeatedly used the “find_element_by_*” method to locate elements on the screen, even though that method is no longer functional. Simply asking ChatGPT to update the code with the correct method is a reliable way to correct the problem, but it does require the ability to recognize what the issue is.

Another problem with the above code is the step description does not always accurately reflect what is being tested in the script. For example, the step “I should see my username “{username}” is in the top right corner” uses the method “expected_conditions.presence_of_element_located(),” which only checks to make sure that the element is present on the DOM of the page, not that it is located in a particular region of the screen.

Finally, ChatGPT will assume some common page structures in its test code. The previous examples have both the username and password entered on the same page before submitting, but some websites have you first input the username, then click a button like “Next” and then input the password. A user can use ChatGPT’s dialogic nature to correct issues such as these, but, again, one must be able to recognize they are an issue to request an updated corrected version.

Conclusions

ChatGPT is a very powerful natural language model with enormous potential. What it can do is significant and it likely will lead the way in low-code testing solutions to many problems. It has the potential to do that for testing, but one still needs to have a reasonable understanding of both the language being used and the app under test to make use of ChatGPT as it is. But we shouldn’t underestimate its potential: ChatGPT is truly impressive code generation that wasn’t possible with previous natural language generation models.

If you want to learn more about ChatGPT and test automation, there are additional videos on the topic available, such as AI writes automation test code for any tool – OpenAI’s ChatGPT, AI writes Development Code/Unit/Integration/Mutant Tests – OpenAI’s ChatGPT and How ChatGPT Impacts Automation, Performance, and Security Testing.

Ready to start your automated testing journey? Sign up for a SauceLabs free trial today.

Author

Lena Reed, Senior Data Scientist, SauceLabs

Lena Reed is a Senior Data Scientist at Sauce Labs. She has a PhD in Computer Science from University of California, Santa Cruz, where she studied Natural Language Processing and Natural Language Generation. She likes learning and testing new technologies to see how prompt engineering can affect the results being generated. She also has experience generating and cleaning datasets, such as natural language inputs used to train models, and enjoys getting into the data to see what can be automatically improved to create the best training sets for models. During her time at Sauce Labs she has learned and worked on multiple areas of testing, such as Appium and model-based testing and is very interested in automatic test generation via large language models.

SauceLabs is an EXPO Gold partner at EuroSTAR 2023, join us in Antwerp

Filed Under: Test Automation, Uncategorized Tagged With: 2023, EuroSTAR Conference

Get your EuroSTAR 2023 ticket

April 4, 2023 by Fiona Nic Dhonnacha

Europe’s best software testing conference is just a few weeks away! The whole software testing community will gather together in Antwerp, 13-16 June. Don’t miss the 2023 experience at Antwerp Zoo: four days of infinite inspiration, networking, and career-defining moments.

Right now, you can book your tickets at the best price with our Early Bird offer – the more you bring, the more you save! Check out our group bundles and get your EuroSTAR 2023 ticket now.

get early bird tickets

We’ve got so much in store for you this year, from the valuable learnings, to the networking, to the impromptu chats at coffee and networking breaks. Here’s just some of what you can expect from the conference this year…

66 Expert Speakers

Industry leaders, innovators, hackers, problem solvers, gamers, team leaders, entrepreneurs, engineers and lots more will be sharing their knowledge with YOU! With in-depth tutorials, interactive track talks, and inspiring keynotes, it’s a melting pot of creativity and knowledge like you haven’t seen anywhere else.

Topics to cover all areas of testing

The 2023 conference theme is ‘Software Development is a Social Activity’, and we have a packed programme of top quality content for you. Topics include test management, team building, QA strategy, test automation, and lots more.

EuroSTAR 2022 Richard Edgren presenting a keynote session

1000+ testers across every spectrum of testing

Join QA engineers, test managers, DevOps engineers, test analysts & more from 50 countries, all coming together to spark new ideas. This is your space and your community – jump into conversations with people who are having the same problems as you, and spark ideas together.

300 global companies

Connect with fellow attendees in banking, government agencies, telecoms, manufacturing, automotive, security, tech and more – all coming to EuroSTAR 2023. Get new perspectives and fresh insights from all areas of testing and quality assurance, and benefit from the experiences of your peers.

Chats and games in the Huddle Community Area

Located in the heart of the EuroSTAR EXPO hall, Huddle will play host to a variety of activities and networking events, run by the Huddle team of volunteers. It’s aimed at connecting diverse and international delegates, and enabling knowledge sharing outside of the official talks. Take a break from the busy day and make some new friends!

47 leading test tools & service providers

This is Europe’s biggest testing EXPO – meet with the leading tools & services providers under one roof. Compare solutions or find new ways to improve your testing, check out cool demos, get 1:1 help, & win swag and prizes.

Don’t delay – book your ticket now and we’ll see you in Antwerp in June!

BOOK now

Filed Under: Uncategorized

EuroSTAR talks for…veteran testers

March 28, 2023 by Fiona Nic Dhonnacha

Next up from Programme Chair Kristoffer Nordström’s recommendation on which talks you should attend based on your role in testing – sessions for veteran testers! Check out the previous blog here.

Prefer to listen instead? Check out the audio version below:

As a veteran tester, I know it’s easy to feel like you’re stuck in a rut, unsure of where to go or what to do next. That’s why I wanted to write this next installment of the blog post series of recommended sessions and tutorials for various roles when attending Eurostar 2023. This time I’m writing for all of you hardened and seasoned veteran testers.

get tickets

EuroSTAR 2023 is a large conference, and the programme committee is proud of the diverse set of speakers from different backgrounds, topics, and experience that we have assembled. While I’m highlighting some talks, it’s important to note that it doesn’t mean that the other sessions are sub-par; these are just the ones that may be particularly relevant to the roles I’m writing for.

Inspiring Quality: Becoming an Exceptional Quality Leader

First up, we have my very good friend Fiona Charles’ tutorial, ‘Inspiring Quality: Becoming an Exceptional Quality Leader‘. When people ask me for advice on testers to listen to, Fiona is with few exceptions one of the first persons I bring up. If she’s ever giving a tutorial at a conference I tell people it’s a must for me to attend. And if you find me at the conference and ask, I’ll even let you in on the tutorial she gave many years ago that literally changed my life and career. In this tutorial, you’ll learn how to lead by example, inspire your team, and build a culture of quality within your organization. There will be questions asked such as “What does it take to be a leader?”, “Where do exceptional quality leaders focus their energies?”, “How can you get to be an exceptional quality leader?”, and “What does “leadership” mean in an Agile context?”  

An Ancient Science for Advanced Critical Thinking for Testers and Teams

Next, Lalitkumar Bhamare’s session, ‘An Ancient Science for Advanced Critical Thinking for Testers and Teams‘, will teach you how to apply ancient Indian philosophical principles to your critical thinking in testing. By learning how to question assumptions, recognize biases, and think outside the box, you’ll be able to approach your customers and ask critical questions on what they expect and need to be delivered. Lalit will help and guide you into the Indian philosophical school of ‘Erotetics’. Don’t miss this chance to attend a session given by a long standing member of our community and the long time chief editor for “Tea-time with testers”.

That Which Isn’t Good for the Hive Isn’t Good for the Bee

Sanne Visser’s session, ‘That Which Isn’t Good for the Hive Isn’t Good for the Bee‘, will explore how to build a more collaborative and inclusive testing environment. By recognizing and addressing power imbalances, you’ll see the problems attached to ‘addicted to gold-stickers’ in your work and organization. Sanne asks questions such as “Someone has to ‘run the hive’ and tell us what to do. Don’t they?”, “If competition disappears, won’t people stop buzzing around?”, and “When I don’t have clear goals, nectar to collect. How will I tell if I’m successful?”. Sanne is a delight to listen to and one of the founding members of the podcast “9 out of 10 Testers”.

Hope is Not a Plan

Fredrik Håård’s session ‘Hope is Not a Plan‘ addresses the need for leaders and managers to recognize the dangers of optimistic thinking in an organization and to focus on solutions rather than problems. In roles such as tester, security analyst, or similar, it’s easy to come across as a naysayer and doomsday prophet. As organizations, we have a tendency to remain optimistic about outcomes and stick our heads in the sand until it’s too late. In this talk, Fredrik will discuss how to communicate risks and allow for informed decisions before your organization becomes the next headline in international news for leaked user credentials and credit card information. So far I’ve written four installments in this series for different roles, and this is the third time I’m mentioning Fredrik’s session, why? Well, because Fredriks insights and way of presenting is simply a session you can not miss. Fredrik always delivers his talk with a touch of mirth and humor and draws extensively from his experience working as an international consultant across three continents (and still counting).

We’re Off to Rehab: Communicating in an Isolated, Decoupled World

Finally, Anne-Marie Charrett’s session, ‘We’re Off to Rehab: Communicating in an Isolated, Decoupled World‘, will explore how to communicate effectively in a distributed testing environment. A lot of us have gone through the last years experiencing how it is to work remote. Some love it, myself included, while others have found challenges and perhaps not the rose garden they were hoping for previously. Anne-Marie who won the testing excellence award at EuroSTAR is a beloved member of our community and in this session learn how to build relationships, establish trust, and leverage technology in the context of remote working. Go see Anne-Marie as it is not every day that this tester comes to Europe from down under where she lives in Australia.

In summary, attending the Eurostar 2023 software testing conference is a great way to break out of your rut as a veteran tester. Whether you’re looking to become an exceptional quality leader, improve your critical thinking skills, build a more collaborative testing environment, create a comprehensive testing strategy, or communicate more effectively in a distributed team, there’s a tutorial or session for you. So why wait? Register now and take the first step towards your next great adventure!

book now

Right now, you save 10% on EuroSTAR tickets with our Early Bird offer – and over 35% if you bring a team. Secure your tickets, and we’ll see you in Antwerp in June!

Filed Under: Uncategorized

  • « Previous Page
  • Page 1
  • Page 2
  • Page 3
  • Page 4
  • Page 5
  • …
  • Page 8
  • Next Page »
  • Code of Conduct
  • Privacy Policy
  • T&C
  • Media Partners
  • Contact Us