• Skip to main content

EuroSTAR Conference

Europe's Best Software Testing Conference

  • Programme
    • 2024 Programme Committee
    • 2023 Programme
    • 2023 Speakers
  • Attend
    • Venue & Hotels
    • Bring your Team
    • Testimonials
    • Why Attend
    • EuroSTAR Awards
    • Academic Discount
  • Partner
    • Download our EXPO Brochure
    • Partner Opportunities
    • Partner Testimonials
  • About
    • About EuroSTAR
    • FAQ
    • EuroSTAR Timeline
    • EuroSTAR Blog
    • Supporting Organisations
    • Contact Us
  • Book Now

EuroSTAR Conference

3 methods of test book testing

July 4, 2022 by Fiona Nic Dhonnacha

Thanks to UNGUESS for providing us with this blog post.

How do you choose the right methods of test book testing? What are the possible methods? Let’s explore all the alternatives.

When it comes to functional testing, there are multiple options to choose from. Which option is right for your business will depend on the type of software you want to test, your target audience, and your overall testing goals. With that in mind, let’s explore the three possible methods of test book testing.

What is a test book?

Functional testing validates software against the functional requirements or specifications. Each company has unique goals they want to achieve with their software, and functional testing is an excellent way to determine whether you’re meeting those goals. A test book is simply the list of test cases that will be executed during functional testing. The test cases are designed with your specific software in mind. For example, if shopping giant Amazon wanted to test the functionality of its search function, they (or an external company on its behalf) would develop the test cases from the ground up. The test cases would be specific to the website’s design, the available filters, the shopping cart process, and so on.

The key benefit of functional testing is that it helps you close the gap between how your software should work and how it actually works. Sometimes software seems logical to the people who create it but creates confusion for end-users. And sometimes, the software simply doesn’t work as intended and needs to be reworked.

The 3 methods of test book testing

1.  Crowd testing

Crowd testing, sometimes called crowdsourced testing, tests usability, functionality and user-friendliness under real-world conditions.

Crowdsourced testing is a manual approach to software testing. Testers are real professionals who get involved only for the testing process, meaning they get paid by project and performance instead of being full-time employees.

 The Benefits of Crowd Testing

  • Real-world conditions: It allows you to test the software’s functionality using real end-users in real-world conditions. Due to the large number of users, you can test functionality at any time of any day.
  • A wealth of testing data: Crowd testers use their real devices and operate different operating systems, browsers, and other native software. This makes the testing data rich, both in terms of diversity of feedback and high coverage.
  • Fast, unbiased reporting: Testers give feedback on the go, which means you get a constant stream of valuable feedback. Additionally, while all competent testers must be impartial, crowd testers have a distinct advantage in this area. They have no stake in the success of your software, so their feedback is guaranteed to be honest. Honest feedback is critical when creating highly functional and user-friendly apps.
  • Target demographics: You can source crowd testers from a vast range of demographic criteria, including age, gender, profession, income level, hobbies, and more. Crowd testing is an excellent way to see how well your target audience perceives your software. This aspect is extremely important in UX Testing.

 In UNGUESS, we have created a vast community of testers (over 20,000 in Italy, our main market, and we can reach over 250,000 worldwide). Since they are highly profiled, we can select professional testers and/ or real users of your product. Our Customer Success Managers (CSM) write the test cases for the test book specifically aimed at testing our clients’ software. Testers submit the bugs they find and the CSM approves them only if they are unique and in scope. Our clients’ Dev team can receive an export of all the bugs, or can use a bug tracking tool integration.

2.   Test automation

In automated functional testing, a computer runs the functional testing for you using tools and scripts. Typically, the software is categorised into distinct areas of functionality, and a script is designed to test the functionality in a series of steps. For each step, pre-selected elements of the UI are interacted with. For example, the automated test software might fill in a text box, select a dropdown option, or push a button. After running the automated test, you can check the results to compare them with what should have happened.

At AppQuality, for example, we have a range of tools we can leverage for our clients’ test automation, depending on their specific software specifications. As for specifics, we often use Robot Framework – a robust open-source automation framework with a rich ecosystem of libraries and tools (we have talked about it here) and Seleniumlibrary (or other libraries). Again, this is just an example as we use different tools and frameworks depending on our clients’ necessities.

The benefits of automated functional testing

  • 24/7: A computer runs automated testing, so you’re not restricted by human testers who need to take breaks, sleep, or only work set hours.
  • Faster: Without having to rely on people (unless writing the code for the test Automation is necessary), you can execute your tests much quicker. On average, automated testing is 70% faster than manual testing—faster testing results in an early time to market for your application. Warning: with Test Automation you can only find what you are looking for. Nothing out of your path can be found.
  • Improves accuracy: When you eliminate humans, you also eliminate human error. Any errors can reduce the accuracy and reliability of your results.
  • Cost-efficient: Automated testing is often significantly cheaper than manual testing.
  • Ensures consistency: Automated testing can be repeated as you make improvements and tweaks to your software. This promotes consistency across every stage of your functional testing, making it much easier to compare the results of multiple tests objectively.

3.   FTE (Full Time Equivalent)

FTE (Full-Time Equivalent) or Third-party Test Factories can mean one of two things in the testing world:

  • The company hires a top tester who writes the entire test book and then executes the tests on the application, website, or other software.
  • The company writes the test book, and the tester runs the test cases for them.

FTE offers a high level of flexibility and allows companies to engage with the testing process in the way that best suits your company. Many companies don’t have the time or resources (skilled testers) to develop the test cases themselves and benefit from the help of an outside party for a short amount of time. Other companies have a clear idea of the tests they want to run but want an impartial expert tester to execute the test cases.

To learn more about FTE or Third-party Test Factories, see how it works and the possible results, read this article and the case study of buddybank.

The benefits of third-party test factories in functional testing

  • Objectivity: Testers are fair and unbiased in reporting their results. This can be difficult to achieve with in-house testing because employees have a vested interest in the software – they might overlook negative aspects of the functionality to appease senior management.
  • Specialisation: Testers have formal training and certifications in testing related disciplines. They have a deep knowledge of testing methodologies, best practices, and appropriate tools. They bring a level of professionalism to the table that results in highly accurate, fast, and reliable results.
  • Risk reduction: Introducing a third party can help reduce the risks associated with functional software testing. Testers can spot blind spots or errors in your software that you might otherwise miss.
     

Which method of test book testing should you use? 

It is impossible to answer that question out of the blue. It is crucial to understand the context, the goals, the available resources, tools and stage of development.

UNGUESS offers a Crowd testing platform (together with tens of thousands of testers), Test Automation technologies and knowledge and even the Body Rental approach.

If you want to know which method is the most suitable for your product, contact us and schedule a short call with our experts.

UNGUESS is an exhibitor at EuroSTAR 2022.

Filed Under: EuroSTAR Conference, Software Testing Tagged With: 2022

Speak on the EuroSTAR stage

June 29, 2022 by Fiona Nic Dhonnacha

The EuroSTAR Call for Speakers is open for our 2023 conference at Antwerp Zoo, Belgium. Nestled in the heart of Antwerp conservation zoo and embedded in nature, for 2023 we are going beyond conventional space – think sprawling architecture, rooms gleaming with natural sunlight, magnificent nature, and majestic animals.

If you’re keen to speak on the EuroSTAR stage and share your software testing knowledge and expertise in this incredible space, we’d love to hear from you! Our community is all about coming together and sharing knowledge – so whether you’re a novice or seasoned tester, this is your opportunity to share your real-life experiences, and help others learn and grow.

see call for speakers

2023 conference theme

This year’s theme is ‘Software Development is a Social Activity‘, in homage to Jerry Weinberg and his legacy. Often, the hard part of working in software isn’t the technology and the tools – the actual challenge is all the social interactions that go into the work in a team, company, or between companies and their customers.

People work together in teams, and our day-to-day lives are filled with social interactions and communication. How do we interact with the software? What are the tests performed communicating back to us?  How does a human communicate with a machine, and then back to another human? The tools and languages we use influence our work and the shape of the organizations, the interactions, and the communications.

Speak on the EuroSTAR stage - a packed auditorium in a keynote talk at EuroSTAR 2022

Below are some suggestions on topics that are welcome. However, tools and technology submissions are welcome too – communication and interaction take many shapes and forms.

● Communication
● Software testing
● Test automation and its role in the work we do
● Team building
● Creating healthy organisations
● The pandemic, increased remote work, and its impact on how we work with it
● Communities
● Group dynamics
● Leadership & Management
● Recruiting good testers and developers
● Methods of collaboration
● Accessibility testing
● Internationalisation and localisation testing
● Diversity, equality, and inclusion

We’re so excited to see everyone at Antwerp Zoo, and we’re blown away by the inspirational setting – did you know that all profits made by the convention centre are invested into research and nature conservation worldwide?

If you’re feeling inspired, check out the full submission details, and start thinking about the story you want to share. We look forward to getting your submission, and seeing you speak on the EuroSTAR stage next June.

start submission

Filed Under: EuroSTAR Conference Tagged With: 2023, Call for submissions

How to test mobile apps in the cloud

June 18, 2022 by Fiona Nic Dhonnacha

Thanks to Mike Wagner from Keysight for providing us with this blog post.

In 1973, Martin Cooper did something that no one had ever done before. He placed a call on the world’s first commercial cell phone. The Motorola DynaTAC 8000X might have been slightly smaller than a house phone with limited functionality and poor call quality by today’s standards, but this was a significant step in developing mobile device technology.

Mobile devices today are wildly different. They have multiple screen sizes, resolutions, and functions, with capabilities expanded by millions of apps to meet our specific needs at work and home. Brilliantly convenient for the user, but with different models, operating systems (OS), and manufacturers, ensuring your app works across each of these variables is challenging. Get it wrong, and your app goes in the trash can. In the US alone, 43% of apps are uninstalled after 30 days, according to Statista.

Testing mobile applications is the only way to ensure functionality before it hits the end-user. However, you need to execute thousands of test cases because there is no guarantee that an app performing on an Android device will work without issues on the latest Apple iPad or Samsung handset. Even simple problems with screen aspect ratios or orientation can occur on the same OS but different device models. So, what’s the answer?

Cloud-based mobile app testing

Cloud-based device farms optimize mobile app testing by executing test cases in virtual software environments. Mobile conditions are replicated using simulators, emulators, or on actual devices hosted in the cloud.

Compared to physical on-premise device farms, cloud-based device farms are easier to maintain and allow multi-site project teams to test applications on numerous devices in various real-user situations without setting up a physical lab. Competition is fierce, with thousands of apps released daily around the world. Testing app performance and functionality are essential to prevent bugs from entering production. If bugs are missed, costly delays occur, damaging the user experience and losing customers.

Benefits of cloud-based mobile app testing

One obvious benefit to cloud-based mobile app testing is that it is much easier to maintain than physical device farms, but there are a few more advantages that are at hand

  1. Cost-effective

Executing hundreds of different test scenarios across multiple OS, devices, and models is hugely expensive when acquiring and managing physical hardware yourself. Cloud-hosted testing platforms, such as Sauce Labs, offer monthly charges for a range of old and new devices, so you don’t need to constantly update yours when new models are released.

  1. Better availability

When your testing environment is hosted in the cloud, a massive number of devices, platforms, and OS are available via simulators and emulators. Real devices are also available to instantly access the latest models and features to execute accurate testing.

  1. Easily accessible

Quality Assurance and test engineers can access a device farm and execute tests from anywhere, which is helpful in distributed teams as it supports collaboration. By accessing mobile device farms in the cloud and from various geolocations, testing can occur 24/7, accelerating time to market.

  1. Enhance collaboration

Cloud-based device farms can be integrated with test management tools to accelerate feedback loops and software delivery. Various people are involved in software development, so good communication is critical to identify if they are any issues with a release so fixes can be made as quickly as possible.

  1. Increased productivity

Utilizing a device farm in the cloud allows parallel testing of various devices, rather than testing one after another, which dramatically speeds up test cycles. Testing is often a bottleneck to software development, so being able to execute multiple test scenarios unique to your end-users accelerates production deployments.

Mobile app testing in the cloud with Keysight’s Eggplant software test automation

Manually testing apps on multiple devices can be unreliable, costly, and slows down release cycles. With Keysight’s Eggplant test automation solution, you can accelerate the end-to-end testing of every user journey in your app across multiple mobile device types.

Eggplant’s flexibility allows integrations with mobile device farm providers, so your teams can access numerous devices and execute test cases specific to their end-users needs.

Using Eggplant test automation software, testers can expand test coverage incrementally to parts of an application with discovered issues via intelligent automated exploratory testing. The artificial intelligence and machine learning algorithms proactively explore areas close to these defects, as the probability of other bugs is high. Missed defects are costly to fix, especially when discovered in production.

Eggplant’s ability to increase test coverage helps reduce the chance of a buggy user experience, which can cause brand damage, dissatisfied end-users, and negatively impact an organization’s bottom line.

To discover how to optimize your app testing and accelerate releases, read our guide on mobile device testing in the cloud.

Keysight is a Platinum Partner at EuroSTAR 2022.

Author

How to test mobile apps in the cloud - Mike Wagner

Mike Wagner

Mike Wager is a Product Marketing Manager at Eggplant, a Keysight Technologies company. Over the last decade, he has brought product ideas and digital services to life for engineering technologies, SaaS applications, and software development. In his current role, Mike communicates the unique benefits of AI-driven test automation for enterprise applications, retail services, and mobile testing.

Filed Under: EuroSTAR Conference, Software Testing, Uncategorized

Software testing platforms – the X factor in QA teams’ advancement

June 13, 2022 by Fiona Nic Dhonnacha

Thanks to Armando Wirshing, Director of Product Marketing at Katalon, for providing us with this blog post:

A common dilemma in software development is how the work of quality engineers, or testers, is often considered as simple as authoring automation and running it repeatedly. This notion is outdated and harmful to implementing a sustainable, end-to-end software quality process.

Establishing a cross-functional approach for QA requires capabilities that mirror and complement DevOps methodologies. Perhaps this is why over the past few years, some have coined the term DevTestOps. In reality, it’s more akin to how security takes part within the DevOps or software development pipeline and stands apart to provide another layer of independent validation to the process. Similarly, testing teams must cohesively support software development pipelines and provide independent validation and oversight.

Development teams use tools such as GitLab, Jira, etc., to plan, organize, and enable their processes. Operations teams leverage Splunk, ELK, AppDynamics, Datadog, etc., to monitor, analyze, and report on operations. Quality teams, unfortunately, have historically not had the benefits of a quality-focused platform to perform both core tasks and automate reporting and analysis. They often have to employ DIY tactics to their end-to-end quality platform, thereby reducing the overall value of their tool due to the amount of technical debt created by the DIY approach.

Experts know that CI/CD or DevOps is unattainable without test automation. Yet many teams struggle to implement true automated testing beyond unit testing within limited frameworks, leaving the vast majority of testing to continue manually.

Test automation, in general, is challenging, and test planning, authoring, maintenance, management, execution, and reporting/analysis all present individual challenges as quality teams level up in their maturity models. Fortunately, recent advances in testing technology are addressing many of those factors. However, teams must find the most practical tool or platform that fits the budget constraints and integrates seamlessly into the existing tools and services in the CI/CD or DevOps pipeline.

Quality organizations can learn from the DevOps industry usage of platforms that have key functional and reporting capabilities while enabling collaboration and integration into the software development pipeline. The following presents the key features and value of a modern test automation platform that delivers speed, scale, and value to QA teams and the business.ng

Essential features of a test automation platform

1. Automation and authoring

Ease of use and adoption

Creating a solid foundation of automated tests will ensure scalability and quality as the product expands. Two critical criteria to achieve this are effortless operation and flexibility in test scripting Amode. As teams typically consist o testers with varied experience in coding, test automation platforms should include advanced script creation functions for testers with strong coding skills and easy-to-learn low-code scripting modes. With every team member involved, the team’s overall capabilities and collaboration will significantly improve.

Low Maintenance and robust automation

The additional rework of some automated testing tools adds to the overall cost of automation maintenance. The platform should include “smart healing” capabilities that increase text execution quality and overall testing efficacy. These capabilities help eliminate false positives and reduce overall testing flakiness.

2. Orchestration and optimization

Centralized planning and self-service scheduling

A centralized testing calendar is necessary to smoothly organize and plan testing activities across all platforms and projects within an organization. For this reason, the tool should come with a built-in UI that gives all quality stakeholders, whether quality teams, software, or product teams, ownership of scheduling, scaling, and planning.

Operational reporting

Options to reduce testing time by running tests in parallel or adjusting test scheduling to allow emergency release is possible with effective reporting and visibility. A centralized command and control view of the process should enable such flexibility.

Software testing platforms  - visibility and understanding across testing and DevOps

3. Insights

Visibility and understanding across testing and DevOps

Effective quality processes start with collaboration across stakeholder teams. By incorporating intelligent and relevant insights into standard reports, a shared understanding of quality status is set from the organizational level to the individual project or build

The quality platform should integrate automated real-time reporting that generates insights on data from both quality activities and CI/CD pipelines. Data from Git repositories, test results, and project management tools can be centralized and analyzed for flakiness, requirements coverage, and release readiness across all projects and releases.

Actionable insights from AI

The industry is rapidly adopting AI reporting to gain more advanced insights into code coverage, test prioritization, release readiness, etc. Actionable insights generated by a proficient AI-infused reporting tool reduce analysis across data points and enable rapid decision-making. With AI-based analytics and insights, every decision that impacts the quality of your software will be more informed and decisive.

4. Scale

Multiple options for test execution

Having both on-premises and cloud environments is a must-have for organizations to be agile and address the changing demands of software delivery. Having multiple options for execution, whether on-prem or in the cloud with on-demand SaaS solutions, means your critical software deliveries will never be constrained by test execution capacity. Contrarily, using a tester’s environment to create and execute tests limits your ability to scale.

User platform coverage

An equally important capability alongside multiple environments is the ability to support the plethora of browsers, OSs, and available devices that end users leverage. This is further complicated by the variety of versions of each browser or platform. Not only do your IT or DevOps engineers need to keep up with new releases, but also some number of “N – x” versions that continue to be used.

Ability to support huge upsurge of workload

Large enterprises benefit from having a platform that allows instant scalability and parallel testing to maintain matching capacity without disrupting the release timelines of the business. In the case of a sudden upsurge in test capacity demand, a cloud-based QA platform comes in handy as it can satisfy rapid increases in demand for test execution

Unified world-class support

Even a minor issue can slow down your entire workflow and negatively impact the business. Having a single platform with a global community and 24/7 professional support from skilled staff can make a world of difference and yield long-term business success.

Author

Armando Wirshing, Director of Product Marketing at Katalon, Inc

Armando has extensive experience in the technology field, ranging from software supply chain, CI/CD, ITSM, Cloud Hybrid, to DevOps, Agile, Automation. His career has spanned from Mainframes to Cloud, Development to Operations, Delivery to Sales. He is dedicated to empowering teams and businesses to overcome challenges and in turn celebrate success

Filed Under: EuroSTAR Conference Tagged With: software testing tools

The future is here – is QA ready for it?

June 3, 2022 by Fiona Nic Dhonnacha

Thanks to Jani Haapala, DevOps Architect at Gofore, for providing us with this blog post.

William Gibson famously said, “the future is already here – it’s just not evenly distributed”. In software development, we constantly talk about future trends and future methods that are coming one day, but so often fail to see that in most cases, the future is already here – but maybe not yet in your context.

The problem with QA and testing is that it always seems to play catchup with these trends and methods. QA is mostly reacting and adapting to challenges that the future is bringing to us, rather than proactively creating or defining them.

So, if there’s already a recognized need, how come we’re not answering it? In my opinion, it’s because, in some cases, those of us in QA are still living in the past.

Living in the past

Some of us are still living in the past – although we might not realize it. In these situations, I see that QA as a function fails to provide the value that it should. I have found out that there are two major contexts where QA is still living in the past:

QA is a step: in many cases companies have adopted agile, and are driving fast toward DevOps. Some companies are already doing DevOps in some form. But QA still seems to be mostly a manual step on these fast phased cycles. Developers are having CI/CD pipelines; build and unit testing is happening automatically but when it comes to functional testing, end to end testing, and system testing, it often tends to be something that “needs to be asked from QA people”. This separates QA to its own manual silo, and creates unnecessary handovers. Instead of this, QA people should drive toward automated testing that is fully integrated to the CI/CD pipelines that development is using. Test automation is not a tool that testers use; it’s an automatically triggered set of tests by commit of code.

QA is about running tests: QA is often still fixated on ‘creating tests’ and ‘running tests’, and it seems that the only function of QA is to be the executor of tests. This leads to a situation – with test automation in particular – where there’s thousands of test cases that might pass and might not, or nobody cares. Tests are becoming mere ‘experiments’ that fade over time. So, QA and testing should be all about bringing valuable feedback to people that need it. They should focus on thinking about the most valuable thing that we can get from the system, and how to present that to those who need it. And remember, most valuable things change, and so does the importance of any given tests.

The future that we live today

So, what are the latest trends that QA should take note of sooner rather than later? What’s already on our doorstep, and needs immediate attention? Let’s hop on the train early, figure out what kind of feedback is needed, and learn how to provide that. Some of the future scenarios that we are already living include:

The ‘XaaS’ movement: in the past there was a huge ‘XaaS’ moment – ‘Something as a Service’ shaped our daily lives. Rather than creating everything themselves, development teams started to consume services. Cloud services, version controlling services, deployment services, and so on. After that, we found out that these services and platforms could be controlled programmatically. So, the new ‘XaC’ boom started. ‘XaC’ means ‘Something as Code’, and that means that everything is done by creating a code file for it. Examples of these are Infrastructure as Code, Configuration as Code, Storage as Code, and Detection as Code. For this modern need, QA also needs the ability to test, verify, and provide feedback on how these code files are working

QaSec ownership: if QA is considered to be a silo, then I have a story about InfoSec! InfoSec is often an even more separate action than QA. InfoSec is mostly done completely separately, and if InfoSec tries to suggest something, it’s only considered if it doesn’t affect the daily work, or make it more complicated. In the future, InfoSec should follow in the footsteps of QA – and be part of QA. Having QA and Sec combined to one QaSec needs to have real impact to development. Too often agile managers are cut down so much that nobody is in charge of overall quality or security. That’s why QaSec needs to be owned and recognized as one of the key stakeholders of development.

New architectures: The world is truly evolving. People who were running systems of physical machines or virtual machines are now running the same services on clouds in the form of service meshes and microservice architectures; or even without any maintained resources on just lambda functions. QA needs to be very aware of both today and the future architectures to be able to give proper feedback on these. QA can’t also test those if they are stuck on ‘just using’ something that was set up for them; they need to be actively involved in creating and improving these to define safe production environments, and get production-like environments for their needs.

Get ahead of the future

So, if we want to be proactive instead of reactive towards new trends and ensure our validity, what could we do? I don’t want to start doing too much guesswork here, but some future “safe bets” that will eventually take over the world are:

New methods: new methods are coming, and those methods need the capability to adopt. QA might be done on different stages or in different contexts. For example, one of the promising ‘new’ methods is Google’s Site Reliability Engineering (SRE): that’s Google’s take on DevOps, and focusing on reliability. SRE is an extremely useful practice when companies are having more large scale software production. There’s likely more than one, and they can grow rapidly in popularity.

RPA: Robotic Process Automation is something that we already see companies experimenting with. This enables companies to automatically connect legacy systems together to form long, fully automated business processes. RPA used to be available only from vendor companies with a high price tag, but more and more RPA is now done with pure open-source software. In many cases, QA can repurpose their test automation knowledge and tooling to start implanting RPA solutions also.

Artificial Intelligence: If RPA is the first step into full process automation, AI is mostly the target. Even though AI sounds both cool and scary, the reality is that we are far away from general AI that works just like a human. In QA’s scope the near future is very interesting when it comes to testing early AI applications, or using AI as part of the testing. People’s first reactions is to point out that AI can’t do the same things as humans, or make decisions – but that’s not the point (at least right now). The point is to help people make the right decisions with a faster lead time.

Get into the high performers group

If you recognize yourself embracing the future, please, be loud about it! Keep presenting at conferences, and spread the word on how to get ahead of the game, and help your fellow QA people join the high performers group. But if you see yourself in the past or present group, please start taking advantage of all of this, and begin the journey to elevate yourself to the high performers group. It just takes a willingness to learn, and some time, to get you started on your journey.

Author

Jani Haapala, DevOps Architect at Gofore

 Jani Haapala, DevOps Architect at Gofore

Jani works as a DevOps architect at Gofore. He is passionate about measurement, feedback, and continuous automated quality feedback loops. Jani’s journey started from manual testing and has evolved to full-scale software development automation. Jani thinks that automation can help everybody and increase value in anything.

Filed Under: EuroSTAR Conference, Quality Assurance Tagged With: software testing conference

4 trends in DevOps testing practices to watch

April 21, 2022 by Fiona Nic Dhonnacha

This blog post was provided by Izzy Anzeri, co-founder of mabl.

Quality assurance and software testing have evolved significantly in the past two years, and concepts such as quality engineering and shift-left are becoming more commonplace. To fully understand these winds of change, due in large part to the pandemic, mabl conducted its third annual Testing in DevOps Report. The idea was to ascertain how DevOps adoption continues to impact CI/CD adoption, development speed, and quality practices in the past year. Over 600 quality assurance and software development professionals found that software professionals are very interested in seeing DevOps mature, and while there is significant progress, the study also found – not surprisingly – there are still some challenges in the road ahead. 

There’s still work to fully adopt DevOps

Overall, the mabl study found that only 11% of teams have fully adopted DevOps practices. Another 76% of teams are starting, or already on the journey. What’s holding back that adoption? Over half of teams cited slow process, inability to change, and budget as the primary reasons for the hold-up.

As software teams continue making progress against DevOps adoption, there are plenty of opportunities for quality teams to get involved. The study found that of the fully DevOps teams, only 50% indicated they have a culture of quality where testing is performed early and often in the development lifecycle. Testers should search for opportunities to integrate quality activities into sprints and get more team members involved.

take the 2022 survey

Test coverage equals a better customer experience

In the past, DevOps success has typically been evaluated based on how quickly new code can be shipped into production, Since the pandemic, this attitude had to change as DevOps and QA teams are forced to take a more customer-centric view. 

Today, the customer experience is an ever more critical factor in business success, as life has become primarily a digital-first scenario. How can QA contribute to creating a better customer experience? The report stipulated that 80% of teams with high test coverage reported high customer happiness, with only 3% reporting low customer happiness. Compare this to teams that said they had low test coverage, with a nearly 50/50 split between high and low happiness customers. By emphasizing quality engineering metrics like test coverage, QE leaders will be better prepared to showcase the business value of testing and tackle the cultural shifts that continue to inhibit DevOps maturity.

Build a culture of quality

One of the main tenets of DevOps is to ensure the software development process is a team sport – ridding the concept of silos that can stymie both inclusion and innovation. The mabl study found that one in three quality practitioners report that testing involves multiple team members.

Quality teams should look at metrics such as bugs in production and MTTR, to assess how the broader development team is collaborating on quality. For a baseline, fully DevOps teams were able to fix bugs within 8 hours, compared to peers on the adoption journey where it may take more than two days to address. 

In fact, research has shown that an equitable work situation actually results in lower stress for the entire development organization, often resulting in DevOps teams’ ability to catch bugs early in the development process.

Continuous testing becomes a requirement

Continuous testing (CT) has taken off with the adoption of Agile and DevOps practices. CT encourages software development teams to break away from the idea that testing is a phase of development right before production. Rather, testing activities are integrated throughout the software development lifecycle.

According to the report, 33% of teams are actively transitioning to CI, while 46% of teams have fully adopted CI, and 36% of teams have fully adopted CD. As pipelines become more automated, QE teams won’t be able to keep up with velocity. To further integrate testing into development, QE teams are testing processes with select developers to coach them on proper testing practices in the coding or pull request phases of development. As confidence in these processes grows, teams are automating tests at that phase to open up the testing bottleneck.

take survey

It’s an exciting time to work in quality. With processes and trends constantly changing, there are opportunities to integrate testing further into development, contribute to velocity, and optimize customer experiences. Want to see how DevOps is impacting testers in 2022? Take the Annual Testing in DevOps Survey now.

Mabl is a Gold Partner at EuroSTAR 2022. It’s our first in-person event in 2 years – and it’s going to be a massive celebration of testing! Learn from 70 testing experts, and connect with your peers at Europe’s best testing event. Get your ticket now – book by April 22nd and save 10% on individual tickets; up to 35% on group bundles. See you in Copenhagen.

Author

Izzy Anzeri

Izzy Azeri Izzy is the co-founder of mabl, the leading test automation tool built for DevOps. He co-founded mabl in 2017 with Dan Belcher after realizing that existing software testing tools were unable to support the needs of high-velocity development teams. Mabl is now the leading intelligent test automation tool for quality-centric brands like Ritual, Charles Schwab, and Arch Insurance. The company has raised over $36 million from prominent VC firms such as GV (formerly Google Ventures), Amplify Partners, CRV, and Presidio Ventures. Building mabl is at the heart of Izzy’s passion for creating solutions that solve essential problems in software innovation. His previous startup was Stackdriver, a cloud-based infrastructure monitoring company that was acquired by Google just 21 months after raising its Series A funding. Izzy was then a Senior Product Manager in Google Cloud for several years where he led pricing strategy across all cloud products.

Filed Under: EuroSTAR Conference

Be A EuroSTAR Volunteer

March 29, 2022 by Ronan Healy

The EuroSTAR Volunteer programme is back. Along with out live event, we are delighted to welcome back our volunteers for the EuroSTAR 2022 Conference.

Taking place from 7-10 June in Copenhagen, the EuroSTAR Conference will take place in the Bella Centre. With over 70 sessions, exhibition space and social events taking place, there is lots to see and be part of at the event. So let’s take a look at the roles you can be part of at the event.

Huddle

The Huddle is the home of the “corridor talks” at EuroSTAR. Located in the heart of the Exhibition space, the Huddle is home to lots of events across the Conference.

A scene from the Huddle in 2019

At the Huddle you will find activities like Couch Sessions, 1st Timers welcome, Games and Giveaways, the famous Soapbox sessions and lots more. You should volunteer for this area if you like engaging with people, sharing knowledge and sharing experiences.

Test Lab

The Test Lab is the home of Testing at the Conference. Growing stronger after 14 years in existence, the Test Lab is led by our Lab facilitators each year who set the challenges for delegates.

Test Lab at EuroSTAR 2019

If you like the challenge of figuring out puzzles and testing robots, and other testing challenges, then this is a place for you.

Test Clinic

The Doctors of the Test Clinic are at the Conference to solve the problems of all delegates at the Conference. They are experts in their area of testing helping testers with the issues that they have.

Test Clinic doctor at EuroSTAR 2019 helping an attendee

You should apply for the Test Clinic if you are very knowledgeable in testing, or a particular area of testing. If you love helping others out and solving problems for testers, then this role is for you.

Cadets

The EuroSTAR Cadets get involved in every aspect of the Conference. They are in the exhibition space, directing delegates to rooms, helping out in the tutorials or being part of the EuroSTAR team.

Anyone who would like to apply for the Cadets, should be interested in being part of the whole Conference experience and doesn’t mind adopting to different roles throughout the day.

As part of the Volunteer roles, you will have access to the event with the opportunity to attend sessions. Catering will be provided throughout the week. You will get to be part of a great team and experience the Conference from a new perspective.

You can apply to volunteer at EuroSTAR 2022 here.

Filed Under: EuroSTAR Conference Tagged With: 2022, EuroSTAR Conference

How to automate functional UI testing using the I-A-C pattern

March 28, 2022 by Fiona Nic Dhonnacha

This article was provided by Alfonso Nocella, Co-founder and Sr. Software Engineer at Maveryx.

Automated Functional UI Testing

We generally used to consider Functional UI Testing the most critical type of testing because, first of all, an application must do WHAT it is supposed to do. And indeed, it verifies that an application works as expected through its user interface.

Or, if you prefer, according to Glenford Myers, Functional UI testing means to verify the software through its UI with the intent of finding errors (i.e., “find discrepancies” between the program and its specification).

Given these points, Functional UI testing verifies, from the end user’s perspective (through the user interface), that the software behaves according to its functional requirements specification without knowing the internal logic and implementation (black-box). In simple words, Functional UI testing verifies an application in the same way an end-user would.

Also, Functional UI tests are effective because they go “end-to-end” from the UI layer. Indeed, end-to-end means testing all the different parts of an application from the beginning (the user interface) to end (the underlying services, the database connectivity, etc.).

From the automation point of view, Functional UI tests imitate users’ actions such as clicking on a control, entering text, scrolling a page, and so on by also comparing the actual outputs with the expected outcomes.
With this in mind, in automated Functional UI scripts, working with user interface elements, a tester should always use this pattern:

  1. IDENTIFY: locating the elements to use
  2. ACT: performing the requested (test) actions
  3. CHECK: comparing expected results vs. actual results

In fact, this pattern recurs continuously in an automated test script.
This paper will explain how to write automated Functional UI tests using this pattern.

The I-A-C pattern

In an automated Functional UI script, each test “step” consists of two parts: “action” and “checkpoint.” An “action” is an interaction between the test tool and the UI object-to-test. An example of action is clicking a button or entering a text into a text field.
Besides, an action consists of a “target”, a UI object-to-test, and a “command”, a test action to perform (e.g., “click”, “type”, …). A command may have inputs (e.g., type “some text”) or not (e.g., “click”).

A “checkpoint” compares the actual outcome from the application under test and the expected one. For example, the predicted result could be that typed text is in the text field.

Hence, writing an automated Functional UI script, a tester shall:

  1. (uniquely) IDENTIFY the UI object-to-test
  2. set the test ACTion to perform
  3. CHECK that expected results and actual results match
    All this is what we call the I-A-C pattern.

For this article, we will use the login system of the OrangeHRM demo software. (See Figure 1).

Learn how to automate functional UI testing using the I-A-C pattern, and create more effective automated tests.
Figure 1 – OrangeHRM login sample

Test case design

First of all, let’s start designing the test case.
On a login system, a tester wants to insert valid credentials (username and password), and after clicking on the login button, she automatically wants to log in.

Schematically:

  • Enter a valid username
  • Type a valid password
  • Click the login button
  • Check user is successfully logged in

Let’s automate this test case by using the I-A-C pattern.

IDENTIFY the UI object-to-tests

The first step is to locate the test objects.
So, in the OrangeHRM login test case (Figure 2)

  1. username text field
  2. password text field
  3. login button
Learn how to automate functional UI testing using the I-A-C pattern, and create more effective automated tests.
Figure 2 – the UI elements to use to log in

Selenium and Selenium-based tools use “Locators.” Basically, a locator enables testers to identify the web element to interact with.
Selenium offers several different methods like by className, cssSelector, id, name, XPath, etc.

WebElement usrName = driver.findElement(By.id(“txtUsername”));
WebElement password = driver.findElement(By.name(“txtPassword”));
WebElement login = driver.findElement(By.xpath(“/html/body/div[1]/div/div[3]/div[2]/div[2]/form/div[5]/input”));

Other tools use “pre-recorded” GUI Maps (or Object Repositories) to locate objects.
A GUI Map represents an abstraction of the structure of the user interface. Hence, it contains the logical names and physical descriptions of the UI objects to test.

For example, a tool based on this technology reads an object’s description in the pre-recorded GUI map (Figure 3) and then looks for an object with the same properties in the tested application.

Figure 3 – a GUI map

Other tools like Maveryx do not use any locator or map: the UI elements to test can be described into the test scripts as they simply appear in the application (on-screen).

In this example, the username text field is identified by its placeholder “Username” (Figure 4).

//the Username text field
GuiText usrName = new GuiText(“Username”);

Figure 4 – the username text field

Also, the password text field can be identified by its placeholder “Password” (Figure 5).

//the Password text field
GuiPasswordText password = new GuiPasswordText(“Password”);

Figure 5 – the password text field

The same applies to the login button, which can be identified by its caption “Login” (Figure 6):

//click Login button

GuiButton login = new GuiButton(“LOGIN”);

Figure 6 – the login push button

So, the test objects defined in the test script are identified directly at runtime without using any pre-recorded GUI Map.

ACT on the UI object-to-tests

Once the test objects are located, a tester wants to perform the relevant test ACTions on them, such as clicking on a widget, typing some text, selecting an item in a drop-down list, etc.
In our example, we would enter some text into the username and password fields and click on the login button.

For instance, to execute these actions with Maveryx, a tester can use setText() and click() methods:

//the username
String username = “Admin”;

//the Username text field
GuiText usrName = new GuiText(“Username”);

//set the username
usrName.setText(username);

//click Login button
GuiButton login = new GuiButton(“LOGIN”);
login.click();

Obviously, the same goes for Selenium or other tools.

CHECK the results

Now that the test interacts with a GUI object, it’s time to verify that the expected output is produced.

Assertions are the best tools to verify that the expected test results match the actual results. They are assumptions on the application under test that must always be true. If any assertion fails, the test will fail. Similarly, if all assertions pass, the test will pass.
Hence, assertions help testers to quickly check if the application under test behaves as expected or not.
For example, if you write a method that calculates the sum of two numbers (e.g., 3 + 5), you might assert that the sum is correct (= 8).

Each assertion contains a boolean expression that you believe will be true when executed. The experience proved that assertions are one of the quickest and most effective ways to detect and correct bugs.

There are several assertions libraries like JUnit, Hamcrest, AssertJ, etc.
For instance, JUnit provides assertEquals() and assertNotEquals() methods to compare the equality and inequality of values. So, the assertion passes when values are equal; otherwise, it fails and throws an AssertionError.

The first parameter passed to the assertEquals method corresponds to the expected value, and the second parameter is the actual value.
In functional UI testing checkpoints include:

  • values checking: a string is entered, a message is displayed, a drop-down contains a list of items, a web element has a given property, etc.
  • property checking: a UI object is enabled, is editable, focused, etc.
  • presence checking: a UI object is present or not in the current user interface

Our example will check if the username and password text field contain the entered search key.

//the username
String username = “Admin”;
//check that the username has been correctly inserted
assertEquals(username /* expected /, usrName.getText() / actual */);

//the password
String pwd = “admin123”;
//check that the password has been correctly inserted
assertEquals(pwd /* expected /, password.getText() / actual */);

If we would check that the “login” button is enabled, we can use: assertTrue(login.isEnabled() /* actual status */);

We could also check the URL of the landing page after logging in
//the Dashboard page URL
String dashboardURL = “https://opensource-demo.orangehrmlive.com/index.php/dashboard”;
//check the Dashboard page URL
assertEquals(dashboardURL, new GuiBrowser().getCurrentPageUrl());

Another way to check that an application behaves correctly is to verify that a UI object is present or not.
We can also verify that the “Welcome” message popped up to check the successful login (Figure 7). For this purpose, testers can use the waitForObject() method:

//check that the “Welcome Shinchan” is present
new GuiHtmlElement(“Welcome Shinchan”).waitForObject(5, 1);

Learn how to automate functional UI testing using the I-A-C pattern, and create more effective automated tests.
Figure 7 – the welcome message

The waitForObject() function waits until the given object exists. It returns if successful or raises a (catchable) ObjectNotFoundException exception on failure, i.e., if the function times out and the UI object is not displayed. By this approach, you can use this method to check if an object exists (i.e., is shown) or not.

Conclusion

Functional UI testing is essential because it verifies that an application does what it is supposed to do through its user interface, as the end-user would. Also, it allows testing an application end-to-end from the start (the UI layer) to end (services, DB, etc.).
From the automation point of view, Functional UI scripts consist of two main blocks: UI interactions and assertions.
UI interactions consist of firstly identifying the UI objects to test and then acting on them.
On the other hand, assertions allow comparing expected results from specifications to actual results from the application-under-test.

A tester should keep in mind the I-A-C pattern to write good automated Functional UI tests:

  1. IDENTIFY the UI object-to-test;
  2. ACTing on it;
  3. CHECK that expected outcomes and actual outcomes match.

In conclusion, this pattern allows creating more effective automated tests, ensuring maximum test flow control and at least more readable and maintainable scripts.

Maveryx is exhibiting at EuroSTAR 2022. It’s our first in-person event in 2 years – and it’s going to be a massive celebration of testing! Learn from 70 testing experts, and connect with your peers at Europe’s best testing event. Get your ticket now – book by April 22nd and save 10% on individual tickets; up to 35% on group bundles. See you in Copenhagen.

Author

Alfonso Nocella

Co-founder and Sr. Software Engineer at Maveryx, Alfonso led the design and development of some core components of the Maveryx automated testing tool. He collaborated in some astrophysics IT research projects with the University of Napoli Federico II and the Italian national astrophysics research institute (INAF). Over the decades, Alfonso worked on many industrial and research projects in different business fields and partnerships. Also, he was a speaker at several conferences and universities.

Today, Alfonso supports critical QA projects of some Maveryx customers in the defense and public health fields. Besides, he is a test automation trainer, and he takes care of the communication and the technical marketing of Maveryx.

Filed Under: EuroSTAR Conference, Test Automation

  • « Previous Page
  • Page 1
  • Page 2
  • Page 3
  • Page 4
  • …
  • Page 13
  • Next Page »
  • Code of Conduct
  • Privacy Policy
  • T&C
  • Media Partners
  • Contact Us