Software Testing

3 methods of test book testing

July 4, 2022 by Fiona Nic Dhonnacha

Thanks to UNGUESS for providing us with this blog post.

How do you choose the right methods of test book testing? What are the possible methods? Let’s explore all the alternatives.

When it comes to functional testing, there are multiple options to choose from. Which option is right for your business will depend on the type of software you want to test, your target audience, and your overall testing goals. With that in mind, let’s explore the three possible methods of test book testing.

What is a test book?

Functional testing validates software against the functional requirements or specifications. Each company has unique goals they want to achieve with their software, and functional testing is an excellent way to determine whether you’re meeting those goals. A test book is simply the list of test cases that will be executed during functional testing. The test cases are designed with your specific software in mind. For example, if shopping giant Amazon wanted to test the functionality of its search function, they (or an external company on its behalf) would develop the test cases from the ground up. The test cases would be specific to the website’s design, the available filters, the shopping cart process, and so on.

The key benefit of functional testing is that it helps you close the gap between how your software should work and how it actually works. Sometimes software seems logical to the people who create it but creates confusion for end-users. And sometimes, the software simply doesn’t work as intended and needs to be reworked.

The 3 methods of test book testing

1. Crowd testing

Crowd testing, sometimes called crowdsourced testing, tests usability, functionality and user-friendliness under real-world conditions.

Crowdsourced testing is a manual approach to software testing. Testers are real professionals who get involved only for the testing process, meaning they get paid by project and performance instead of being full-time employees.

The Benefits of Crowd Testing

Real-world conditions: It allows you to test the software’s functionality using real end-users in real-world conditions. Due to the large number of users, you can test functionality at any time of any day.
A wealth of testing data: Crowd testers use their real devices and operate different operating systems, browsers, and other native software. This makes the testing data rich, both in terms of diversity of feedback and high coverage.
Fast, unbiased reporting: Testers give feedback on the go, which means you get a constant stream of valuable feedback. Additionally, while all competent testers must be impartial, crowd testers have a distinct advantage in this area. They have no stake in the success of your software, so their feedback is guaranteed to be honest. Honest feedback is critical when creating highly functional and user-friendly apps.
Target demographics: You can source crowd testers from a vast range of demographic criteria, including age, gender, profession, income level, hobbies, and more. Crowd testing is an excellent way to see how well your target audience perceives your software. This aspect is extremely important in UX Testing.

In UNGUESS, we have created a vast community of testers (over 20,000 in Italy, our main market, and we can reach over 250,000 worldwide). Since they are highly profiled, we can select professional testers and/ or real users of your product. Our Customer Success Managers (CSM) write the test cases for the test book specifically aimed at testing our clients’ software. Testers submit the bugs they find and the CSM approves them only if they are unique and in scope. Our clients’ Dev team can receive an export of all the bugs, or can use a bug tracking tool integration.

2. Test automation

In automated functional testing, a computer runs the functional testing for you using tools and scripts. Typically, the software is categorised into distinct areas of functionality, and a script is designed to test the functionality in a series of steps. For each step, pre-selected elements of the UI are interacted with. For example, the automated test software might fill in a text box, select a dropdown option, or push a button. After running the automated test, you can check the results to compare them with what should have happened.

At AppQuality, for example, we have a range of tools we can leverage for our clients’ test automation, depending on their specific software specifications. As for specifics, we often use Robot Framework – a robust open-source automation framework with a rich ecosystem of libraries and tools (we have talked about it here) and Seleniumlibrary (or other libraries). Again, this is just an example as we use different tools and frameworks depending on our clients’ necessities.

The benefits of automated functional testing

24/7: A computer runs automated testing, so you’re not restricted by human testers who need to take breaks, sleep, or only work set hours.
Faster: Without having to rely on people (unless writing the code for the test Automation is necessary), you can execute your tests much quicker. On average, automated testing is 70% faster than manual testing—faster testing results in an early time to market for your application. Warning: with Test Automation you can only find what you are looking for. Nothing out of your path can be found.
Improves accuracy: When you eliminate humans, you also eliminate human error. Any errors can reduce the accuracy and reliability of your results.
Cost-efficient: Automated testing is often significantly cheaper than manual testing.
Ensures consistency: Automated testing can be repeated as you make improvements and tweaks to your software. This promotes consistency across every stage of your functional testing, making it much easier to compare the results of multiple tests objectively.

3. FTE (Full Time Equivalent)

FTE (Full-Time Equivalent) or Third-party Test Factories can mean one of two things in the testing world:

The company hires a top tester who writes the entire test book and then executes the tests on the application, website, or other software.
The company writes the test book, and the tester runs the test cases for them.

FTE offers a high level of flexibility and allows companies to engage with the testing process in the way that best suits your company. Many companies don’t have the time or resources (skilled testers) to develop the test cases themselves and benefit from the help of an outside party for a short amount of time. Other companies have a clear idea of the tests they want to run but want an impartial expert tester to execute the test cases.

To learn more about FTE or Third-party Test Factories, see how it works and the possible results, read this article and the case study of buddybank.

The benefits of third-party test factories in functional testing

Objectivity: Testers are fair and unbiased in reporting their results. This can be difficult to achieve with in-house testing because employees have a vested interest in the software – they might overlook negative aspects of the functionality to appease senior management.
Specialisation: Testers have formal training and certifications in testing related disciplines. They have a deep knowledge of testing methodologies, best practices, and appropriate tools. They bring a level of professionalism to the table that results in highly accurate, fast, and reliable results.
Risk reduction: Introducing a third party can help reduce the risks associated with functional software testing. Testers can spot blind spots or errors in your software that you might otherwise miss.

Which method of test book testing should you use?

It is impossible to answer that question out of the blue. It is crucial to understand the context, the goals, the available resources, tools and stage of development.

UNGUESS offers a Crowd testing platform (together with tens of thousands of testers), Test Automation technologies and knowledge and even the Body Rental approach.

If you want to know which method is the most suitable for your product, contact us and schedule a short call with our experts.

UNGUESS is an exhibitor at EuroSTAR 2022.

Talking to developers about testing

June 21, 2022 by Fiona Nic Dhonnacha

Thanks to PractiTest for providing us with this blog post:

Let’s start by reminding ourselves why we need to talk with developers about testing.

According to the 2022 State of Testing report by PractiTest and Tea-time with testers, 86% of organizations now report working in an Agile or Agile-like approach. Close to 40% of organizations are working in DevOps methodology. And while the more traditional Waterfall approach is still practiced at 17% of organizations, it continues to decline.

This means that the traditional testing roles and teams, as we used to know them, are gradually disappearing, and instead, testing is becoming the responsibility of an entire agile team that also includes developers. Hence, it is of no surprise that in 77% of organizations testing is executed by testers and non-testers alike.

This shift often encounters resistance from developers with various claims about lack of knowledge on how to test, being too busy to do so, the perception that it’s a waste of their expensive time, or simply lack of desire to do so.

In order to help you bring everyone on board with the concept of product quality being the team’s responsibility we have prepared a 5 steps guide to help you:

Talking to developers about testing - communication

1. Communication

The business environment is no different than others: when it comes to human interaction, communication is the basis for a successful relationship. In order to get everyone on board, you need to start with making people understand that performing testing is not a punishment, but rather a contribution to the company’s bottom line, due to the ability to deliver faster with higher quality that will meet customers’ expectations. It requires executive sponsorship, to support the required change both in culture and in methodology.

Communication is a two-way channel, and as a result, it’s also about listening, understanding, and addressing the developers’ concerns. Pay close attention to the developers’ feedback and take it into account in order to maintain an open communication environment.

Talking to developers about testing - coaching and mentoring

2. Coaching & Mentoring

One potential reason for people to object to a change, is their fear of dealing with the unknown and the concern of not excelling in their performance. That’s why we need to dedicate time and effort to guide developers with essential information about testing. We need to convey to developers that testing is not ‘simple’ work. We want them to understand the software testing process, implement test management platform usage, and so forth.

In addition, invest some time in short pre & post briefing sessions. These meetings should include the developer of the tested feature, the testing developer, and a product team member. By doing so, we will bring a better understanding and efficiency to the process.

3. Shift left

Adopting Agile and DevOps methodologies has also created a shift in what is tested and when. We see a major shift left with the introduction of Behavior Driven Development (BDD) support and Test-Driven Development (TDD).

BDD/TDD are approaches that are focused on shift left. It aligns people and generates most of the tests before the testing is actually performed. Mistakenly, the common thinking is it’s used only for automation, but it can also be applied in manual testing.
Testing as a part of the user story – this means that user stories should also include testing elements.
Personas – a representation of the typical users that are going to make use of our development and their characteristics. For example, in an application that aims for the elderly, we need to think about them when creating the scenarios.
Testing sessions during development – you can have pair testing and have sessions while features are being written. It doesn’t have to be completely built to be shifted left.
Dashboard for unit/integration – include in your dashboard unit/integration test results like any other testing. This will place developers’ unit/integration tests at the same level as tests executed by testers.
Stability – the features shouldn’t be stable only before the release, it should be stable all the time. It is very important, it signals developers that they are already testing if they open their eyes.

4. Other testing

Sometimes it is easier for developers to relate to testing which isn’t the main core of the functional testing. The easiest way to use the “other testing” is simply to ask the developers and use their own ideas. It’s significantly easier to implement your ideas instead of someone else’s. We can also use developers’ knowledge to improve our automation framework and make the testing process closer to their hearts.

Specialized testing is another great way to leverage our developers’ knowledge to contribute to the testing efforts. Testing types like security, chaos, and performance are simpler to developers than testers. Developers would have a strong motivation to execute those specific specialized tests.

Talking to developers about testing - transparency and visibility

5. Transparency & Visibility

This is extremely important, and yet, it is amazing to see how often this is left aside. Transparency and visibility have a massive added value when we involve different people in the process.

End-to-end dashboards – if you are going to have multiple testing types such as unit tests, functional testing, and integration tests, you should have everything in the same place and give the same legitimacy to all tests. The moment you see that “my tests” are also included in the dashboards, that’s when you understand the importance of the tasks and your contribution to the mutual goal.
Count every test I have – regardless of the type or complexity.
Add testing to standups – a lot of time people do talk about testing in standups but we leave it to the last minute and don’t talk about challenges other than sharing ongoing activities.
Celebrate achievements – even in testing, if we have a developer that found a show stopper it is important to celebrate that because it has a positive effect on the staff members.
Testing retrospective – especially when a bug slipped and was found in production, it has a lot of value when we learn from it. Our victories and errors are equally important and valuable.

Summary

In conclusion, with most organizations working in an Agile or Agile-like environment, the software testing process is no longer exclusively assigned to testers, and other staff members like developers are taking part in the testing process. Developers have great expertise and in-depth knowledge that could substantially contribute to testing efforts. Therefore, you should follow the 5 steps mentioned above in order to effectively make them an integral part of the testing process.

Practitest is an exhibitor at EuroSTAR 2022.

Author

Joel Montvelisky, Co-Founder and Chief Solution Architect at PractiTest.

Joel has been in testing and QA since 1997, working as a tester, QA Manager and Director, and a Consultant for companies in Israel, the US and the EU. Joel is a Forbes council member, a blogger and is constantly imparting webinars on a number of testing and Quality Related topics. In addition, Joel is the founder and Chair of the OnlineTestConf, the co-founder of the State of Testing survey and report and a Director at the Association of Software Testing.

How to test mobile apps in the cloud

June 18, 2022 by Fiona Nic Dhonnacha

Thanks to Mike Wagner from Keysight for providing us with this blog post.

In 1973, Martin Cooper did something that no one had ever done before. He placed a call on the world’s first commercial cell phone. The Motorola DynaTAC 8000X might have been slightly smaller than a house phone with limited functionality and poor call quality by today’s standards, but this was a significant step in developing mobile device technology.

Mobile devices today are wildly different. They have multiple screen sizes, resolutions, and functions, with capabilities expanded by millions of apps to meet our specific needs at work and home. Brilliantly convenient for the user, but with different models, operating systems (OS), and manufacturers, ensuring your app works across each of these variables is challenging. Get it wrong, and your app goes in the trash can. In the US alone, 43% of apps are uninstalled after 30 days, according to Statista.

Testing mobile applications is the only way to ensure functionality before it hits the end-user. However, you need to execute thousands of test cases because there is no guarantee that an app performing on an Android device will work without issues on the latest Apple iPad or Samsung handset. Even simple problems with screen aspect ratios or orientation can occur on the same OS but different device models. So, what’s the answer?

Cloud-based mobile app testing

Cloud-based device farms optimize mobile app testing by executing test cases in virtual software environments. Mobile conditions are replicated using simulators, emulators, or on actual devices hosted in the cloud.

Compared to physical on-premise device farms, cloud-based device farms are easier to maintain and allow multi-site project teams to test applications on numerous devices in various real-user situations without setting up a physical lab. Competition is fierce, with thousands of apps released daily around the world. Testing app performance and functionality are essential to prevent bugs from entering production. If bugs are missed, costly delays occur, damaging the user experience and losing customers.

Benefits of cloud-based mobile app testing

One obvious benefit to cloud-based mobile app testing is that it is much easier to maintain than physical device farms, but there are a few more advantages that are at hand

Cost-effective

Executing hundreds of different test scenarios across multiple OS, devices, and models is hugely expensive when acquiring and managing physical hardware yourself. Cloud-hosted testing platforms, such as Sauce Labs, offer monthly charges for a range of old and new devices, so you don’t need to constantly update yours when new models are released.

Better availability

When your testing environment is hosted in the cloud, a massive number of devices, platforms, and OS are available via simulators and emulators. Real devices are also available to instantly access the latest models and features to execute accurate testing.

Easily accessible

Quality Assurance and test engineers can access a device farm and execute tests from anywhere, which is helpful in distributed teams as it supports collaboration. By accessing mobile device farms in the cloud and from various geolocations, testing can occur 24/7, accelerating time to market.

Enhance collaboration

Cloud-based device farms can be integrated with test management tools to accelerate feedback loops and software delivery. Various people are involved in software development, so good communication is critical to identify if they are any issues with a release so fixes can be made as quickly as possible.

Increased productivity

Utilizing a device farm in the cloud allows parallel testing of various devices, rather than testing one after another, which dramatically speeds up test cycles. Testing is often a bottleneck to software development, so being able to execute multiple test scenarios unique to your end-users accelerates production deployments.

Mobile app testing in the cloud with Keysight’s Eggplant software test automation

Manually testing apps on multiple devices can be unreliable, costly, and slows down release cycles. With Keysight’s Eggplant test automation solution, you can accelerate the end-to-end testing of every user journey in your app across multiple mobile device types.

Eggplant’s flexibility allows integrations with mobile device farm providers, so your teams can access numerous devices and execute test cases specific to their end-users needs.

Using Eggplant test automation software, testers can expand test coverage incrementally to parts of an application with discovered issues via intelligent automated exploratory testing. The artificial intelligence and machine learning algorithms proactively explore areas close to these defects, as the probability of other bugs is high. Missed defects are costly to fix, especially when discovered in production.

Eggplant’s ability to increase test coverage helps reduce the chance of a buggy user experience, which can cause brand damage, dissatisfied end-users, and negatively impact an organization’s bottom line.

To discover how to optimize your app testing and accelerate releases, read our guide on mobile device testing in the cloud.

Keysight is a Platinum Partner at EuroSTAR 2022.

Author

Mike Wagner

Mike Wager is a Product Marketing Manager at Eggplant, a Keysight Technologies company. Over the last decade, he has brought product ideas and digital services to life for engineering technologies, SaaS applications, and software development. In his current role, Mike communicates the unique benefits of AI-driven test automation for enterprise applications, retail services, and mobile testing.