Quality Assurance

Why Crowdtesting Should be an Imperative Pillar of Quality Assurance

August 2, 2023 by Lauren Payne

Thanks to MSG for providing us with this blog post.

Users are looking for products that inspire – or at least don’t bother them

Future generations – all of them digital natives – will no longer enter their business relationships as traditional customers. The changed demands and the constant transformation through digitalization are turning customers into users. But where no human interaction can create trust, dispel doubts, and answer questions, the product alone is in the spotlight and must have the ability to convince in a very short time and with a reduced attention span.

Attractive, easy to use and – best of all – with a higher range of functions.

Constantly available and nearly unlimited offerings are no longer disruptive but common standards. This applies to products, services, and public offerings at the same time. So, whatever your offer is, you must make sure, the users find it attractive, easy to use and with a suitable range of functions.

The users – not a homogeneous mass

Another challenge is to meet the different target groups and to create a digital infrastructure that covers their different needs equally. Those of Generation Y and Z, which have the purchase power and demand of the future expect modern forms of interaction, purchasing products and services fully digital. The future “everything is now” generation, which is no longer tied to long-term contracts and is used to getting whatever they are looking for on demand.

The competition among web-offerings, which compete without ties and with the promise of a “change of supplier in minutes”, meets this need. The time span to inspire or disturb new users is accordingly very short. Not at least because the tolerance for errors also decreases with the rising use of digital products. By now, most users have gained so much experience with apps and online products that they have a clear expectation of functions and usability. If these expectations are disappointing, they simply download the next app. And even if this is sometimes tied to opening an account, today this can be done quickly enough and with reasonable efforts.

The subjective experience counts

As good as product design and functionality may be, the product experience is and remains subjective. Every product will always create a subjective use case for the user, and this must work to store a positive experience.
A subjective use case could be that a user carries out his transactions exclusively while commuting on a mobile device and expects for instance a banking app to be compatible with his mobile device. The app should be so intuitive to use that external distractions do not disrupt the user flow and ideally the data flow should adequately handle the switch from 3G/4G mobile networks to WLAN networks. If all this fits, the experience is consistently positive.

This in turn not only brings the advantage that the individual user is satisfied, but providers also benefit from the fact that an experience is always communicated to others.

Position yourself on the market through assured quality

By assessing the product quality, you may influence your positioning on the market towards an outstanding product experience. This inherits the following to be ensured:

The smooth functionality of the product on the most popular devices in the market.
The provision of the appropriate range of functions with the right characteristics for the target group.
Covering as many subjective use cases as possible to avoid negative surprises after go-live.

While the first point can still be tested internally and in the laboratory, for example with emulated devices, as part of a verification, the other two points can only be tested as part of a validation.

Crowdtesting offers solutions

Crowdtesting is the validation of digital products involving your target group – remotely via the internet. Leaving this rather rigid definition behind, this method offers good tools to meet the three challenges of digital assurance. It allows positioning towards the upper right quadrant of digital excellence and thus can serve to stand out from the masses with an outstanding product.

Figure. 1: The quadrants of digital excellence

Crowdtesting helps you to cover subjective use cases and perceptions in any phase of the life cycle. You get a direct insight into whether your target group feels heard and can adapt at any time. In addition, with the variety off devices and mindsets added to your testing process you will be enabled to find functional and technical issues which wouldn’t be uncovered in the lab. And if there are no functional problems, that’s worth a pat on the back for your development and builds confidence in your product.

Feedback will always be a part of this testing process and even if the insights and “bugs” gathered in this process may not be fixed, they can be incorporated into the further development of the product. In the meantime, the results help customer support to prepare for possible enquiries and to create meaningful FAQ lists.

Conclusion – Crowdtesting is useful in any phase of a products lifecycle

It gives a good insight into the technical and functional stability of your product and provides the opportunity to understand the (future) users from the beginning and develop with a focus on their added value. You don’t have to wait for feedback from customers who may be disappointed once, not return to your site at all and not using your app a second time.

Author

Johannes Widmann

Johannes Widmann has been working in the field of software quality and digital assurance for over 22 years. He is a dedicated desciple of crowdtesting since 2011 and has built up passbrains, one of the leading service providers for crowd-sourced quality assurance. Since January 2021 passbrains is part of the msg group.

MSG is an EXPO Exhibitor at EuroSTAR 2023, join us in Antwerp

Testing and QA Key to Cloud Migration Success

July 27, 2023 by Lauren Payne

Thanks to iOCO for providing us with this blog post.

In the global rush to go serverless and in the cloud, many organisations neglect quality assurance and testing – an oversight that can seriously impair performance and increase organisational risk.

There are numerous reasons for this, but a key one is that cloud migrations are complex projects usually managed by infrastructure teams. Those tasked with driving it aren’t always quality focused, and their views of what QA is might differ significantly from what QA should be.
Should the organisation neglect thorough testing as part of its application cloud migration plan, the smallest mistake left undiscovered, could cause major failures down the line.

Lift and shift migration, the most popular approach and the second-largest cloud services sector by revenue, should not be seen as a simple copy-and-paste operation. Without a concerted effort, accurate planning and coordinated migration testing, a copy-and-paste approach could have devastating consequences for scalability, databases, and application and website performance.

Cloud Migration Testing and QA Priorities and Pillars

Thorough cloud migration testing uses quantifiable metrics to pinpoint and address potential performance issues, as well as exposing opportunities to improve performance and user experience when applications are in the cloud. However, teams should be cautious of scope creep at this stage – adding new features during migration could have unforeseen impacts.

Proper testing and QA rests on four key pillars – security, performance, functional and integration testing.

Security testing must ensure that only authorised users access the cloud network, understanding who has access to the data, where, when and why users access data. It must address how data is stored when idle, what the compliance requirements are, and how sensitive data is used, stored or transported. Suitable procedures must also be put in place against Distributed Denial of Service (DDoS) attacks.

To realise the performance and scalability benefits of the cloud, testing must validate how systems perform under increased load. Unlike stress testing, performance testing verifies the end-to-end performance of the migrated system and whether response times fulfil service level agreements under various load levels.

Functional validates whether the application is ready to be migrated to the cloud, and whether it will perform according to the service level agreement. In complex applications, it is necessary to validate the end-to-end function of the whole application and its external services.

Even in basic applications where microservices architecture is not required, we see some sort of integration with third-party tools and services, making integration testing important. Therefore, cloud migration testing should identify and verify all the dependencies to ensure end-to-end functionality, and should include tests to verify that the new environment works with third-party services, and that the application configuration performs in a new environment.

With well-architected testing carried out, the organisation can rest assured that cloud migration risks have been mitigated and opportunities harnessed across security, operational excellence, reliability, performance efficiency, cost optimisation and sustainability.

A Testing and QA Framework for AWS Cloud Migration

As an AWS certified partner provider, iOCO has tailored our Well Tested Cloud Framework (WTCF) for cloud migration to align with the AWS Well Architected Framework, to ensure customer migrations to the AWS cloud are not only successful, but actually exceed expectations. iOCO resources will lead and manage execution from initial assessment, risk identification and recommendations; through a comprehensive set of checklists and guidelines across each of the four QA pillars; to full migration testing.

In tandem with the AWS Well Architected Framework, iOCO’s WTCF is designed to fast-track AWS migration testing using clear and structured guides and processes and customised options to suit the organisation’s budget and needs.

Author

Reinier Van Dommelen, Principal Technical Consultant – Software Applications and Systems at iOCO

As a seasoned Technical Consultant with a wealth of experience, Renier Schuld has a proven track record of delivering successful IT projects for a diverse range of clients. He excels at bridging the gap between business and technical requirements by identifying and implementing systems solutions, guiding cross-functional teams through the project life-cycle, and ensuring successful product launches.

Renier’s expertise in Testing is extensive and includes developing functional specification documents, designing test strategies, creating and executing test scripts to ensure accuracy and quality, developing project and organizational software test plans, providing user support, and building automated test frameworks. He has a passion for continuously improving processes and ensuring that quality is always top of mind throughout the project life-cycle.

iOCO is an EXPO Exhibitor at EuroSTAR 2023, join us in Antwerp

Choosing the best Test Automation Tool for your QA Team

March 29, 2023 by Lauren Payne

Thanks to Leapwork for providing us with this blog post.

In this short article we’re going to cover how QA teams can set themselves up for success with test automation and test management that fosters collaboration.

But first, what causes these issues within QA, and how does test automation come into the picture?

Test automation has typically been a very siloed operation. The solutions that we’ve been relying on until now are typically code-based.

And here-in lies the problems with code-based solutions:

They are adopted by a small group of people. People who normally have a developer-type profile. This makes automated testing unusable to those who need it – product owners, test managers, ITOps, and business analysts. Collaboration is hindered.
Developers (as we know too well) are expensive to hire and difficult to find. We need developers to spend their time where they create the most value for the business – developing features, and fixing issues that arise during testing.
The maintenance it takes to keep these automated tests running is immense. Take Selenium as an example which requires more time to maintain than is spent testing. This simply isn’t scalable. And it creates another collaboration blocker.
It can take anywhere between 6-12 months to learn a code-based tool. If there is only one person (we’ve discovered that this is quite often the case) building and maintaining these tests, what happens if they’re on holiday? Or sick? Or quit? If a test breaks, there’s no one to catch regressions or run tests. No one to quickly pick up the build and maintenance of said tests. You can’t meet your deadlines. And the business can’t push out releases or customizations quickly. Progress is put on pause.

What can you do to avoid these bottlenecks?

For starters, test automation doesn’t have to be code-based. In fact, test automation shouldn’t require a tester to code at all.

Testing isn’t a domain that’s specific to developers. It’s a task that’s carried out by an entire profile of people. By that logic, test automation should also be accessible to the people who are testing – business users and analysts (aka. User acceptance testers), test managers, and product owners.

And lastly, test automation shouldn’t require more maintenance than the time you spend testing.

A visual automation framework can help you overcome these challenges.

What exactly is a visual automation framework?

Well, instead of using code to describe test cases or processes, tests are built in a way that people find easy to understand – as simple as cobbling lego blocks together.

In the image below, you can see an example of what a visual automation framework looks like. A series of blocks that enable any tester to build automation in minutes, and an entire team can learn, build and maintain in a month.

Seeing this picture for the first time, it probably won’t capture your imagination right away. But this approach to automation means that you can close the skills gap between developers and testers. You’re enabling anyone with the tester profile to build and collaborate on/with(?) automation.

And, you can keep maintenance to a minimum. You don’t need to comb through lines and lines of code to ensure that a test is functional.

The outcome? Your life becomes a whole lot easier. New features, customizations and products are released much faster.

As a result, there is more room for collaboration between developers and testers during the development lifecycle for the work that you find valuable. And for the work that brings value to the organization.

If you want to learn more about why visual automation frameworks are key for collaboration, and how they can foster a future of faster releases, visit our solutions page on test automation.

Author

Anna Thorsen, Automation Expert at Leapwork

Anna Thorsen is an automation expert and writer and covers a range of topics on test automation. These topics range from tackling technical debt, getting the best return on investment from test automation, and assessing your testing maturity so you can build an efficient QA function.

Leapwork is an EXPO Gold partner at EuroSTAR 2023, join us in Antwerp

The future is here – is QA ready for it?

June 3, 2022 by Fiona Nic Dhonnacha

Thanks to Jani Haapala, DevOps Architect at Gofore, for providing us with this blog post.

William Gibson famously said, “the future is already here – it’s just not evenly distributed”. In software development, we constantly talk about future trends and future methods that are coming one day, but so often fail to see that in most cases, the future is already here – but maybe not yet in your context.

The problem with QA and testing is that it always seems to play catchup with these trends and methods. QA is mostly reacting and adapting to challenges that the future is bringing to us, rather than proactively creating or defining them.

So, if there’s already a recognized need, how come we’re not answering it? In my opinion, it’s because, in some cases, those of us in QA are still living in the past.

Living in the past

Some of us are still living in the past – although we might not realize it. In these situations, I see that QA as a function fails to provide the value that it should. I have found out that there are two major contexts where QA is still living in the past:

QA is a step: in many cases companies have adopted agile, and are driving fast toward DevOps. Some companies are already doing DevOps in some form. But QA still seems to be mostly a manual step on these fast phased cycles. Developers are having CI/CD pipelines; build and unit testing is happening automatically but when it comes to functional testing, end to end testing, and system testing, it often tends to be something that “needs to be asked from QA people”. This separates QA to its own manual silo, and creates unnecessary handovers. Instead of this, QA people should drive toward automated testing that is fully integrated to the CI/CD pipelines that development is using. Test automation is not a tool that testers use; it’s an automatically triggered set of tests by commit of code.

QA is about running tests: QA is often still fixated on ‘creating tests’ and ‘running tests’, and it seems that the only function of QA is to be the executor of tests. This leads to a situation – with test automation in particular – where there’s thousands of test cases that might pass and might not, or nobody cares. Tests are becoming mere ‘experiments’ that fade over time. So, QA and testing should be all about bringing valuable feedback to people that need it. They should focus on thinking about the most valuable thing that we can get from the system, and how to present that to those who need it. And remember, most valuable things change, and so does the importance of any given tests.

The future that we live today

So, what are the latest trends that QA should take note of sooner rather than later? What’s already on our doorstep, and needs immediate attention? Let’s hop on the train early, figure out what kind of feedback is needed, and learn how to provide that. Some of the future scenarios that we are already living include:

The ‘XaaS’ movement: in the past there was a huge ‘XaaS’ moment – ‘Something as a Service’ shaped our daily lives. Rather than creating everything themselves, development teams started to consume services. Cloud services, version controlling services, deployment services, and so on. After that, we found out that these services and platforms could be controlled programmatically. So, the new ‘XaC’ boom started. ‘XaC’ means ‘Something as Code’, and that means that everything is done by creating a code file for it. Examples of these are Infrastructure as Code, Configuration as Code, Storage as Code, and Detection as Code. For this modern need, QA also needs the ability to test, verify, and provide feedback on how these code files are working

QaSec ownership: if QA is considered to be a silo, then I have a story about InfoSec! InfoSec is often an even more separate action than QA. InfoSec is mostly done completely separately, and if InfoSec tries to suggest something, it’s only considered if it doesn’t affect the daily work, or make it more complicated. In the future, InfoSec should follow in the footsteps of QA – and be part of QA. Having QA and Sec combined to one QaSec needs to have real impact to development. Too often agile managers are cut down so much that nobody is in charge of overall quality or security. That’s why QaSec needs to be owned and recognized as one of the key stakeholders of development.

New architectures: The world is truly evolving. People who were running systems of physical machines or virtual machines are now running the same services on clouds in the form of service meshes and microservice architectures; or even without any maintained resources on just lambda functions. QA needs to be very aware of both today and the future architectures to be able to give proper feedback on these. QA can’t also test those if they are stuck on ‘just using’ something that was set up for them; they need to be actively involved in creating and improving these to define safe production environments, and get production-like environments for their needs.

Get ahead of the future

So, if we want to be proactive instead of reactive towards new trends and ensure our validity, what could we do? I don’t want to start doing too much guesswork here, but some future “safe bets” that will eventually take over the world are:

New methods: new methods are coming, and those methods need the capability to adopt. QA might be done on different stages or in different contexts. For example, one of the promising ‘new’ methods is Google’s Site Reliability Engineering (SRE): that’s Google’s take on DevOps, and focusing on reliability. SRE is an extremely useful practice when companies are having more large scale software production. There’s likely more than one, and they can grow rapidly in popularity.

RPA: Robotic Process Automation is something that we already see companies experimenting with. This enables companies to automatically connect legacy systems together to form long, fully automated business processes. RPA used to be available only from vendor companies with a high price tag, but more and more RPA is now done with pure open-source software. In many cases, QA can repurpose their test automation knowledge and tooling to start implanting RPA solutions also.

Artificial Intelligence: If RPA is the first step into full process automation, AI is mostly the target. Even though AI sounds both cool and scary, the reality is that we are far away from general AI that works just like a human. In QA’s scope the near future is very interesting when it comes to testing early AI applications, or using AI as part of the testing. People’s first reactions is to point out that AI can’t do the same things as humans, or make decisions – but that’s not the point (at least right now). The point is to help people make the right decisions with a faster lead time.

Get into the high performers group

If you recognize yourself embracing the future, please, be loud about it! Keep presenting at conferences, and spread the word on how to get ahead of the game, and help your fellow QA people join the high performers group. But if you see yourself in the past or present group, please start taking advantage of all of this, and begin the journey to elevate yourself to the high performers group. It just takes a willingness to learn, and some time, to get you started on your journey.

Author

Jani Haapala, DevOps Architect at Gofore

Jani works as a DevOps architect at Gofore. He is passionate about measurement, feedback, and continuous automated quality feedback loops. Jani’s journey started from manual testing and has evolved to full-scale software development automation. Jani thinks that automation can help everybody and increase value in anything.