Test Automation

This article was provided by Alfonso Nocella, Co-founder and Sr. Software Engineer at Maveryx.

Automated Functional UI Testing

We generally used to consider Functional UI Testing the most critical type of testing because, first of all, an application must do WHAT it is supposed to do. And indeed, it verifies that an application works as expected through its user interface.

Or, if you prefer, according to Glenford Myers, Functional UI testing means to verify the software through its UI with the intent of finding errors (i.e., “find discrepancies” between the program and its specification).

Given these points, Functional UI testing verifies, from the end user’s perspective (through the user interface), that the software behaves according to its functional requirements specification without knowing the internal logic and implementation (black-box). In simple words, Functional UI testing verifies an application in the same way an end-user would.

Also, Functional UI tests are effective because they go “end-to-end” from the UI layer. Indeed, end-to-end means testing all the different parts of an application from the beginning (the user interface) to end (the underlying services, the database connectivity, etc.).

From the automation point of view, Functional UI tests imitate users’ actions such as clicking on a control, entering text, scrolling a page, and so on by also comparing the actual outputs with the expected outcomes.
With this in mind, in automated Functional UI scripts, working with user interface elements, a tester should always use this pattern:

1. IDENTIFY: locating the elements to use
2. ACT: performing the requested (test) actions
3. CHECK: comparing expected results vs. actual results

In fact, this pattern recurs continuously in an automated test script.
This paper will explain how to write automated Functional UI tests using this pattern.

The I-A-C pattern

In an automated Functional UI script, each test “step” consists of two parts: “action” and “checkpoint.” An “action” is an interaction between the test tool and the UI object-to-test. An example of action is clicking a button or entering a text into a text field.
Besides, an action consists of a “target”, a UI object-to-test, and a “command”, a test action to perform (e.g., “click”, “type”, …). A command may have inputs (e.g., type “some text”) or not (e.g., “click”).

A “checkpoint” compares the actual outcome from the application under test and the expected one. For example, the predicted result could be that typed text is in the text field.

Hence, writing an automated Functional UI script, a tester shall:

1. (uniquely) IDENTIFY the UI object-to-test
2. set the test ACTion to perform
3. CHECK that expected results and actual results match
   All this is what we call the I-A-C pattern.

For this article, we will use the login system of the OrangeHRM demo software. (See Figure 1).

Test case design

First of all, let’s start designing the test case.
On a login system, a tester wants to insert valid credentials (username and password), and after clicking on the login button, she automatically wants to log in.

Schematically:

• Enter a valid username
• Type a valid password
• Click the login button
• Check user is successfully logged in

Let’s automate this test case by using the I-A-C pattern.

IDENTIFY the UI object-to-tests

The first step is to locate the test objects.
So, in the OrangeHRM login test case (Figure 2)

1. username text field
2. password text field
3. login button

Selenium and Selenium-based tools use “Locators.” Basically, a locator enables testers to identify the web element to interact with.
Selenium offers several different methods like by className, cssSelector, id, name, XPath, etc.

WebElement usrName = driver.findElement(By.id(“txtUsername”));
WebElement password = driver.findElement(By.name(“txtPassword”));
WebElement login = driver.findElement(By.xpath(“/html/body/div[1]/div/div[3]/div[2]/div[2]/form/div[5]/input”));

Other tools use “pre-recorded” GUI Maps (or Object Repositories) to locate objects.
A GUI Map represents an abstraction of the structure of the user interface. Hence, it contains the logical names and physical descriptions of the UI objects to test.

For example, a tool based on this technology reads an object’s description in the pre-recorded GUI map (Figure 3) and then looks for an object with the same properties in the tested application.

Other tools like Maveryx do not use any locator or map: the UI elements to test can be described into the test scripts as they simply appear in the application (on-screen).

In this example, the username text field is identified by its placeholder “Username” (Figure 4).

//the Username text field
GuiText usrName = new GuiText(“Username”);

Also, the password text field can be identified by its placeholder “Password” (Figure 5).

//the Password text field
GuiPasswordText password = new GuiPasswordText(“Password”);

The same applies to the login button, which can be identified by its caption “Login” (Figure 6):

//click Login button

GuiButton login = new GuiButton(“LOGIN”);

So, the test objects defined in the test script are identified directly at runtime without using any pre-recorded GUI Map.

ACT on the UI object-to-tests

Once the test objects are located, a tester wants to perform the relevant test ACTions on them, such as clicking on a widget, typing some text, selecting an item in a drop-down list, etc.
In our example, we would enter some text into the username and password fields and click on the login button.

For instance, to execute these actions with Maveryx, a tester can use setText() and click() methods:

//the username
String username = “Admin”;

//the Username text field
GuiText usrName = new GuiText(“Username”);

//set the username
usrName.setText(username);

//click Login button
GuiButton login = new GuiButton(“LOGIN”);
login.click();

Obviously, the same goes for Selenium or other tools.

CHECK the results

Now that the test interacts with a GUI object, it’s time to verify that the expected output is produced.

Assertions are the best tools to verify that the expected test results match the actual results. They are assumptions on the application under test that must always be true. If any assertion fails, the test will fail. Similarly, if all assertions pass, the test will pass.
Hence, assertions help testers to quickly check if the application under test behaves as expected or not.
For example, if you write a method that calculates the sum of two numbers (e.g., 3 + 5), you might assert that the sum is correct (= 8).

Each assertion contains a boolean expression that you believe will be true when executed. The experience proved that assertions are one of the quickest and most effective ways to detect and correct bugs.

There are several assertions libraries like JUnit, Hamcrest, AssertJ, etc.
For instance, JUnit provides assertEquals() and assertNotEquals() methods to compare the equality and inequality of values. So, the assertion passes when values are equal; otherwise, it fails and throws an AssertionError.

The first parameter passed to the assertEquals method corresponds to the expected value, and the second parameter is the actual value.
In functional UI testing checkpoints include:

• values checking: a string is entered, a message is displayed, a drop-down contains a list of items, a web element has a given property, etc.
• property checking: a UI object is enabled, is editable, focused, etc.
• presence checking: a UI object is present or not in the current user interface

Our example will check if the username and password text field contain the entered search key.

//the username
String username = “Admin”;
//check that the username has been correctly inserted
assertEquals(username /* expected /, usrName.getText() / actual */);

//the password
String pwd = “admin123”;
//check that the password has been correctly inserted
assertEquals(pwd /* expected /, password.getText() / actual */);

If we would check that the “login” button is enabled, we can use: assertTrue(login.isEnabled() /* actual status */);

We could also check the URL of the landing page after logging in
//the Dashboard page URL
String dashboardURL = “https://opensource-demo.orangehrmlive.com/index.php/dashboard”;
//check the Dashboard page URL
assertEquals(dashboardURL, new GuiBrowser().getCurrentPageUrl());

Another way to check that an application behaves correctly is to verify that a UI object is present or not.
We can also verify that the “Welcome” message popped up to check the successful login (Figure 7). For this purpose, testers can use the waitForObject() method:

//check that the “Welcome Shinchan” is present
new GuiHtmlElement(“Welcome Shinchan”).waitForObject(5, 1);

The waitForObject() function waits until the given object exists. It returns if successful or raises a (catchable) ObjectNotFoundException exception on failure, i.e., if the function times out and the UI object is not displayed. By this approach, you can use this method to check if an object exists (i.e., is shown) or not.

Conclusion

Functional UI testing is essential because it verifies that an application does what it is supposed to do through its user interface, as the end-user would. Also, it allows testing an application end-to-end from the start (the UI layer) to end (services, DB, etc.).
From the automation point of view, Functional UI scripts consist of two main blocks: UI interactions and assertions.
UI interactions consist of firstly identifying the UI objects to test and then acting on them.
On the other hand, assertions allow comparing expected results from specifications to actual results from the application-under-test.

A tester should keep in mind the I-A-C pattern to write good automated Functional UI tests:

1. IDENTIFY the UI object-to-test;
2. ACTing on it;
3. CHECK that expected outcomes and actual outcomes match.

In conclusion, this pattern allows creating more effective automated tests, ensuring maximum test flow control and at least more readable and maintainable scripts.

Maveryx is exhibiting at EuroSTAR 2022. It’s our first in-person event in 2 years – and it’s going to be a massive celebration of testing! Learn from 70 testing experts, and connect with your peers at Europe’s best testing event. Get your ticket now – book by April 22nd and save 10% on individual tickets; up to 35% on group bundles. See you in Copenhagen.

Author

When everyone talks about digital acceleration and how the pandemic came to boost digital transformation in organizations, we all realize that part of the “secret” to respond to this context is the speed with which companies are/were able to reinvent themselves and to adapt business models to the digital world.

The pressure on IT departments has increased, application development and the introduction of new solutions and technologies as well. And it is in this scenario that talking about Quality – Quality Assurance and Quality Management – makes even more sense. These are issues that business decision-makers and managers should be aware of. Talking about Quality, Quality Assurance, software testing, it cannot be just an IT theme, it is a theme that impacts the entire organization, different stakeholders, customers, in short, the business.

Some facts that demonstrate it

– According to Forrester, 25% of users abandon a web or mobile application after a delay of just 3 seconds. 3 seconds is an eternity in customer perspective, who quickly leaves the Site or application … and looks for alternatives.

– According to the IBM System Science Institute, the correction of a critical incident in Production has a cost 10x higher than it would be in the final testing phase, and 100x higher than the design phase. These costs are only incurred in IT processes, that is, without adding the possible impacts on the business, which often result in tangible and intangible losses on unaffordable scales.

– And the most “impressive” data… 50% of most organizations’ IT effort and budget are spent on maintenance tasks, due to problems related to poor performance, errors, or problems with their applications (internal or external), according to us says techrepublic.com.

Have you calculated the direct costs, the intangible impact, and the reputational and commercial damage that this type of incident can cause in your business?

Implementing an efficient Quality Management strategy is essential to minimize this impact, with proven data that guarantees, from the start, significant increases in customer satisfaction, for example, when they have an excellent online experience with your brand and applications or your company’s website; reduction of releases and versions of the developed application; significant reduction in critical incidents, increasing levels of user confidence (internal or external) in the reliability of the systems and even the information they generate or the significant reduction in time-to-market, not to mention the effective cost reduction, related to maintenance, carrying out redundant tests or other types of inefficiencies related to their systems and applications.

Test Automation

Automation is one of the top companies’ priorities nowadays and is one of the biggest trends in the technology industry. Some even speak of Hyper Automation to describe the central role that automation will play in the organizations of the future. Also, in Quality, the automation of software tests is already a reality. All, or almost all, the large companies that have applications and systems critical to their business, are looking to automation as a way of reducing effort, reducing the performance of manual tests, optimizing resources, and, above all, reducing application development times and associated costs.

Despite the obvious gains that Test automation represents, it should not, however, be a “miraculous” solution in the context of quality and should be implemented gradually and always considering an important KPI – ROI (Return on investment). Automation costs must be taken into account, and it is very important to estimate the balance between effort-investment benefits of implementing an automation solution. It is necessary to consider the effort and frequency of development of the application in question, its criticality to the business, the support/maintenance it will require, and the test tasks that are carried out manually.

In short, it is increasingly important for organizations to adopt a Quality Strategy, to guarantee a faster arrival to their target audience, with reliable technological solutions – ensuring its functionality and a good customer experience. Failure to achieve this goal is nowadays penalizing for most organizations, and the short/medium term will simply be decisive in their ability to survive in a market increasingly eager for adequate solutions and made available at the right times.

Noesis is exhibiting at EuroSTAR 2021 Online. Looking to learn more on test automation? Check out the EuroSTAR 2021 programme, and join us September 28-30th.

Author

Eduardo Amaral, Quality Management Director at Noesis