Blog

go back to the blog

Example of Custom Test Tools from a Test Developer

  • 08/10/2012
  • 7160 Views
  • no comments
  • Posted by EuroSTAR
-->

Last year I was lucky enough to get invited to write for the EuroSTAR blog and I wrote a piece called “Why every system test department should have a test developer (…or two)”. This year I am privileged enough to have been invited as a speaker to EuroSTAR 2012 for the same topic.

When I talk about what a test developer could do for the testers in a system test department I give several examples of customized test tools that they can build, and I also claim that a semi-skilled test developer can build them in no time at all. But of course they can also put in as much time and effort as required depending on the amount of features that your testers find valuable for their testing.

In this blog post I have planned to show an example of a simple test tool you could get from your very own test developer, I have more examples for my upcoming EuroSTAR presentation:

/conferences/session/378/why-every-system-test-department-should-have-test-developer-or-two

The test tool I would like to write about today is: “data diggers”.

Data diggers:

A lot of times when we test a system that system is producing a lot of data, often saved into a log file. It can for example be incoming SSH connections, system events like CPU utilization, memory allocation and more, all sorted according to timestamps.

sys_events_1_399x155

And these log files can be very long and often all data is not gathered in only one log file. But it’s common instead that the data is spread over the system in different log files, for example log files containing data on money transactions for mobile phone subscriber’s accounts in the system.

msisdn_events_400x84

And when we perform certain test activities we are only interested in a subset of the data produced into the log files. It might be that I have noticed strange behavior with the systems memory consumption and I suspect it might be related to when the system does a refill of money into a subscriber’s account.

Now I have to perform my test case, and then after it has been executed I need to open up the log file looking for the memory allocations.

sys_events_2_399x154

And then after that I need to look at the log file for account refills, and note the timestamps there, and start trying to analyze what specific memory activities that occurred when a specific refill occurred (often on a separate pen a paper next to my computer).

msisdn_events_2_400x105

In this example we only have ~10 lines of log files, but remember that it can easily be several thousands of lines of logs in these files to look through.
And for humans this process of gathering data and sorting through it can be very tiresome and error-prone (unless you happen to have super-powers like “Specialisterne” – http://specialisterne.com/).

Gathering and sorting the data is something that computers and software is really good at. Analyzing the results and drawing intuitive conclusions is something humans can be great at, but something computers aren’t. So let’s use the computer for what it’s good at, so that we testers can focus on what we are great at.

For this example, the best approach is for your test developer to use some sort of scripting language like Python or Perl (absolutely no need for a compiled language, and shell scripts can become ugly very easily IMHO).
My weapon of choice in these situations is to use Python, since I’m most comfortable working with it, it has great built in libraries, and I like the expressiveness of the language and the clean code you can produce with it.

This way we can easily run our data digger during runtime/test time and see the filtered output in a separate terminal, all as we do our tests in real-time. We don’t need to wait until after the test has been performed to see what’s happening, thereby shortening the feedback loop.

data_filter_1_400x92

If your test developer is smart he/she writes the test tool with the possibility for you as a tester to specify which log files to monitor, and what data strings to look for. That way, you and the rest of the test department can reuse the test tool going forward making the short amount of time spent by the test developer a worthwhile investment.

data_digger_help_400x148

Just think about how much time, effort, frustration and human errors that can be saved with this type of very simple test tool.

Biography

photo_kristoffernordström_107x160Kristoffer Nordström is a test developer that works for Electric Cloud in San Jose, California. Electric Cloud is a company which specializes in accelerating and distributing the build of software and DevOps automation. He is also the owner and founder of Northern test Consulting.

His background covers development of large scale test automation frameworks, test tools development, and he has also worked as a test lead, scrum master and developer.

A twitter addict and loves to talk test. Can be found as: @kristoffer_nord

“I´m a test developer who is absolutely nuts about anything to do with testing, development or agile practices … I just can´t decide and pick a specialty. My blog can be found at http://contextdriven.blogspot.se/

Blog post by

go back to the blog

eurostar

Leave your blog link in the comments below.

EuroSTAR In Pictures

View image gallery