Bloggo back to the blog
Test Data Masking and Tool Pilot Experiences-->
In the first part of this series we discussed the challenges regarding data protection and criteria used to evaluate a number of test data management tools for data masking purposes.
Ultimately a tool was chosen based on its score in technical characteristics and economical evaluations. The chosen Data Management Tool’s components were used to provide the necessary ETL functions for Data extraction/Anonymisation for the extraction:
– Selection Criteria Module – defines data to extract
– Data Privacy Module – defines the rule to apply to transform the extracted data
– Data Convertor Module – defines the output format and schema
Once extracted from the production data set a “Gold standard” data extract was loaded to the Test and Development Environments and then used to populate the various testing instances. Data sub-sets were created and loaded into various targets. A repository was defined that contained all data transformation rules to be used for data extraction and loading.
As a part of the proof-of-concept the team was able to meet the data privacy and fraud, risk and security guidelines and received approval for the tool usage. This was done in a localised environment emulating production-like database environments and loaded onto test environments.
The tool you choose to anonymise may need to have:
– Features to simplify test data creation and have features to protect sensitive data by data masking, translating, generating, aging, analysing and validating test data against a set of requirements.
– Ability to implement different types of encryption algorithms to anonymise and obfuscate data and have a feature to work with all standard file types regardless of record length or file format.
– Ability to integrate with middleware APIs to inject data using XML to create “semi synthetic end-to-end data creation” by sourcing synthetic extracts from production and loading them into downstream systems.
– During our pilot this resulted in key productivity improvement for the test data management team.
– Effort savings were in the vicinity of about 0.5 FTE per week for the 20 data delivery processes getting automated (against a base test data management team size of 3, a 17% effort reduction).
– Continuous benefit realisation as we are implementing more automated processes using the tool
As per the earlier blog written by my colleague Matthias Rasking ‘Why Testing Data is Difficult’, Test data management is becoming complicated for newer and upcoming areas such as big-data , social media, mobility and cloud based testing – new generation and data gathering techniques using tools/languages such as Hbase, Hadoop, Hive, Cassandra, BigTable, Voldemort, Dynamo, NoSQL etc – may take the data collation and creation to next levels.
When you add the complexity of unstructured data with security requirements, having the “holy grail” for good quality test data becomes a challenge. With the advances in automation, tools and intelligent planning, this task is getting easier and better.
Accenture and Author Profile
Accenture is a global management consulting, technology services and outsourcing company with more than 266,000 people in 54 countries. Accenture Test Services has been providing testing services for more than two decades, both on-site and through our Global Delivery Network with more than 16,000 dedicated testing professionals.
Kalilur Rahman is a Senior Manager working for Accenture. Kalilur is a seasoned IT professional with 17+ years of experience in Telecommunications industry working for Wire line, Wireless, Cable, MSOs, Terrestrial Carriers, Industry standardisation leaders across myriad of telecommunications functional areas. He has played vital IT Delivery roles in complex programs in roles covering all SDLC phases. Currently he is heading a large Test Centre of Excellence at offshore. In his recent role as an End-to-end Program Test Manager for a large Retail Logistics transformation program, Kalilur led Oracle Retail Rollout for a global mobile telecom giant’s UK Geography. During the test delivery of world’s fastest Oracle Retail rollout (at the time of writing this article), Kalilur led the test delivery team supporting client’s testing service towards TMMi level-4 accreditation. Additionally Kalilur has delivered test transformation road-map and delivered testing service transformation tools using a global delivery model. Kalilur was a key/founding member of a large end-to-end testing deal for a leading communications company. As a firm Believer in “Knowledge is Power”, he is passionate about writing and sharing his knowledge. He is an active member in TeleManagement forum and contributed to a lot of internal initiatives in the organisations he worked for.