Test Data Management with SmartQE

Test Data Management with SmartQE

SmartQE helps organizations to automate test data creation with an optimal human intervention. Multiple and frequent releases require a lot of test data to be created and consumed. Agile development increases the creation and consumption of more test data during QA and testing services. SmartQE helps Test data creation dynamically and create at run-time to save extraction and creation time.

SmartQE is a containerized Cloud based SaaS open source framework for mobile native, hybrid and Web platform code inspections and validate against standards. SmartQE can easily integrate CI/CD platforms and tools.

Automate Test Data Creation by SmartQE

SmartQE helps to create data creation in three different ways

1. Configure non prod database with appropriate access and credentials into SmartQE. Access table and columns from DB to create data set and configure to scripts as input test data. Automation scripts get data from DB table columns at run time. This is one of the suggested solutions where data privacy policies are applied
2.  Test data should be extracted or created through API scripts or through SQL stored procedures. This greatly enhance the testing efficacy. This is one of the suggested processes and it’s not a one solution for all scenarios
3. Understand the data requirement and automate those data requirements with an open source tools like SmartQE and write the generated data into a file. Map the file data is an input for automated test scripts. When regression test execution job start, first executes test data generation script, update data file with new test data before start execute main regression scripts every time.

• Certification - Certifying Master Data based on Data criteria for each Test Scenario

• Extraction - If Master Data Certification Fails, Extract the Data based on Data criteria

• Update - Update the Extracted data and make a golden copy for multiple cycles

SmartQE can integrate TDM tools such as IBM Optim, GridTools, DataMaker, QuerySurge

Test Data Management Challenges

• IT enterprises spend 30% of their time & effort in resolving issues related to managing test data

• Cannot reproduce a defect as the test data has been changed from the time of previous execution

• Extract test data which doesn’t meets test condition

• How can we quickly re-create or modify data when it is consumed by test execution

• Synchronizing multiple systems and applications

• Regulatory and compliance policies around FDA, HIPAA, PCI

• Unauthorized access and changes

• Impact of refresh on projects that are in flight

• Traditionally, data used for testing is captured and maintained in spreadsheets. Searching and sorting through this data is difficult

• Test data “re-usability” is minimum

• Non availability of Test Data - Testing will get delay and not completed on time if the Test Data required is not ready before the start date of Testing.

• Lack of processes to dispose of the test data when not needed

• A huge volume of data for validating the business processes

• Managing the multiple sources & formats of data as required for testing

• Managing the changes required to share data that may affect multiple projects

• Availability of the test data range required for end-to-end testing & coverage for all possible combinations.

 

SmartQE’s Test Data Management Strategy

       Inputs:

1. Identify the source systems & data objects to prepare test data set up

2. Understand the test data requirements

3. Understand the relations between dependent fields across the tables for given input

4. List of Test Cases with detailed conditions

5. Systems and application information

6. Test data mining and setup procedures.

      Step Description:

• Identify various data sources and understand the data

• Determine which QA systems and business processes require refresh from production

• Determine which QA systems and business processes require test data creation and modification

• Determine which data require mining from the QA systems (business processes and transactions) and which data require clean up after every test execution cycle

• Identify parallel projects that are getting executed in the QA environment which results in data conflicts

• Identify data privacy, compliance and protection requirements

• Identify how test data will be consumed – number of test cycles that would be executed

• Identify areas that can be automated –

  • Test data creation and mining

  • Test data modification

  • Migration of small sets of data from production etc.

  • Aging and Masking

• Use Test Data Management Strategy template and complete sections using the information collected

 

Methods to Create Data	Pros	Cons
Cloning Production data	Easy to implement	Expensive in terms of hardware, license and support costs Time-consuming: Increases the time required to run test cases due to large data volumes Not agile: Developers, testers and QA staff can’t refresh the test data Inefficient: Developers and testers can’t create targeted test data sets for specific test cases or validate data after Test runs Not collaborative between DBA and testing teams Not scalable across multiple data sources or applications Laborious: Production systems are typically large Risky: Non-production environments might be compromised or misused (developers, testers and QA staff need realistic data to do their jobs—but they do not have a valid business reason to access sensitive data such as corporate secrets, revenue projections or customer information)
Creating data	Safe	Resource-intensive: Requires a huge commitment from highly skilled DBAs with deep knowledge of the underlying database schema, as well as knowledge of implicit relationships that might not be formally detailed in the schema Tedious: DBAs must intentionally include errors and set boundary conditions within the synthetic data set to ensure a robust testing process, which adds time to the test data creation process Challenging: Despite the time and effort put forth by the DBA to generate synthetic test data, testers find it challenging to work with because synthetic test data doesn’t always reflect the integrity of the original data set or retain the proper context Time-consuming: Process is slower and can be error-prone
Copying set of production data and modifying it	Less expensive when compared to cloning and creation of data	Skill-intensive: Without an automated solution, requires highly skilled resources to ensure referential integrity and protect sensitive data

 

SmartQE’s Advantages :

• Saves Time & Storage more than 30% and build one time use many times

• Builds complex queries to extract data satisfying business requirements

• Manages test data requests

• Performs Data standardization and synchronization

• Regulations and compliance

• Data privacy, threats, and breaches

• More aligned to Agile sprints

• Reduces debugging time on data issues and validation components

• Makes it easy to access and share information. Plug & Play UI, can be integrated with any tool

• Reduces execution cycles, minimizes data extraction efforts

• Avoids Data Issues and improve efficiency by 85%

• Increases total cost of ownership (TCO)

Improved :

• Testing effectiveness with accurate data

• Test coverage & test data quality assurance

• Data privacy through data masking techniques

• Risk mitigation through reduced data exposure

• Coordinated & easy-to-manage test data management process