Tips for Automated Testing of New Software Product Releases, Part I

TestComplete is a little bit like chess. The basics of test script development with the tool are easy to learn, but there is a certain artistry involved in establishing a test suite that is value-add and maintainable. Here are some tips for building your automated test suites with TestComplete:

Tip #1. Follow general software development best practices. Developing a test suite is not all that different from developing a program in Java, .NET, or C+. Many of the theories behind efficient software design are applicable when building your TestComplete scripts. One good example of this is the use of reusable, parameter-driven functions and test items. If you have a sequence of test steps that is common to many of your individual test cases, it is better to encapsulate these test steps within their own test item, and to call this test item from each test case where those actions are to be performed. Activities such as logging in to your application, logging out, and navigating the top level menu should all be compartmentalized in this way.

Tip #2. Get the database into a predictable state at the start of the test execution. An automated test script can be rendered ineffective if you do not have a good strategy in place for maintaining the integrity of the test environment's database. For example, imagine you have a test script that inserts a new customer into the customer database. If you ran this test yesterday, and ran the test again today on the same database, the application might display an error messages to the user indicating that the customer already exists. The test script will fail without effectively validating the behavior of the customer insert functionality of the system! One good strategy for ensuring that the database is in a predictable state is to build a TestComplete test item that initiates a database restore operation on the environment you are using for testing, using the most recent production backup. You can then invoke this operation as the first step of every testing suite you assemble.

Tip #3. Create pre-requisite testing data. Many of your more complex test cases may be of the "scenario-based" variety. You will want to validate that your system produces the proper result in a wide array of controlled test scenarios. Each scenario may correspond to a particular configuration of records in the database. For example, you may want to test a customer billing functionality for a customer whose account is in good standing, versus a customer who is late on several payments and owes interest. There are three approaches that can be utilized to ensure that the pre-requisite testing data exists before your test script runs:

a. You can create automated TestComplete scripts to create all of the pre-requisite testing data in the system, and invoke these scripts first before your scenario-based tests are to be run. This solution works well for organizations that do not have a great deal of transparency into the data model. The primary disadvantage to this approach is that it is not the most efficient; it requires the most test-development and test-execution time of the 3 approaches.

b. You can create database stored procedures designed to insert/update the pre-requisite testing data in the system, and then call these stored procedures from within a TestComplete test item before your scenario-based tests are to be run. Using the example above, you might create a stored procedure called "sp_InsertNewCustomer(customerName, outstandingBalanceDue)", and call this stored procedure two times – once to create a customer with an account in good standing, and once to create a customer with an outstanding balance due. This solution works well for organizations that have transparency and expertise in the data model, and where the scenarios include a large number of worst-case or what-if scenarios that are expected to rarely manifest in the production database. The primary disadvantage to this approach is that it requires a commitment of time from an experience database developer.

c. You can create database stored procedures designed to search for and retrieve pre-requisite testing data by inspecting pre-existing records copied over from the production database. You would then create a TestComplete test item to call the stored procedure and load the result into a local variable within the TestComplete project. Using the example above, you might create a stored procedure called "sp_GetCustomerInGoodStanding()" and another stored procedure called "sp_GetCustomerWithBalanceDue()", both of which would return a customer ID. Subsequent test scripts could read the value of this local variable, and use it to find the appropriate record within the system. This solution works well for organizations that have transparency and expertise in the data model, and where the scenarios to be covered in testing are expected to already exist. The primary disadvantage to this approach is that it requires a commitment of time from an experience database developer.

We will continue this series with additional automated testing tips in later posts. Please stay tuned!

To learn more about WinMill Software's application development best practices and how we help make your project a success, contact us at inquiry@winmill.com or (888) 711-6455.