« November 2004 | Main | January 2005 »

December 28, 2004

3. Basic Patterns

This group of patterns applies object-oriented programming principles to reduce test code maintenance costs and facilitate new test creation. The patterns included are Template, Object Genie, Domain Test Object, and Transporter.

Posted by Misha Rybalov at 10:42 PM | Comments (0)

December 26, 2004

4. Basic Pattern #1 - Template

Definition

Group a common sequence of steps for testing a module into a class and have subclasses specify unique test case data without changing the main algorithm's structure.

Discussion

Customer tests involve navigating to a particular part of the application and trying different scenarios. Even though each scenario is unique, there are still many commonalities between test cases. For instance, the way that you navigate to the module and the assertions that are made along the way can be grouped into one place - the Template class, which is a test-specific application of the Template Method design pattern.6

When the application inevitably changes, the majority of updates will occur in the Template class instead of within the test cases. This is because the Template class encapsulates navigation and assertion logic, which is more vulnerable to application changes than the data contained in test cases.

After each update to the Template class, the modifications will cascade automatically to all subclasses. This has the added benefit of making each test case (subclass) simpler since it is only required to specify the unique aspects of the test instead of the entire sequence of test steps and assertions. This reduces test code duplication and standardizes common navigations and assertions of related test cases.

If not using this design pattern, then every time the application changed or a new test case was added, the test writer would have to copy or re-record all the navigations and assertions that are part of each test. In that case, the automation effort would incur serious maintenance costs.

Since the Template pattern is adaptable to application changes, it can be used early in the development cycle without having to wait for application completion. Templates can be created and then continually updated as features are developed. This lends itself to agile software development practices such as Test Driven Development7 where tests are written early in the software lifecycle and are an integral part of the development effort.

Example

Figure 1 presents two traditional tests which are then contrasted with Figures 2 and 3 which use the Template pattern. These tests check the functionality of adding and removing items from a shopping cart.


Figure 1: Non-template test cases.




import junit.framework.*;



public class ShoppingCartTest extends TestCase {

	public ShoppingCartTest(String pName) { super(pName); }

	public static Test suite() { return new TestSuite(ShoppingCartTest.class); }

	public void testAddItem() {  //1st test case

		login();

		searchForItem1();

		addItem1ToCart();

		searchForItem2();

		addItem2ToCart();

		logout();

	}

	public void testRemoveItem() {  //2nd test case

		login();

		searchForItem1();

		addItem1ToCart();

		addItem1ToCart();

		removeItem1FromCart();

		searchForItem2();

		addItem2ToCart();

		removeItem2FromCart();

		logout();

	}

	//test tool specific implementations of methods 

	//(e.g. login(), searchForItem1(), etc.)

}

The first test logs in, searches for item one, adds it to the cart, searches for item two, adds it to the cart, and logs out. The second test logs in, searches for item one, adds it twice to the cart, removes one quantity of item one from the cart, searches for item two, adds it to the cart, removes item two from the cart, and logs out. These two traditional tests have the following common elements: logging in, searching for item one, searching for item two, and logging out. These common steps can be moved into a new Template class.


Figure 2: Template class.




import junit.framework.*;



public abstract class ShoppingCartTemplate extends TestCase {

	public ShoppingCartTemplate(String pName) { super(pName); }



	protected void setUp() {

		login();

		searchForItem1();

		cartAction1();  // <------ test hook 1

		searchForItem2();

		cartAction2(); //<-------- test hook 2

		logout();

	}

	

	//Subclasses are supposed to implement these test hooks anyway they like.

	abstract protected void cartAction1();

	abstract protected void cartAction2();



	//test tool specific implementations of methods 

//(e.g. login(), searchForItem1(), etc.)

}

The Template defines the main sequence of steps as 1) login, 2) search for item one, 3) do something (to be defined by subclasses), 4) search for item two, 5) do something else (to be defined by subclasses), and 6) log out. Note that JUnit's setUp() method is used to define the sequence of steps, which enables this sequence of steps to occur for every test case subclass.

Each test case can now be moved to a separate subclass and implement the abstract methods of the Template.


Figure 3: Two test cases using the Template.




public class ShoppingCartTest_AddItem extends ShoppingCartTemplate {

	public ShoppingCartTest_AddItem(String pName) { super(pName); }

	protected void cartAction1() {

		addItem1ToCart();

	}

	protected void cartAction2() {

		addItem2ToCart();

	}

}



public class ShoppingCartTest_RemoveItem extends ShoppingCartTemplate {

	public ShoppingCartTest_RemoveItem(String pName) { super(pName); }

	protected void cartAction1() {

		addItem1ToCart();

		addItem1ToCart();

		removeItem1FromCart();

	}

	protected void cartAction2() {

		addItem2ToCart();

		removeItem2FromCart();

	}

}

The Add test case (ShoppingCartTest_AddItem) implements the two abstract methods by adding items one and two to the cart, respectively. The Remove test case (ShoppingCartTest_RemoveItem) implements the first abstract method by adding item one twice and then removing it. It implements the second abstract method by adding item two to the cart and then removing it.

Even though it appears that there is not a great reduction in code size in this example, code consolidation becomes more evident in real life scenarios which contain dozens of test steps and hundreds of test cases.

Adding more tests is relatively easy with the Template pattern since one only needs to implement the two abstract methods of the Template (i.e. cartAction1() and cartAction2()) rather than having to write all the setup and navigation code. This consolidation of redundant test steps into one place can drastically reduce maintenance costs as the application continues to evolve. For instance, if the application were to change so that customers now had to go to a specials screen before being able to add any items to the their cart, our Template can be updated to account for this and have the change cascade to every test without having to modify each test individually.


Figure 4: Modification of Template.




import junit.framework.*;



public abstract class ShoppingCartTemplate extends TestCase {

public ShoppingCartTemplate(String pName) { super(pName); }



protected void setUp() {

login();

gotoSpecials();    // <------ new test step added

searchForItem1();

		cartAction1();     // <------ test hook 1

		searchForItem2();

		cartAction2();     // <------ test hook 2

		logout();

	}

	abstract protected void cartAction1();

	abstract protected void cartAction2();

}

Posted by Misha Rybalov at 09:04 PM | Comments (0)

December 22, 2004

5. Basic Pattern #2 - Object Genie

Definition

Relieve tests from having to instantiate and initialize common non-trivial objects by having them ask for those objects from a central authority that grants objects in a pre-defined state. This pattern is also known as ObjectMother.8

Discussion

There are many occasions during testing when multiple tests will require the same object and then each proceed to interact with it in different ways. If it is a simple object, each test can just instantiate it normally (SomeObject object = new SomeObject();) However, there are many occasions where the object that is needed by tests is cumbersome to instantiate or initialize.

The Object Genie consolidates the construction and initialization of that object to just one place, thereby reducing future maintenance costs and facilitating the creation of new tests. If the way an object is instantiated or initialized ever changes, the update would only be done in the Object Genie and the test cases that use that Object Genie would be unaffected.

Example

In our shopping cart application, multiple tests (e.g. test deleting an item from a cart, test the checkout process, test processing a coupon) require a shopping cart with a few items in it. Without the Object Genie pattern, each test would have to re-create and re-populate the shopping cart, increasing code duplication and maintenance costs. Using an Object Genie, a test can request a non-empty shopping cart and then proceed with testing. The Object Genie handles the instantiation of the shopping cart and populates it. In Figure 5, the Object Genie populates the shopping cart with two items (orange juice and cereal).


Figure 5: Defining a shopping cart Object Genie.




public class CartGenie {

     public static ShoppingCart getNonEmptyCart() {

          ShoppingCart shoppingCart = new ShoppingCart();

          Item ojItem = searchForItem("orange juice");

          shoppingCart.add(ojItem);

          Item cerealItem = searchForItem("cereal");

          shoppingCart.add(cerealItem);

          return shoppingCart;

     }

     private static Item searchForItem(String pItemId) { 

          //search for the item based on the item id 

     }

}

Each test would call CartGenie.getNonEmptyCart() to receive an initialized shopping cart and then proceed with the details of the test. The tests would be decoupled from having to know how a shopping cart is instantiated and populated, thereby reducing future maintenance costs (see Dumb Test in Section II: General Design Principles).

Posted by Misha Rybalov at 10:31 PM | Comments (0)

December 16, 2004

6. Basic Pattern - #3 Domain Test Object (DTO)

Definition

Encapsulates an application's visual components into objects that can be reused by many tests.

Discussion

We use object-oriented programming (OOP) to model real world objects inside our applications. We can also use OOP to model visual representations of the application (e.g. a shopping cart, a coupon, and a catalog) in our tests. In other words, the application uses objects to model the real world, while the tests use objects to model the application's world.

The DTO pattern groups visual representations of the application into objects that can be used by tests. Each test communicates to the DTO what data is expected but not how the data should appear (i.e. order and name of input fields to which the data applies or placement of the data on the page). The DTO then checks the test provided data against the actual data in the application. Thus, the DTO allows us to decouple the expected data from its visual representation. When the visual representation of the application changes (e.g. new table columns, new fields, and new labels), the update is done to the DTO instead of to each test. All the tests that use that DTO will be insulated from the change and will not need updating, thereby reducing maintenance costs. In essence, the DTO is a test-specific application of the Model/View/Controller (MVC) design pattern,9 where the tests are the Model and the DTO is the View.

Example

Customer tests need to check the contents of the shopping cart after certain actions are performed (e.g. adding, removing, updating quantity of cart items). A simple table can represent the shopping cart by displaying the list of items, quantity, unit price and total price.


Table 1: Visual representation of shopping cart contents.

Item Quantity Unit Price Price
Milk 1 $2.99 $2.99
Bread 2 $1.50 $3.00
       
Total     $5.99

This table is the application's representation of a shopping cart. Every time a test wants to check the cart's contents it must check this table. Thus, any changes to this table (e.g. a new column being added) could have severe test maintenance consequences. To minimize this maintenance cost, we can model the shopping cart table as a DTO and have tests interact with the DTO instead of with the table directly. The following code sample demonstrates checking the shopping cart page without using a DTO. This is then contrasted with doing the same check using a DTO.


Figure 6: Checking shopping cart contents without DTO.




public void checkCart_noDto() {

     String [][] expectedCartTableCells = {

          {"Item",  "Quantity", "Unit Price", "Price"},

          {"Milk",  "1",            "$2.99",      "$2.99"},

          {"Break", "2",           "$1.50",      "$3.00"},

          {"",           "",             "",               "",    },

          {"Total",  "",              "",              "$5.99"}

     };

     //tool-specific way to retrieve table cell contents

     String [][] actualCartTableCells = getCartTableCells(); 



     //check every web table cell of the cart page

     assertEquals(actualCartTableCells.length,  expectedCartTableCells.length);

     for(int i=0; i
          assertEquals(expectedCartTableCells[i].length, actualCartTableCells[i].length);

          for(int j=0; j
               assertEquals(expectedCartTableCells[i][j], actualCartTableCells[i][j]);

          }

     }

}

Without using the DTO pattern, we are forced to specify the expected content in a rigid order that is vulnerable to application changes. Every time we need to check the contents of the shopping cart page, we would have to repeat the same checking code, but with different values. For example, we would need to replicate the same checking code after adding to the cart, removing from a cart, updating the quantity of items in the cart, etc. The maintenance problem occurs when the layout of the cart changes and we would have to update all the checking code for each cart action. For instance, the application might change so that the shopping cart page has a new column called "Discount" which contains percentage values of the discount for an item. The application might also change by having the quantity and unit price columns switched. In both of these cases, all our test code would break because now the web table wouldn't match our expected table values. By using a DTO, we can avoid this problem and be more adaptable to application changes. The following code demonstrates how a DTO object can be used:


Figure 7: Checking shopping cart contents with DTO.




public void checkCart_withDto() {

     ShoppingCart shoppingCart = new ShoppingCart();

		

     //...add/remove/edit cart using other patterns

		

     ShoppingCartDto shoppingCartDto = new ShoppingCartDto();



     //setup expected values

     shoppingCartDto.setItem("1", "2.99", "2.99", "Milk");

     shoppingCartDto.setItem("2", "1.50", "3.00", "Bread");

     shoppingCartDto.setTotal("5.99");

     shoppingCartDto.check(shoppingCart);

}



public class ShoppingCartDto {

     public void setItem(String pQuantity, String pUnitPrice, String pTotalPrice, String pName) {

     //code to store expected item internally would follow

     }

	

     public void setTotal(String pTotal) { 

          //code to store expected total internally would follow

     }

	

     public void check(ShoppingCart pShoppingCart) { 

          //code to check expected versus actual shopping cart contents would follow

     }

}

From Figure 7, you can see that the DTO is a separate class that is instantiated and configured by test cases. Note that the DTO knows how to check itself so tests never have to explicitly specify the column order or any other details about the shopping cart table. The tests simply tell the DTO the items that are expected to be in the cart and then ask the DTO to check the cart contents. Now even if the entire shopping cart page changed so that the cart is no longer represented by a table, the update would be done to the DTO while leaving the tests insulated from such a drastic change.

Posted by Misha Rybalov at 05:22 PM | Comments (0)

December 10, 2004

7. Basic Pattern #4 - Transporter

Definition

Create a central authority that navigates through the application on behalf of tests.

Discussion

Similar to the Object Genie is the concept of a Transporter. Whereas the Object Genie grants tests whatever objects they need, a Transporter encapsulates navigation code so that tests don't have to know the details of how to move around the application. Instead, tests use the Transporter to navigate throughout the application on their behalf.

If the application changes so that users must now alter the way they navigate to a specific module, the modifications would be localized to the Transporter and changes would automatically cascade to all tests that use that Transporter. This reduces maintenance costs since none of the tests would be affected by this change and facilitates creating new tests since there is less test code to write.

Example

Each test needs a way to navigate to pages and then perform specific actions (e.g. log in, enter a coupon, buy an item on special, etc.). To use a coupon, one must first do the following navigations: login and navigate to the checkout page. There must also be items in the cart to which the coupon applies - this is taken care of by the Object Genie. The following code samples demonstrate the creation and use of a Transporter that navigates to the login and checkout pages.


Figure 8: Defining a Transporter.




public class Transporter {

     public static void login() {

          //Tool specific code to navigate to login screen and login to the application

     }

     public static void gotoCheckout() {

          //Tool specific code to navigate to the checkout page

     }

}


Figure 9: Using a Transporter.




import junit.framework.*;

public class CouponTest extends TestCase {

     public CouponTest(String pName) { super(pName); }

     public static Test suite() { return new TestSuite(CouponTest.class); }

     protected void setUp() {

          Transporter.login();                 // <---- loose coupling

          ShoppingCart shoppingCart = CartGenie.getNonEmptyCart();

          Transporter.gotoCheckout();   //<----- loose coupling 

     }

     public void testCheckoutWithValidCoupon() {   //tests for a valid coupon

          applyValidCoupon();

          checkCouponAccepted();

     }

     public void testCheckoutWithExpiredCoupon() { //test for an expired coupon

          applyExpiredCoupon();

          checkCouponRejected();

     }

     public void testCheckoutWithInvalidCoupon() {  //test for an invalid coupon

          applyInvalidCoupon();

          checkCouponRejected();

     }

     //… specific implementations of test methods (e.g. applyValidCoupon())	

}

By moving navigation to the login and checkout pages to a central location, we allow other tests to reuse that functionality. The CouponTest class is now only responsible for testing coupon functionality without worrying about the details of how to get to the coupon page. This loose coupling makes the test more maintainable.

Posted by Misha Rybalov at 05:08 PM | Comments (0)

December 04, 2004

8. General Design Principles

These patterns introduce general test design principles that can be used in conjunction with the basic patterns introduced previously. This group includes Dumb Test, Independent Test, Don't Repeat Yourself, and Multiple Failures.

Posted by Misha Rybalov at 06:17 PM | Comments (0)

December 02, 2004

9. Design Principle #1 - Dumb Test

Definition

Minimize a test's knowledge about navigation, checking logic and user interface details.

Discussion

This principle is commonly used in application programming where decoupling objects from each other allows each object to change without affecting the other. Applying this principle to test development, the Dumb Test pattern focuses on making an individual test know as little about its surroundings as possible. This makes Dumb Tests less vulnerable to changes in the application, which in turn lead to lower test maintenance costs. Each Dumb Test doesn't know the sequence of steps that were taken to get to the component being tested nor about common checking that has been done by other test modules. Ideally, the only things a Dumb Test is aware of are its inputs and its expected outputs.

Dumb tests work well with the Template, Object Genie, DTO, and Transporter patterns since common test steps can be consolidated into these classes. When the application changes, the bulk of the updates will need to be made in those classes, while individual Dumb Tests will be mostly insulated from the changes. This creates very compact tests that are quick to create and cost less to maintain than tests that provide all the details about navigation and assertions. In a nutshell, a Dumb Test doesn't see the big picture.

Example

Both test cases in the Template pattern example are Dumb Tests (see Figure 3). They have no knowledge of how to login, logout, go to the specials page or search for items. If these parts of the application were to ever change, the Dumb Tests would be unaffected by it. The tests only know that they need to add and remove items from the cart.

Posted by Misha Rybalov at 11:18 PM | Comments (0)