« Automated Testing is Very Manual | Main | Automated Unit Test Creation - AppPerfect »
February 11, 2005
Automated Unit Test Creation - Agitar
Today at work, we got a demo of a piece of software that automates the creation of unit tests for an arbitrary piece of Java code. After going through the obligatory slides, they got to the meat of the presentation by generating a whole slew of tests for a samples piece of code (they call this process "agitation"). While it was impressive, I'm still not fully convinced that it can do this intelligently with a system it knows nothing about. A lot of the test cases were boundary conditions for methods while others were garbage type inputs to see if the code would break. While this type of input testing is very important it doesn't cover the whole spectrum of tests that need to be generated.
It would be extremely helpful if this tool could generate tests with input data particular to an application. For instance, if I were using this tool on eBay's code, I'd want it to automatically create test auctions, test bidders, test sellers and so on. That way, you'd be exercising the application that's specific to the rules and objects of that application. In order to make this happen, the tool needs you, the developer, to write a slew of factories that create this application specific data.
It's still unclear to me whether the time taken to create all these factories is less than the time it would take to write out all the unit tests manually. If it is, then this tool has some real value. What's ironic is that at the end of the demo, their sales guy predicted people's skepticism by saying "you probably think this tool is not going to work on your program because you think your program is unique". I thought that was an accurate assessment of the situation. So he's going to come back in a couple of weeks time and redemo this tool against the company's program. Stay tuned...
I find this area fascinating since I feel this could be the next big thing to affect software quality. Automated test creation is one of those aspects of automated testing that has always remained manual (see my previous posts: Automated Testing is Very Manual) and I admire this company for attempting to bring this area to the same level of automation as test execution.
Posted by Misha Rybalov at February 11, 2005 05:01 PM