|
|
Features The Reality of Software Testing in an Agile Environment
Ten QA Myths Blown Apart
By: Kate Mackinder
Aug. 11, 2009 05:15 PM
Introduction Agile is a methodology that is seeing increasingly widespread adoption and it is easy to understand why-especially if you consider the developer and user point of view. Users: Don't want to spend ages being quizzed in detail about the exact requirements and processes for the whole system, and then have to review a large specification, which they know could come back to haunt them. Developers: Don't want to have to follow a tight specification, without any expression of their own imagination and creative talents, especially if they can see a better way. Yet for the QA professional an Agile approach can cause discomfort - In the ideal world they would have a ‘finished' product to verify against a finished specification. To be asked to validate a moving target against a changing backdrop is counter intuitive. It means that the use of technology and automation are much more difficult, and it requires a new approach to testing, in the same way that it does for the users and the developers. All the agile approaches have (at least) one characteristic in common in that they impact the role of the QA professional. This in itself is not a bad thing when the outcome is a step change for the better. However, when decisions are made on the basis of an invalid paradigm, change is not always analogous with progress. When a new paradigm is proposed for software development, by software developers, it is not a surprise that it is developer-centric. Abraham Maslow said that "He that is good with a hammer tends to think everything is a nail." The responsibility of the QA profession is not to bury its head and pretend that agile development will go away, it is our responsibility to engage in discussion to ensure that someone with a hammer is not pounding on a screw! With the emergence of Test Driven Development** some suggest the role of QA is now questionable citing Test Driven Development (TDD) as the key to testing. But, what is most important is that QA is directly involved in the agile scrums all the way through, to be an integral part of the team designing the tests, at the same time as the requirements and solutions evolve. QA teams need to know the real impact of an Agile methodology, there are boundless myths circulating the industry. Here is our reply to ten of our favorite myths: Ten QA Myths Regarding Agile Testing. Myth One: "You only need to unit test - TDD testing is sufficient" TDD programmers rely on these tests to verify their code is correct. If the requirements (test cases) are specified incorrectly, then you'll build robust code that fails to meet the objective. Therefore, most agile projects include investigative testing efforts (black-box) which support rather than compete with white-box testing. Good investigative testing will reveal problems that the developer missed before they get too far downstream. Myth Two: "You can reuse unit tests to build a regression test suite" Although this sounds feasible, it isn't realistic, and here's why: The granularity and objectives of white-box unit tests developed in TDD serve a different purpose from downstream black-box testing. While the overall objective of a unit test is designed to prove that the code will do what is expected, the aim of regression testing is to ensure that no unexpected effects result from the application code being changed. These two objectives are not synonymous - e.g. checking that an attribute has a valid date format, is not the same as checking that for a given input, the value of the field contains an expected date. Myth Three: "We no longer need testers, or automation tools" It is widely recognized that traditional test automation tools have failed to live up to the hype of the vendors. Original Software's roots are as providers of products that improve the productivity of the database testing environment. It was because there were no adequate tools to provide User interface testing that we extended our solutions into this market sector; our heritage is aligned to effective testing rather than screen-scraping automation. When we developed TestDrive, we did it with the benefit of twenty years hindsight of missed opportunities from the traditional vendors. Often, TDD projects have at least as much test code as application code and, therefore, are themselves software applications. This test code needs to be maintained for the life of the target application. Myth Four: "Unit tests remove the need for manual testing" By automatically capturing the tester's process and documenting their keystrokes and mouse-clicks, the tester will have more time for the interesting, value-add activities, such as testing complex scenarios that would be difficult or impossible to justify automating. Though manual testing is a time-consuming (and therefore expensive) way to find errors, the costs of not finding them are often much higher. Myth Five: "User Acceptance Testing is no longer necessary" Myth Six: "Automation is impossible" To begin with, almost all testing from a user and QA perspective will be manual but this testing effort and design can be beneficial later if captured and re-used. Knowing the right time to automate is tough, so using technology that proactively supports early manual testing but provides a path to evolve this into automation is key. Myth Seven: "Developers have adequate testing skills" An independent testing team serves as an objective third-party, able to see "the big picture", to validate the functionality and quality of the deliverable. While developers tend towards proving the system functions as required, a good tester will be detached enough to ask "what happens if...?" When you include business user testing as well, you are more likely to have a system that is fit for purpose. Finally, and I'm sure this point will be disputed, most developers don't actually want to spend much time first writing tests and then developing code to prove the tests work. Using the collaborative process described below, the developer gets ample assistance in describing the functional tests and can focus on writing lean, accurate and robust code. Myth Eight: "The unit tests form 100% of your design specification" Defining test cases to check the accuracy and succinctness of the requirements is nothing new. The V-model, for example, is a valid approach to understanding testing requirements - and by implication, functional requirements. Like TDD, the V-model and most other models, fall down when the practitioner's approach is rigid, while development processes are fluid. Software engineering is not like mechanical engineering and trying to force conformity is wasted effort. Whichever approach you chose, correct thinking challenges each user requirement by asking "how would I test that?" The important factor here is that test cases are challenged before they are committed to code, otherwise you'll be doing a lot of recoding and calling it "refactoring". When the requirements are refined through collaboration, the developer receives a more robust specification that is less likely to change because it has been openly appraised from several people's perspectives. Myth Nine: "TDD is applicable on every project" Integration Testing - "Which tests do I need to run to ensure the new code works seamlessly with the surrounding code?" System Testing - "Does the functionality supported by the new code dovetail with functionality elsewhere in this system, or in other systems within the process flow?" Regression testing - "How often do I need to run a regression test to ensure there are no unforeseen impacts of the new code?" Automated regression testing provides a safety net that can affirm agile development techniques. User Acceptance Testing - "While TDD (in collaboration with business users) should ensure that a specific function performs correctly, is the cumulative impact of changes still acceptable to the business users?" However, in today's environment it is unacceptable to think of these testing stages as discrete serial activities. Often they will need to be run in parallel as we get a new ‘code drop'. As the size of the project team increases (along with testing effort), it is no longer acceptable for the tests to be considered "self-documenting". Whenever the number of participants increases, the project's risks become exposed to its members' different interpretations of the requirements - the definition of what constitutes "success" can be misconstrued. As the size of the project increases, so would the amount of test code that needs to be developed. Any test code developed needs to be supported for the life of the application - effectively doubling the ongoing maintenance effort. As the size of the testing workload increases the project needs to add test automation to its armory, to minimize the human intervention and elapsed times for each of these testing cycles. Myth Ten: "Developers and Testers are like oil and water" The discussion should be focused on:
Just as improvements in developer's software tools and methods have enabled a shift in development approaches, next generation technology for test automation is similarly reframing the opportunities for testers to automate earlier in the delivery cycle without incurring the heavy burden of script maintenance so often associated with traditional automation tools. Conclusion * Definition taken from The Glossary of Software Testing Terms from Original Software (www.origsoft.com) **Defined on Wikipedia as "A software development technique consisting of short iterations where new test cases covering the desired improvement or new functionality are written first, then the production code necessary to pass the tests is implemented, and finally the software is refactored to accommodate changes." |
|||