Se ha denunciado esta presentación.
Se está descargando tu SlideShare. ×

7 steps to Software test automation success

Cargando en…3

Eche un vistazo a continuación

1 de 15 Anuncio

Más Contenido Relacionado

Presentaciones para usted (20)

Similares a 7 steps to Software test automation success (20)


Más reciente (20)

7 steps to Software test automation success

  1. 1. 1 Ten Steps to Test Automation Success by Bret Pettichord Presented by Wen Yin
  2. 2. 2 Introduction • What is software test automation?  “The management and performance of test activities, to include the development and execution of test scripts so as to verify test requirements, using an automated test tool.”  Significantly reduce the effort required for adequate testing;  Significantly increase the testing which can be done in limited time.
  3. 3. Software Testing Life Cycle. • Requirement/Design Review. • Test Planning. • Test Designing • Test Environment Setup. • Test Execution. • Test Reporting. • Retesting .. 3
  4. 4. Test Selection • Choosing what and when to automate involves an analysis of the following :Automate items that are tedious for manual execution, but relatively easy to automate. • Items that need to be executed over and over again. • Items that will increase coverage beyond what will realistically be done manually • In addition, the decision depends on the goals of the organization such as cost goals, efficiency goals, and risk mitigation goals. 4
  5. 5. Automated Test Design and Development • Automated test design and development are tied directly to the interface in which the tests will be created, the way quality attributes are applied at the test level, the elements that are common to automated tests, and the standards used for creation of tests. 5
  6. 6. 6 Problems in test automation • Spare time test automation. • Misunderstanding the requirement. • Improper Test design. • Lack of clear goals. • Lack of experience. • Reaction to desperation. • Reluctant to think about testing. • Unable to reproduce the issues. • Lack of known about new technologies. • High turnover.
  7. 7. 7 Solution to the problem • Follow the rules of software development • Follow the seven steps to test automation success: – Improve the Testing Process – Define Requirement – Prove the Concept – Champion Product Testability – Design for Sustainability – Plan for Deployment – Face the Challenges of Success
  8. 8. 8 Step 1. Improve the Testing Process • Improve test automation documentation – Explicit test approach (names, data, test design, expected result) • Get more computers • Use advanced technology for in your testing tool. • Improve the product for easier testing – Improve the product install program – Add error scanner to the product – Identify and report slow functionality as a defect – Identify and report which are not as per as Requirements. – Write test cases for achieve the 100% of test coverage.
  9. 9. 9 Step 2. Define Requirements • Different parties have different automation goals. • Come to agreement on what will count as success. • Focus on getting biggest payback, not trying automate every part of tests.
  10. 10. 10 Step 3. Prove the Concept • Prove the feasibility of test automation as early as possible; • A quick, meaningful test suites; • Also an excellent way to evaluate tools • Candidates for proof of concept: – Regression testing – Configuration tests – Test bed setup – Non-GUI testing
  11. 11. 11 Step 4. Champion Product Testability • Asking early for testability as a product requirement; • Getting testability features designed right into the product; • Encouraging to include CLI or API; – GUI test automation is more difficult, expensive and undependable; – API and CLI are easier and cheaper;
  12. 12. 12 Step 5. Design for Sustainability • Focus on design of long term sustainability – test automation efforts are prone to being dropped. • Characteristics of a sustainable test suite – Reviewability – Maintainability – Integrity – Independence – Repeatability
  13. 13. 13 Step 6. Plan for Deployment • Package the automated tests for others to use; • Test your test suite; • Get others to use your test suite as soon as it’s ready; • Make your test suite widely available to the entire product team. • Make sure your test suit should be easily understand to your product team mates.
  14. 14. 14 Step 7. Face the challenges Success • Testers are lack of automated testing skill; • Improvement of test suite over time; • Timing for when test automation should be done
  15. 15. 15 Summary • Test automation brings us both hopes and disappointments. • Seven Steps to the Success: – Improve the Testing Process; – Define Requirements – Prove the Concept – Champion Product Testability – Design for Sustainability – Plan for Deployment – Face the Challenges of Success

Notas del editor

  • I am going talk about Bret Pettichord’s article “Seven Steps to Test Automation Success”. Bret Pettichord is a specialist in software test and test automation. He is also the founder of Austin Workshop for Test Automation. In his paper, he identifies the major problems when automating the software tests and presents seven key steps to software test automation success. He emphasizes that the key is to follow the rules of software development process when automating testing. I like this paper because it gives me a very practical guidance on test automation. Many examples are provided in explaining the concepts and supporting the author’s arguments. Most of these examples are from his personal working experience on test automation. I am very convinced of most the points he made in his article.
  • What is software test automation?....
    Software testing needs to be effective at finding any defects, but it should also be efficient, performing the test as quickly and cheaply as possible. Software test automation can significantly…
  • However,Test automation is not as easy as it seems to be: just buy one of the popular test execution tools, record the manual tests, and then play them back whenever you want to. The fable told in Bret Pettichord’s article shows us that implementing automated tests can create as many problems as it solves. Those problems that plague test automation projects includes:
    Spare …, which means staff are allowed to work on test automation on their own time. This keeps test automation from getting the time ant focus it needs. In the fable, when the first staff Jerry Overworked is assigned the job of automating the tests, he has many other responsibilities and he has to try the new tool between his other tasks. Therefore, no surprise that he failed his automation project.
    Lack of clear goals, Different parties have different good reasons for doing test automation. If those different hopes are not stated, disappointment is inevitable.
    Lack of experience, it is always a problem for most of the projects, not just test automation.
    Test automation is often a ready answer to a desperation of endless testing and of the upcoming deadline. However, it may be more of a wish than a realistic proposal.
    Reluctant to think about testing. Since many people find automating a product is more interesting than testing it, some automation projects’ contributors are not more involved in the testing. How the software can be automated is technologically interesting problem. When people focus on the technology of automation, they can also lose sight of whether the result meets the testing need.
  • The core of the solution to the problems of test automation is to run test automation project just as we do our other software development projects. Remember test automation is itself a software development activity and therefore it needs to adhere to critical development practices. You need dedicate resources to test automation, design test automation before coding, manage test automation source code, test the code, and document the test automation.
    With special considerations that particular to test automation, Bret recommends seven steps to the test automation success. They are ….
    Now let’s see the detail of the seven steps.
  • The first step to the success is to improve the testing process before starting to automate. If you want to improving the efficiency of a task, you first should make sure that the process is well defined. Same for test automation. You should look for some simple and cheap way to make things go easier before you invest the time and money in automating the system using computers. I totally agree with it.
    Here is the things you can do to improve the testing process:…specify the names and data which should be used in the test; specify the test design because your test design will be the primary requirements statement for your automation. Also state your expected result.
    Another easy way to improve the efficiency of the testing is to get more computers. Many testers can easily keep a couple of machines busy. Test automation can be an expensive and risky way of dealing with an equipment shortage.
    Improving the product for easier testing can also help improve the testing process: for example, most testers find themselves spending a lots of time on installing and reinstalling, on visually scanning error logs, on waiting for a slow product …all of these can save testers a lot of time.
    Improving the test process may buy you some time for test automation and will certainly make your automation project go more smoothly.
  • Here, Bret claims that since goals will be different between development manager, test manager, the testers and the automators, agreement on what will count as success of test automation should be reached in advance. What he means is that different people has different expectation on test automation, but you should not try to include all of these expectations into your test automation requirement.
    Test automation efforts often fail by trying to do too much. For success of test automation, your requirement should focus on where you are going to getting the biggest payback. Don’t try to automate every part of the test.
    Defining requirements for your test automation project will set different party’s expectation reasonably. Another step towards test automation success.
  • After defining the requirement, you can start to implement test automation. Now it is very important to remember Bret’s suggestion: that you should prove the feasibility….. You need validate your tools and approach as soon as possible. Is it even possible to automate tests for your product? Will your tools and approaches work for your product and staff. To find out the answers for these questions, you need a proof of concept, which is a quick, meaningful test suites that demonstrates that you are on the right track. Proof of concept is also an excellent way to evaluate the tools.
    I think prove the concept is a very valuable suggestion. Step 3 is a wise step to take for your success of test automation.
  • For the success of test automation, you also should asking as early as possible for product testability as a requirement of the product. Automating test on GUI is much more difficult, expensive and unreliable than on API or CLI. Therefore, if you require your developers to include CLI or API in the product and getting your testability features designed into the product, you remove more obstacles away from your path to the success.
  • If automators focus on just getting the automation to work, Test automation efforts are more likely to being dropped. Success of test automation require long term focus. Concern for the future is an important part of test automation design. Therefore you should try to make a sustainable test suite. A sustainable test suite should has following properties:
    Reviewability: It is critical that people be able to review a test suite and understand what is being tested. Good documentation is one way to achieve this.
    Maintainability: One of the biggest challenges to using automated test suites is keeping them functional as the product interface changes. It’s particular challenge for GUI automation.
    Integrity means your test suite should get the correct result. should not skip any error or generate any false alarm. Integrity also means the right test was run.
    Independence: Each test needs to setup its test environment and run individually without being affected or affecting other tests. Independent test will be convenient for developers to use when diagnosing reported defects.
    Repeatability means your tests should work the same way every time they are run.
    By focusing on design goals of long term sustainability and choosing an appropriate architecture, you will be moving along on the road to success.
  • Step 6 …
    Package the auto…this means documenting the setup and maybe even making the test suite easier to install and run.
    Then you need test your test suite and make sure it does not depend on any special libraries or services you have installed on your machine.
    Make ….It is ideal to make it downloadable from an internal web site.
  • After your test suite is documented and delivered, you are done. Now your success brings your additional challenges.
    You are probably blamed of some problems not due to your automation but due to tester’s lack of skills on diagnosing failures found by the automated tests. You probably will be called in to help diagnose every run.
    Another challenger is Improving …Tests will be needed to be added to improve the coverage or to test new features. With time, your tests are likely to stop finding problems since developers improves their design or code to fix those errors found by automated testing. Then you need to raise the bar by updating your test suite.
    You are encouraged to continue to invest in automation, but the particular challenge lies in the timing for when test automation should be done. Usually, once the testers know how to use the automation, it’s a good time for automators to get a jump on the next release and improve the test tools and libraries.
    Continuing to invest in automation will help you face the challenges of success and ensure the road remain clear for continuing success.
  • A successful automated software test can significantly improve the efficiency of software testing and contribute to higher product quality, more on time deliver, and lower costs.
    However test automation can be a nightmare too. The seven steps directs us to the success. They are: … Among these seven steps, I think, Improve the test process, prove the concept, design for sustainability are specially important.
    That’s my presentation today. I hope you find some useful and interesting information from it. Now it’s time for me to answer your questions.