The Right Tool for the Right Project
•Descargar como PPTX, PDF•
2 recomendaciones•520 vistas
Selecting the right automation framework is hard. It might be a critical decision in implementing your continuous testing agenda. Today, there are so many possibilities. From open source solutions (Selenium Appium, etc.), to HPE developed commercial tools (QTP, UFT, Mobile Center). What was the process that we did in order to select the right automation framework. Our approach was to 1st define our needs from such a framework and we ended up with a list of 10 must have requirements for such a framework and then went on to pick the right tool. We ended up with different frameworks for different projects with a wide combinations of open source to our own tools (UFT, LeanFT, etc.)
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Destacado
Destacado (20)
Similar a The Right Tool for the Right Project
Similar a The Right Tool for the Right Project (20)
Más de Ori Bendet
Más de Ori Bendet (12)
Último
Último (20)
The Right Tool for the Right Project
- 1. Welcome!
- 2. Agenda –18:00-18:20 Gathering –18:20-19:00 The right tool for the right project | Ori Bendet –19:00-19:05 PIZZAS! –19:00-19:45 Surviving the Storm - continuous testing in the world of SaaS & Cloud | Karim Fanadka –19:45-20:30 networking and Pokémon party outside the office (2 Pokéstops w/ lures)
- 3. Have any ideas for a future meetup? –Contact me: r2d2@hpe.com 3
- 4. Please give us a 5-star rating! 4
- 5. The Right Tool for the Right Project @bendet_ori
- 6. 6y in HPE Software in various managerial QA rolesToday - Inbound PM for a new cloud testing offering ABOUT ME
- 14. Now Let’s Get To Business!
- 15. Assumption #1 15 Vendors only use their own tools
- 16. Assumption #2 16 Evil corporates hate Open Source
- 17. Assumption #3 17 Everybody is doing automated testing
- 19. Project #1 AUT: Analytic Platform Automation: Internal Standalone Tool 19
- 20. Analytic Platform for IT Executives (v 1.0) – AUT technology Stack – Glassfish Server – Flex + GWT – SAP BODS for ETLs – SAP BOE for BI – MSSQL – Automation – People doing automation: 0.5 / 5 – Focusing mainly on APIs – Java Beans (EJBs) – Internal tool called FIST – ROI: LOL 20 3.5 hours to install
- 21. 21
- 22. 22
- 23. 23
- 24. 24 An extendable Java class
- 25. 25
- 26. Why? 26
- 27. Lessons Learned StandaloneTool developed internally – Cons – Nobody knew about the tool, wasn’t cool, no buzz around it – External tool: unable to get DEV to cooperate with automation (or even install the tool) – Manually triggered (no part of the CI process) – Almost every new test required changes to be done by DEV (expose new API methods) – No direct access to source code – Pros – Small investment – QTTV – Easily extendable – Stability 27
- 28. Analytic Platform for IT Executives (v 1.0 2.0) – AUT technology Stack – Glassfish Server – Flex + GWT – SAP BODS for ETLs – SAP BOE for BI – MSSQL – Automation – People doing automation: 4 / 10, dedicated developer to assist – Focusing mainly on APIs Automation installation, E2E flows – Java Beans (EJBs) REST APIs – Internal tool called FIST Internally built framework REST client, Selenium, Flex Monkey – Automatic Deployment Solution: internally developed tool called Slick – ROI: 3 MD each sprint 1 MM per release 28 Automatic Deployment Solution
- 29. 29 Saved over 10K MH
- 30. Why? – High demand for automation coverage – Developers selected the automation framework – Wanted to work together with developers – Invested in automated installation as well 30
- 31. Lessons Learned Internally built Automation Framework (inside the IDE) – Cons – Required more coding skills – Large effort of getting things started – Harder to be used by less-technical testers – UI automation stability – Pros – Harness developers into the automation – Developers re-used testing assets for their own benefits – Testers have access to source code – Part of the CI process 31
- 32. Project #2 AUT: Performance Testing tool Automation: Open Source 32
- 33. Performance Testing tool – AUT technology Stack – nodeJS – AngularJS – Internal SQLlite – Automation – People doing automation: 1 / 4 – System tests leveraging developers assets – Focus on API testing, sanity level UI testing – Application Modeling for less-technical testers – Protractor, Mocha, Jasmine – ROI: 1.5 MD / sprint 33 Protractor
- 34. Without the framework With the framework REST API
- 35. Without the framework With the framework UI:
- 36. Test (created by non-technical engineer)
- 37. Why? – Best available choice for the technology stack – 1 automation engineer working on the framework, others re-using the assets 37
- 38. Lessons Learned Leveraging DEV assets and extending into our own framework (+ modeling) – Cons – Modeling takes time – Technical engineer becomes the bottleneck – UI automation stability – Pros – Use existing developers assets – Extend automation coverage using non-technical engineers – Modeling eases tests maintenance 38
- 39. Project #3 AUT: Firefox plugin Automation: Commercial Tool (LeanFT) 39
- 40. TruClient? TruClient is a tool for recordingWeb-based applications. It is used inside LoadRunner for performance testing on the browser level 40 1. TruClient Sidebar 2. TruClient Toolbox 3. Firefox browser 4. Application Browser Window 5. TruClient Sidebar Status Pane
- 41. TruClient – AUT technology Stack – Firefox Plugin – Pure Web – WPF – Windows app – Automation – People doing automation: 2 / 6 – Did not have an automation suite as they could not find a tool which has automation abilities for the full flow – including the three technologies. – Supports the three main browsers and needs an automation tool that can identify and test the objects in all the supported browsers – Selected LeanFT as the tool 41
- 42. The Automation Suite 42 LeanFT Application Model Containing the full AUT the App Model displays Modular view of all the implemented in the application
- 43. The Automation Suite 43 LeanFT test - Everything is written in the IDE, Dev have them for sanity) - Test code is completely reusable among the - The test is authored once, and can be run on - In the test setup, the TruClient launcher – and being used, and during the test, web and technologies are tested.
- 44. Why? – Cross Technology support (Desktop & Web) – Script once – run of all browsers – Re-use, share testing assets with Dev 44
- 45. Want to buy a new automation framework for 1 shekel?? 45
- 47. 47
- 49. Rule #1 49 Accessible in the developers workspace
- 51. Rule #3 51 Ability to easily model the UI
- 52. Rule #4 52 Full support for REST APIs testing
- 54. Rule #6 54 Ability to combine UI/API/DB in 1 flow
- 55. Rule #7 55 Messaging Parser (json, XML, etc.)
- 56. Rule #8 56 Parameterization of tests
- 58. Rule #10 58 Modularity to allow re-use by less tech eng
- 59. 59
- 60. FIST Selenium built FWK LeanFT UFT IDE √ Cross X Mobile/Web Model the UI REST API testing X* DB Layer X* Combine UI/API/DB X* Parsers X* Parameterization CI/CD Modularity for less-tech Cross Platforms Lightweight 60= can be added by user
- 61. Summary Time
- 62. Assumption #1 62 Vendors only use their own tools
- 63. Assumption #1 63 Vendors only use their own tools It’s not about the tool
- 64. Assumption #2 64 Evil corporates hate Open Source
- 65. Assumption #2 65 Evil corporates hate Open Source We love Open Source!!
- 66. Assumption #3 66 Everybody is doing automated testing
- 67. Assumption #3 67 Everybody is doing automated testing Everybody is using automated testing
- 69. It’s never about the tool It’s about finding the right tool for the right project
Notas del editor
- General REST framework written in the same language as the developers (preferable as part of a UI framework) Infrastructure that supports all REST functionalities Validators for REST responses Logs Error handling JSON handling Support different browsers (UI framework) Modular framework Parameterization of tests Backend tests: • Grunt module to build the entire framework – according to CI/CD request • REST Wrapper, extending functionality • DB Layer – Postgresql • Payload templates • Logs • JSON parser • Expectation module • Parameter mechanism • Support for E2E Flows Client testing: Grunt module to build the entire framework – according to CI/CD request Protractor wrapper – add more functionality to the driver API’s DB Layer – Postgresql – to help in DB commination and functionality UI Controls – map all the product elements and controls Parameter mechanism Reports Multi browsers / OS JS help function (XML, JSON, etc..)
- General REST framework written in the same language as the developers (preferable as part of a UI framework) Infrastructure that supports all REST functionalities Validators for REST responses Logs Error handling JSON handling Support different browsers (UI framework) Modular framework Parameterization of tests Backend tests: • Grunt module to build the entire framework – according to CI/CD request • REST Wrapper, extending functionality • DB Layer – Postgresql • Payload templates • Logs • JSON parser • Expectation module • Parameter mechanism • Support for E2E Flows Client testing: Grunt module to build the entire framework – according to CI/CD request Protractor wrapper – add more functionality to the driver API’s DB Layer – Postgresql – to help in DB commination and functionality UI Controls – map all the product elements and controls Parameter mechanism Reports Multi browsers / OS JS help function (XML, JSON, etc..)
- A new product Startup mode no UI automating investment No DEV involvement in automation (asked for new public APIs from DEV for automation purposes)
- “Why do I want to learn another tool?? I have my IDE”
- A new product Startup mode no UI automating investment No DEV involvement in automation (asked for new public APIs from DEV for automation purposes)
- Extending unit/integration to include more cases
- Enlrage the left and map to right side
- The TruClient user interface is made up of the following sections: TruClient Sidebar. The heart of the interface, the sidebar contains all the tools you need to develop your TruClient scripts. TruClient Toolbox. The toolbox contains all of the steps that you can add to a TruClient script. The toolbox opens and closes by clicking on the tab, and moves by dragging it up or down. Browser Navigation Bar. Enter the URL of the application for which you are developing a script. Application Browser Window.The window containing the browser where you develop and replay your script interactively. TruClient Sidebar Status Pane. A pane that displays status details about the active action in the TruClient Sidebar.
- Extending unit/integration to include more cases
- Came ACoE and wanted us to use their own automation framework
- General REST framework written in the same language as the developers (preferable as part of a UI framework) Infrastructure that supports all REST functionalities Validators for REST responses Logs Error handling JSON handling Support different browsers (UI framework) Modular framework Parameterization of tests Backend tests: • Grunt module to build the entire framework – according to CI/CD request • REST Wrapper, extending functionality • DB Layer – Postgresql • Payload templates • Logs • JSON parser • Expectation module • Parameter mechanism • Support for E2E Flows Client testing: Grunt module to build the entire framework – according to CI/CD request Protractor wrapper – add more functionality to the driver API’s DB Layer – Postgresql – to help in DB commination and functionality UI Controls – map all the product elements and controls Parameter mechanism Reports Multi browsers / OS JS help function (XML, JSON, etc..)
- General REST framework written in the same language as the developers (preferable as part of a UI framework) Infrastructure that supports all REST functionalities Validators for REST responses Logs Error handling JSON handling Support different browsers (UI framework) Modular framework Parameterization of tests Backend tests: • Grunt module to build the entire framework – according to CI/CD request • REST Wrapper, extending functionality • DB Layer – Postgresql • Payload templates • Logs • JSON parser • Expectation module • Parameter mechanism • Support for E2E Flows Client testing: Grunt module to build the entire framework – according to CI/CD request Protractor wrapper – add more functionality to the driver API’s DB Layer – Postgresql – to help in DB commination and functionality UI Controls – map all the product elements and controls Parameter mechanism Reports Multi browsers / OS JS help function (XML, JSON, etc..)
- General REST framework written in the same language as the developers (preferable as part of a UI framework) Infrastructure that supports all REST functionalities Validators for REST responses Logs Error handling JSON handling Support different browsers (UI framework) Modular framework Parameterization of tests Backend tests: • Grunt module to build the entire framework – according to CI/CD request • REST Wrapper, extending functionality • DB Layer – Postgresql • Payload templates • Logs • JSON parser • Expectation module • Parameter mechanism • Support for E2E Flows Client testing: Grunt module to build the entire framework – according to CI/CD request Protractor wrapper – add more functionality to the driver API’s DB Layer – Postgresql – to help in DB commination and functionality UI Controls – map all the product elements and controls Parameter mechanism Reports Multi browsers / OS JS help function (XML, JSON, etc..)
- General REST framework written in the same language as the developers (preferable as part of a UI framework) Infrastructure that supports all REST functionalities Validators for REST responses Logs Error handling JSON handling Support different browsers (UI framework) Modular framework Parameterization of tests Backend tests: • Grunt module to build the entire framework – according to CI/CD request • REST Wrapper, extending functionality • DB Layer – Postgresql • Payload templates • Logs • JSON parser • Expectation module • Parameter mechanism • Support for E2E Flows Client testing: Grunt module to build the entire framework – according to CI/CD request Protractor wrapper – add more functionality to the driver API’s DB Layer – Postgresql – to help in DB commination and functionality UI Controls – map all the product elements and controls Parameter mechanism Reports Multi browsers / OS JS help function (XML, JSON, etc..)
- General REST framework written in the same language as the developers (preferable as part of a UI framework) Infrastructure that supports all REST functionalities Validators for REST responses Logs Error handling JSON handling Support different browsers (UI framework) Modular framework Parameterization of tests Backend tests: • Grunt module to build the entire framework – according to CI/CD request • REST Wrapper, extending functionality • DB Layer – Postgresql • Payload templates • Logs • JSON parser • Expectation module • Parameter mechanism • Support for E2E Flows Client testing: Grunt module to build the entire framework – according to CI/CD request Protractor wrapper – add more functionality to the driver API’s DB Layer – Postgresql – to help in DB commination and functionality UI Controls – map all the product elements and controls Parameter mechanism Reports Multi browsers / OS JS help function (XML, JSON, etc..)
- General REST framework written in the same language as the developers (preferable as part of a UI framework) Infrastructure that supports all REST functionalities Validators for REST responses Logs Error handling JSON handling Support different browsers (UI framework) Modular framework Parameterization of tests Backend tests: • Grunt module to build the entire framework – according to CI/CD request • REST Wrapper, extending functionality • DB Layer – Postgresql • Payload templates • Logs • JSON parser • Expectation module • Parameter mechanism • Support for E2E Flows Client testing: Grunt module to build the entire framework – according to CI/CD request Protractor wrapper – add more functionality to the driver API’s DB Layer – Postgresql – to help in DB commination and functionality UI Controls – map all the product elements and controls Parameter mechanism Reports Multi browsers / OS JS help function (XML, JSON, etc..)
- General REST framework written in the same language as the developers (preferable as part of a UI framework) Infrastructure that supports all REST functionalities Validators for REST responses Logs Error handling JSON handling Support different browsers (UI framework) Modular framework Parameterization of tests Backend tests: • Grunt module to build the entire framework – according to CI/CD request • REST Wrapper, extending functionality • DB Layer – Postgresql • Payload templates • Logs • JSON parser • Expectation module • Parameter mechanism • Support for E2E Flows Client testing: Grunt module to build the entire framework – according to CI/CD request Protractor wrapper – add more functionality to the driver API’s DB Layer – Postgresql – to help in DB commination and functionality UI Controls – map all the product elements and controls Parameter mechanism Reports Multi browsers / OS JS help function (XML, JSON, etc..)
- General REST framework written in the same language as the developers (preferable as part of a UI framework) Infrastructure that supports all REST functionalities Validators for REST responses Logs Error handling JSON handling Support different browsers (UI framework) Modular framework Parameterization of tests Backend tests: • Grunt module to build the entire framework – according to CI/CD request • REST Wrapper, extending functionality • DB Layer – Postgresql • Payload templates • Logs • JSON parser • Expectation module • Parameter mechanism • Support for E2E Flows Client testing: Grunt module to build the entire framework – according to CI/CD request Protractor wrapper – add more functionality to the driver API’s DB Layer – Postgresql – to help in DB commination and functionality UI Controls – map all the product elements and controls Parameter mechanism Reports Multi browsers / OS JS help function (XML, JSON, etc..)
- General REST framework written in the same language as the developers (preferable as part of a UI framework) Infrastructure that supports all REST functionalities Validators for REST responses Logs Error handling JSON handling Support different browsers (UI framework) Modular framework Parameterization of tests Backend tests: • Grunt module to build the entire framework – according to CI/CD request • REST Wrapper, extending functionality • DB Layer – Postgresql • Payload templates • Logs • JSON parser • Expectation module • Parameter mechanism • Support for E2E Flows Client testing: Grunt module to build the entire framework – according to CI/CD request Protractor wrapper – add more functionality to the driver API’s DB Layer – Postgresql – to help in DB commination and functionality UI Controls – map all the product elements and controls Parameter mechanism Reports Multi browsers / OS JS help function (XML, JSON, etc..)
- General REST framework written in the same language as the developers (preferable as part of a UI framework) Infrastructure that supports all REST functionalities Validators for REST responses Logs Error handling JSON handling Support different browsers (UI framework) Modular framework Parameterization of tests Backend tests: • Grunt module to build the entire framework – according to CI/CD request • REST Wrapper, extending functionality • DB Layer – Postgresql • Payload templates • Logs • JSON parser • Expectation module • Parameter mechanism • Support for E2E Flows Client testing: Grunt module to build the entire framework – according to CI/CD request Protractor wrapper – add more functionality to the driver API’s DB Layer – Postgresql – to help in DB commination and functionality UI Controls – map all the product elements and controls Parameter mechanism Reports Multi browsers / OS JS help function (XML, JSON, etc..)
- General REST framework written in the same language as the developers (preferable as part of a UI framework) Infrastructure that supports all REST functionalities Validators for REST responses Logs Error handling JSON handling Support different browsers (UI framework) Modular framework Parameterization of tests Backend tests: • Grunt module to build the entire framework – according to CI/CD request • REST Wrapper, extending functionality • DB Layer – Postgresql • Payload templates • Logs • JSON parser • Expectation module • Parameter mechanism • Support for E2E Flows Client testing: Grunt module to build the entire framework – according to CI/CD request Protractor wrapper – add more functionality to the driver API’s DB Layer – Postgresql – to help in DB commination and functionality UI Controls – map all the product elements and controls Parameter mechanism Reports Multi browsers / OS JS help function (XML, JSON, etc..)
- General REST framework written in the same language as the developers (preferable as part of a UI framework) Infrastructure that supports all REST functionalities Validators for REST responses Logs Error handling JSON handling Support different browsers (UI framework) Modular framework Parameterization of tests Backend tests: • Grunt module to build the entire framework – according to CI/CD request • REST Wrapper, extending functionality • DB Layer – Postgresql • Payload templates • Logs • JSON parser • Expectation module • Parameter mechanism • Support for E2E Flows Client testing: Grunt module to build the entire framework – according to CI/CD request Protractor wrapper – add more functionality to the driver API’s DB Layer – Postgresql – to help in DB commination and functionality UI Controls – map all the product elements and controls Parameter mechanism Reports Multi browsers / OS JS help function (XML, JSON, etc..)
- General REST framework written in the same language as the developers (preferable as part of a UI framework) Infrastructure that supports all REST functionalities Validators for REST responses Logs Error handling JSON handling Support different browsers (UI framework) Modular framework Parameterization of tests Backend tests: • Grunt module to build the entire framework – according to CI/CD request • REST Wrapper, extending functionality • DB Layer – Postgresql • Payload templates • Logs • JSON parser • Expectation module • Parameter mechanism • Support for E2E Flows Client testing: Grunt module to build the entire framework – according to CI/CD request Protractor wrapper – add more functionality to the driver API’s DB Layer – Postgresql – to help in DB commination and functionality UI Controls – map all the product elements and controls Parameter mechanism Reports Multi browsers / OS JS help function (XML, JSON, etc..)
- General REST framework written in the same language as the developers (preferable as part of a UI framework) Infrastructure that supports all REST functionalities Validators for REST responses Logs Error handling JSON handling Support different browsers (UI framework) Modular framework Parameterization of tests Backend tests: • Grunt module to build the entire framework – according to CI/CD request • REST Wrapper, extending functionality • DB Layer – Postgresql • Payload templates • Logs • JSON parser • Expectation module • Parameter mechanism • Support for E2E Flows Client testing: Grunt module to build the entire framework – according to CI/CD request Protractor wrapper – add more functionality to the driver API’s DB Layer – Postgresql – to help in DB commination and functionality UI Controls – map all the product elements and controls Parameter mechanism Reports Multi browsers / OS JS help function (XML, JSON, etc..)
- General REST framework written in the same language as the developers (preferable as part of a UI framework) Infrastructure that supports all REST functionalities Validators for REST responses Logs Error handling JSON handling Support different browsers (UI framework) Modular framework Parameterization of tests Backend tests: • Grunt module to build the entire framework – according to CI/CD request • REST Wrapper, extending functionality • DB Layer – Postgresql • Payload templates • Logs • JSON parser • Expectation module • Parameter mechanism • Support for E2E Flows Client testing: Grunt module to build the entire framework – according to CI/CD request Protractor wrapper – add more functionality to the driver API’s DB Layer – Postgresql – to help in DB commination and functionality UI Controls – map all the product elements and controls Parameter mechanism Reports Multi browsers / OS JS help function (XML, JSON, etc..)