Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Similar a Automated testing made easy for start ups
Similar a Automated testing made easy for start ups (20)
Último
Último (20)
Automated testing made easy for start ups
- 1. TESTINGMIND www.testingmind.com ANNUAL TEST AUTOMATION AND DIGITAL QA SUMMIT #TAS18 Auckland November 1st , 2018
- 2. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland AUTOMATED TESTING MADE EASY FOR START-UPS How to test your project and not burn a hole in your pocket
- 3. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland VP of Support and QA at LinguaLeo.com CIO & CTO at Vesta-Central.com Founder of ÜberIT.co.nz About me
- 4. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland CLASSICAL TESTING PYRAMID
- 5. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland UNIT TESTS • Consume developers’ time • Require that code be structured in a specific way • Need to be updated regularly
- 6. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland INTEGRATION TESTS • Are detached from real life • Require that custom interfaces be added
- 7. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland GUI TESTS • Do not show visual changes • Hard-to-maintain • Unstable • Challenging to debug
- 8. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland AN ALTERNATIVE: THE TESTING RECTANGLE
- 9. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland STATIC CODE ANALYSIS • Helps to identify logical errors and unexpected behaviours • Detects, not only bugs, but also sloppy and inefficient code • Requires only minimal maintenance • Helps to prevent vulnerabilities
- 10. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland 20+ languages, €0 - €120 15+ languages, $0 - $15 https://sonarsource.com/ https://codacy.com/
- 11. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland
- 12. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland CATCHING VULNERABILITIES
- 13. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland CATCHING VULNERABILITIES • The logic of the program does not comply with the code formatting: judging by the alignment, we get the impression that both goto statements refer to the if statement, but it isn't so. The first goto is really in the condition, but the second - not. • Unreachable code: as the second goto runs without a condition, the code following it won't get executed.
- 14. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland CATCHING VULNERABILITIES • The logic of the program does not comply with the code formatting: judging by the alignment, we get the impression that both goto statements refer to the if statement, but it isn't so. The first goto is really in the condition, but the second - not. • Unreachable code: as the second goto runs without a condition, the code following it won't get executed. Apple's SSL/TLS bug (22 Feb 2014)
- 15. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland API TESTING • Close to real life • Covers several layers • Relatively fast
- 16. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland $79-$599 $0-$100 Free (in beta-testing) https://runscope.com https://apiprove.com https://assertible.com
- 17. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland
- 18. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland
- 19. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland Before After
- 20. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland
- 21. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland SCREENSHOT-BASED TESTING • Can be done without any programming • The only way to catch rendering bugs automatically • Simple and time-efficient
- 22. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland $0-$75 https://screenster.io/ $149-$649 https://www.chromaticqa.com
- 23. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland
- 24. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland
- 25. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland
- 26. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland
- 27. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland Before After
- 28. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland
- 29. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland
- 30. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland
- 31. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland THE OUTCOME
- 32. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland THE OUTCOME
- 33. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland THE OUTCOME
- 34. TESTINGMIND CONSULTING Annual Test Automation and Digital QA Summit | Auckland THANKS FOR YOUR ATTENTION! ANY QUESTIONS?
Notas del editor
- Hello, my name is Daos Knox, and I am a guy from the start-up world. Who curr work - Raise hands who is currently working in a startup Wants to work - -||- who has never worked but wants do work some day Two controversial goals – you have to move forward really fast, but at the same time you don’t have enough time and money No testing/devs testing - Some brave people audacious enough to skip testing at all, allowing the users to find all the bugs, Others are making the developers to do all the testing. Another approach – So I want to tell you about an approach to achieve quite good level of testing for very little money and without spending much time.
- A little bit about myself. I worked as VP of Support and QA at LinguaLeo. It is an international educational service, originated in Russia, with more than 14 million users. Currently I am a CIO and CTO at Vesta-Central, it is a b2b product data sharing platform for manufactures and merchants. Plus I have my own software development company called ÜberIT. We help people to achieve their dreams by implementing state-of-the-art applications, plus build our own projects.
- This is - the standard, classical automated testing pyramid. Proper way if no shortage - It’s a solid, proper way to do automated testing, when you are not in a shortage of time and money. Alas, in the start-up world you are kind of in a shortage of both. Reason - And there is a perfectly good reason for it. Fulfil needs - The only way for a start-up to survive and achieve success is to fulfil users’ needs faster and better than competitors. Trial and error - Plus even founders usually have quite vague understanding of these needs, so they have to move forward by trial and error. All that mean that: Fast changing - First, everything’s changing really fast: new features implemented, existing ones changed, old ones removed, and the testing has to keep up with the development No recourses - Second, testing is very short in resources. Let’s have a look why this pyramid does not really fit in a startup.
- Unit-tests. In theory they are great: fast, simple, efficient. Written by devs - But they have to be written and maintained by the developers. That means that the developers will deliver less new features. Changes freq – And since everything changes frequently, there is no point to spend time writing tests for code that is going to be changed or removed. --You have to write your code in a way it can be covered by unit-tests. That’s also an evasion in a way of delivering new features. --All new code require to have unit tests immediately, otherwise test coverage will go down, depreciating the whole idea of having unit tests. And, after all, they don’t guarantee that your system is correctly working, after all! To deal with the last problem, there are…
- Two components, no reallife Proper, by-the-book int tests should test only two components at once, so they don’t really test real-life scenarios. If you have enough time - Yes, if you have enough time to implement thorough testing – no probs. But if you’re in a rush and everything is hectic – they are not an option. Again, developers – And again, most probably you have to ask the developers to spend their precious time to add custom interfaces in order to integrations tests be performed.
- Time For web applications the most common toolchain would be Selenium, WebDriver, plus one of the popular languages, like Java or Python. Since Selenium uses a real browser, these test are quite slow and prone to have an unexpected behaviour. And, what is kind of funny, these GUI tests can’t show you visual changes!
- An alternative – I offer an alternative, by looking a little bit outside of the box. I call it the “testing rectangle”. Three levels - It also consists of three levels, but these levels aimed to test completely different aspects of the application from different angles. Combination - Their combination covers all important parts of code while require very little maintenance. And they are very cost-effective.
- Close to unit-tests – It is close to unit-tests, it also has to be run by the developers themselves. Testers to encourage – So the role for the testers is to encourage the developers to use it. The great thing – does not require any programming, just run against the code and get the results. List of mistakes – basically, any S.C.A. tool is a huge list of common programming mistakes Can’t capture all - No high-level business logic - Security and low-level logical - But it’s very good spotting common security errors and low-level logical errors.
- Many different – There are many different s.c.a. tools and systems for virtually any language existing. Leitmotif - But since my leitmotif today is time and cost efficiency, I’m going to consider only reasonably-priced cloud-based systems.
- typical example - Here is a typical example of a code analysis report. A programmer missed a clause which might lead to undetermined behaviour under certain conditions.
- And my favourite example of s.c.a. spotting vulnerabilities. That was a security issue in a very popular piece of software by a very huge company. As you can see, the developer duplicated a “goto” operator. By mistake, hopefully. As a result, the SSL implementation was compromised, allowing for an attacker to intercept encrypted traffic between a device and a server.
- Obviously, this problem passed all testing layers and was live for some time. A s.c.a. tool though, would easily spot it by firing two warnings.
- That company was Apple. They had to issues a hotfix for iOS in order to fix this issue.
- Tricky - The next step is API testing. It’s a bit tricky – in order to use it efficiently, the whole application has to be designed in a specific way. API for all – In particular, the back-end should provide common API for all consumers, including your own front-end code. Test business-logic - Hence, by testing it, you can be sure that the business-logic works correctly. Mobile – Mobile apps have to use the API-based back-end by design, so all good there. Not for web- But the traditional web-app architecture uses server-side rendering. Not-a-problem - Usually that’s not a problem to gradually move to the new architecture. For example, at Vesta we have inherited a very old-school application about a year ago. And now we have API endpoints and new front-end code for all core functions and are enjoying all benefits of API testing. Downside – The main downside is that if you found a problem during testing API, sometimes it takes time to find the root of a problem.
- Again, reasonable-priced cloud services. The first two are quite mature and have lots of functions. But not enough, so we at ÜberIT have decided to create our own API testing platform, we called it APIprove. It is at very early beta stage, so you all are welcome to have a look and share your ideas.
- And the last stage is screenshot-based testing. It’s quite new concept that has grew up from regular Selenium testing. The idea is that instead of checking elements and values on the page, you just take a screenshot of it and compare with the screenshot taken earlier. If they are identical – you’re good. The beauty of this approach is that we test both logic and look, two birds with one stone!