CodeFest 2014. Michael Palotas — Test Engineering to Engineering Productivity: the road from bug detection to bug prevention
•Descargar como PPTX, PDF•
0 recomendaciones•1,455 vistas
This document appears to be a presentation about test engineering at eBay. It discusses the transition from waterfall to agile development, the importance of automation, preventing bugs rather than finding them, and how test engineering can increase productivity. It also touches on topics like horizontal and vertical support, measurement and metrics, open source software, and allowing engineers more autonomy. The overall goal discussed is improving quality while also accelerating development cycles.
Recomendados
Recomendados
Más contenido relacionado
Similar a CodeFest 2014. Michael Palotas — Test Engineering to Engineering Productivity: the road from bug detection to bug prevention
Similar a CodeFest 2014. Michael Palotas — Test Engineering to Engineering Productivity: the road from bug detection to bug prevention (20)
Más de CodeFest
Más de CodeFest (20)
Último
Último (20)
CodeFest 2014. Michael Palotas — Test Engineering to Engineering Productivity: the road from bug detection to bug prevention
- 1. TEST ENGINEERING TO ENGINEERING PRODUCTIVITY MICHAEL PALOTAS EBAY INTERNATIONAL 29. MARCH 2014
- 2. Michael Palotas Head of Productivity & Test Engineering Pfingstweidstrasse 60 8005 Zürich Switzerland Email: mpalotas@ebay.com Twitter: @michael_palotas LinkedIn: http://ch.linkedin.com/in/michaelpalotas/ WHO AM I? TESTING @ EBAY 2
- 3. THE COMPANY TESTING @ EBAY 3
- 4. EBAY INC. TESTING @ EBAY 4
- 5. EBAY FACTS Founded in 1995 Based in San Jose, California 28000 employees worldwide >100 million active buyers and sellers worldwide 2500 USD transaction volume every second TESTING @ EBAY 5
- 6. EBAY FACTS TESTING @ EBAY 6 2 billion page views every day 75 billion database calls every day >200 million downloads of eBay Inc’s mobile apps
- 9. AGILE PEOPLE RAISE YOUR HANDS TESTING @ EBAY 9
- 10. NOW SHAKE HANDS TESTING @ EBAY 10
- 11. TESTING @ EBAY 11
- 12. LOOKING BACK TESTING @ EBAY 12
- 13. AGILE? TESTING @ EBAY 13 Development Test Waterfall - 6 months Development Test Agile - 2 weeks
- 14. THE BI(U)G HUNT TESTING @ EBAY 14
- 15. AUTOMATION TESTING @ EBAY 15
- 16. OR IS THIS ACTUALLY THE PRESENT FOR YOU? TESTING @ EBAY 16
- 17. HOW ABOUT PREVENTING BUGS? TESTING @ EBAY 17
- 18. PRODUCTIVITY & TEST ENGINEERING Make the team more efficient & productive TESTING @ EBAY 18
- 19. HORIZONTAL / VERTICAL SUPPORT TESTING @ EBAY 19 TEAM 1 TEAM 2 TEAM 3 VERTICAL VERTICAL VERTICAL HORIZONTAL SUPPORT
- 20. HORIZONTAL SUPPORT TESTING @ EBAY TDD Automation Advice Architecture Infrastructure 20
- 21. VERTICAL SUPPORT Rapid testing / exploratory approach Focus on fast feedback Manual testing: Very very very very important Focus on primary work artifacts Domain Knowledge TESTING @ EBAY 21
- 22. Do we still need testers? Or developers only? TESTING @ EBAY 22
- 23. A FEW WORDS ON (TEST) MANAGEMENT
- 24. A WORD ON MEASUREMENT & METRICS TESTING @ EBAY 24
- 25. THE RACE TO ZERO TESTING @ EBAY Npr.org 25
- 26. OPEN SOURCE TESTING @ EBAY It is free (it is not really free …) Faster innovation cycles Independence Engagement / opportunities to grow Hiring advantage 26
- 27. THE LONG LEASH TESTING @ EBAY 27
- 28. HYGIENE FACTORS TESTING @ EBAY 28
- 29. THE BIG GOAL TESTING @ EBAY 29
- 30. THANK YOU!
- 31. REFERENCES • http://weekendreflection.blogspot.ch/ • http://perfectplate.com/ • http://www.jamshid-ibrahim.net/ • http://simplehomemade.net/ • http://npr.org • http://creatia2013.wordpress.com/ • www.kaushik.net • Sackmesser.ch • http://betterafter50.com • http://estonianfreepress.com/ TESTING @ EBAY 31
Notas del editor
- Expectation would have been that most people are working in testingSince this is the QA track
- Depending on how many devs are in the room:Point out that there are test conferences vsdev conferencesVery few combined conferences like codefest for exampleI always ask myself why are these two disciplines so separateI sopke at German Testing Day and did the keynote: Requirements, Product management, development and test: quality only is created togetherAnyway, is QA something really so special? Do other people not understand us? Do we as testers try to separate ourselves from the rest of the organization?
- Who in the room is a so called embedded tester? Who works in an agile team? (and why were you not able to convince your dev counterparts and product managers to attend the session)
- Tell about Bret Rogers, Sydney KeynoteWhy does he think that way? The way product is built has fundamentally changedBetter tools, better programming languages, smaller incrementsEspecially in agile teams, traditional testers have a really hard time to find their spaceWhat about test driven development? In many agile teams there are no testers at allWhat about Google? Facebook has no testers at all (apparently) In many companies the full effect is not visible yet, because they haven’t gone the agile way yet fullyIn most places there is still a hybrid approach. So Brett Rogers is right, there is a huge wave of change coming to testing in the next 3-5 years
- We are moving to agile, what are our testers going to do? We keep complaining about being left out etc.
- Let’s look back: ebay but also the rest of the industryTeams are separated between dev and test Maybe even sometimes geographically (i.e. test factory) Everything done manuallyExtremely untechnicalLots of secondary artifacts: test plan, test cases, reviews, but not a lot of testsDev and test has contrary goals: dev=few bugs, test=lots of bugsSimilar to DevOps lots vs little change
- Who was agile in the room? Everyone is agile but in my opinion very little has changed in so called agile teamsOk, there are embedded testersThey often do the work that devsdont want to doFirst development then testBefore it was 6 monthsAnd it is 2 weeks, we call it a sprintYet, testers still wait for devs to code something and then to test it
- Main job of a tester used to be to find bugsMy job description used to be: responsible for software quality in europeBUT: I had no influence how the product was developed and how it was built, same for toolsTesters were looking for bugs and were super happy when they found oneBug bashes –> huge exitement about finding lots of bugsIsnt that reilly weird I was thinking about it for a long time: how about preventing bugs instead of finding them Testers have never tried to prevent a bug, instead they were rewarded if they found one
- Probably one of the biggest failures in testingMillions were spent on tools that promised everything and kept nothingPeople automated because it was part of the processAutomation never in synch with developmentTools used that a dev would never touch Automation was the job of a testerAfter running the tests 50% failedNobody caredAfter 3 consecutive fails they were turned off, in oder to make the build green againAutomation not part of the sprintPeople declared autmation as failed
- Always asked myself why we cannot not focus on avoiding and preventing bugs rather than just finding them
- This is what we did at ebay:Renamed the team from quality engineering to productivity & test engineeringQuality is out of the nameTesting is still in thereWhy? Because the whole team is responsible for quality and not just the quality guyFocus is also on productivityWe try to prevent bugs by productive and efficient teamsGood tools, build pipelines, good automated tests
- ConsultingHelp with i.e. automation tools, infrastructureCIVirtualizationTools, i.e. Selenium GridIos-driver, selendroid
- There is no set processMay be different per project and per team Focus is on fast feedback No ceremonial stuffTesting often times does many things outside of testing, but not much actual testing BUT: it may be difficult to get the trust when you cannot show which testcases you have checked offManage coverageArticulate risk No independenceWork across SDLC
- Yes absolutely do we need testersBut a very different kind of testerMch more technical Same language as developersVery versatileLook through user viewCode level fix bugDo code reviewsReview user storiesSet up CI
- We don’t have test managers, only testers
- Maybe unconventional viewAre X bugs good or bad? Is 80% code coverage good or bad? Bug submission goals (15 per week) Testers were measured by how many bugs they submitWho is the better tester: 10 or 20 bugs? It is the change that matters and not the absolute number25% percent of the test cases are automated – huh? We look at metrics like in Sonar, but mot as goals but for information purposesStill important to collect metrics example of 700 bugs, visibility, top management supportBUT: in agile environments it may be difficult to keep track of every bug. Testers can speak directly to dev and dont need to log a bug
- Race to zero products should not cost anything anymoreFor many orgs testing is a pure cost factorCompanies try to lower cost by offshoring, outsourcing Usually it is not cheaper at all actually Outsourcers try to place their engineers as long as possible, thats ok but I as a customer want the oppositeNever seen that an outsources can provide better quality as doing it in houseBrett Rogers: out of 130 outsourced people he would have hired oneWe completely stopped outsourcingReplayed 65 in india with 25 in houseFaster, cheaper and people are super motivated, loyal etc.
- Job TitelWork hoursWork place WFHLet people fail Support them
- WLBMoney, will nichtdassjemandwegen 10% woandershingehtbusiness classzero attrition in 4 yearsshower at work Nice dinnerstake taxi from airport
- Quality needs to be in the DNA of a teamFocus on avoiding bugs instead of finding them afterwardsNo separate team Very technical skill set neededWe need to change on order to keep our jobsTest engineers should be able to fix the bugsMain job of tester: articulate risk, manage coverage, problem solver, mediator2. part: Long leash for peopleInnovation by trust What I told you works in EVERY company – if people want to make it workYou need to be ready for changeMost likely my job as it is today won’t exist in a year from nowThen there will be even more interesting things to do