QA is Broken, Fix it!
•Descargar como PPTX, PDF•
3 recomendaciones•1,590 vistas
So software development has been broken for a long time due to the need to create a formal approach, however the approach that has generally been adopted didn't work and has never worked, but at least the people at the top had a modicum of control which created the illusion that everything was working fine. So in conclusion, software development has been around for a relatively long time and due to that there are a hundred and one ways of doing apparently the same thing, creating software. However compared to the sciences, software development isn't yet out of its teens and as such there really isn't an empirical evidenced based approach to software testing. So we just have to fumble along with the knowledge that we currently have and continue to improve.
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Similar a QA is Broken, Fix it!
Similar a QA is Broken, Fix it! (20)
Último
Último (20)
QA is Broken, Fix it!
- 1. Stephen Blower @badbud65 QA is Broken. Fix It! "In every chain of reasoning, the evidence of the last conclusion can be no greater than that of the weakest link of the chain, whatever may be the strength of the rest.“ Thomas Reid's ”Essays on the Intellectual Powers of Man” 1786
- 2. How many different development/test methodologies are in existence? Question:
- 3. Spiral Model of Software Development (1986) Rational Unified Process (1994)V-Model (1982) Waterfall (1984) Scrum (1995) Crystal Clear (1996) Extreme Programming (1996) Adaptive Software Development (1995) Feature Driven Development (1995) Dynamic Systems Development Method (1995) Agile Manifesto Published (2001) Test Driven Development (2003) Rapid Application Development (2004) Goal-Driven Software Development (2006) Behaviour Driven Development (2009)
- 4. Introduction No one right way
- 5. We all desire perfection but, the road to success is often misleading with self confessed Oracles. Fantasy Reality
- 6. No one Development Team is the Same Numerous Tools Numerous Methodologies Diverse Levels of Experience and Skill Sets Wide ranging levels of Enthusiasm and Energy Optimistic and Pessimistic Attitudes Towards Change Strict Process Driven or Flexible and Informal Practices Personal Ownership or Strict Hierarchical Structures Development Processes Should be about the People Not How to Control the People
- 7. The ideas I would like you to take away from this talk • Software development hasn’t advanced as far as people like to believe • Too many ideas, little consensus and no proven methodologies • No one software development methodology is the holy grail • Change is difficult, move forward slowly and throw away broken processes
- 8. What is QA and what do I mean by Broken? Quality Assurance? Dictionary Definition: Quality, defines a “Certain Level of Excellence”. “Excellence” meaning “Extremely Good” “Assurance” means “Keeping Promises” So Quality Assurance means: Keeping Promises to Supply Products or Services to a Certain Level of Excellence.
- 9. ~ Johanna Rothman put this really well in a talk, she said that when testers tell her that they do “Quality Assurance", she asks questions like these: 1. Do testers have the authority and cash to provide training for programmers who need it? 2. Do testers have the authority to settle customer complaints? Or to drive the handling of customer complaints? 3. Do testers have the ability and authority to fix bugs? 4. Do testers have the ability and authority to either write or rewrite the user manuals? 5. Do testers have the ability to study customer needs and design the product accordingly? If not, the quality of the product and of the complaining customer's experience in dealing with it, are not under the testers' control. This exert is from a Paper Called: The On-going Revolution in Software Testing: Cem Kaner
- 10. We are all doing it wrong, because no one knows how to do it right, including me.
- 11. A Brief History of Software Development How did we get into this mess?
- 12. A Brief History of Software Development Practices The Agile™ Manifesto states: We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan That is, while there is value in the items on the right, we value the items on the left more.
- 13. When was agile first used? Quote: From: Iterative and Incremental Development: A Brief History Craig Larman of Valtech & Victor R. Basili of University of Maryland Gerald M. Weinberg: “We were doing incremental development as early as 1957, in Los Angeles, under the direction of Bernie Dimsdale [at IBM’s Service Bureau Corporation]. He was a colleague of John von Neumann, so perhaps he learned it there, or assumed it as totally natural. I do remember Herb Jacobs (primarily, though we all participated) developing a large simulation for Motorola, where the technique used was, as far as I can tell, indistinguishable from XP.” XP wasn’t published until 1996, nearly a full 40 years later. 1957! Yes 1957!
- 14. Don’t Blame Royce, Blame the US Military. Winston W Royce: “Risky and Invites Failure
- 15. 1984 : American Department of Defense for System Software Development created DoD STD 2167 It promoted a one-pass document- driven waterfall process. Anecdotally: US Military just took this picture to develop the Waterfall model Why did Waterfall gain popularity? Hierarchical Top down structured approach Strict controls Creates illusions of control Now we have 101 ways of doing the same thing: Developing Software Software development is still in its teens and as such there is not any empirical evidenced based approach to software testing.
- 16. So which methodology is the right one? SDLC 100 Success Secrets - Jeremy Lewis Typical Snake Oil - Not one of the 100 (Supposedly) Asked Questions are anything you couldn’t answer yourself if you have a modicum of experience within software development.
- 17. Common Failings • Not knowing what a PROBLEM is • Not knowing how to change • Not knowing what change looks like • Not understanding the model/s chosen fully • Not spending the time to implement the change • No real desire for the change, just a general feeling • Choosing the wrong model or choosing one model • Not having a detailed delivery plan • Not having clearly defined goals “A goal without a plan is just a wish.” Antoine de Saint Exupéry
- 18. Assessing the Current Situation Assess for Success
- 19. What is a Problem? Something is not working? Yes? Not strictly true Who says it’s not working? In what way is it not working? How do you make a problem go away? a) Change the difference b) Change the perception c) Change the desire Jerry Weinberg calls people who rush to solve problems as a "solution problemmer". Not a problem solver, but a solution problemmer. Basically: "If our solution doesn't work, *you should change your problem*.“ “A difference between what is perceived and what is desired.” More specifically: A difference /according to some person/ between what is perceived /by some person/ and what is desired /by some person/. “An undesirable situation that is significant to and maybe solvable by some agent, though probably with some difficulty.” Definitions and Stories, from and inspired on Skype conversation with Michael Bolton
- 20. Test - Serve the Business Testing is a service role. Feel good about that. The service you provide is vital. Service implies clients—the people you serve. Your success is measured primarily by how well you serve your clients' desires and best interests. Project Managers PMs are entitled to know your process and influence it. You serve the project manager by reporting your status on demand, reporting important problems fast, and not being a bottleneck to the project. Document Writers/BAs Like you, the people who write the documentation get incomplete information. You can help them understand how the product really works. Writers can help you, too. As they do their research on the product, they will learn things that you don't know. Technical Support Whatever problems are left in the product they become a burden for the people who provide technical support. You serve support by alerting them to aspects of the product that may trouble the user. If you work with them, they can help make the case that a bug should be fixed. Senior Management and Share Holders You serve the business. That's why you must be careful not to sound or act like a quality fanatic instead of a reasonable person. Express test status reports in crisp, operational terms, so that executives feel they have a basis on which to make decisions. Marketing/Sales Marketing & Sales need to know whether anything in the product is inconsistent with the key benefits the product is supposed to provide to customers. A bug that seems minor to others might be critical to marketing & sales. End User In your heart, you serve the people who will make use of the product. Their satisfaction is in the best interests of your project, of course. But there is also a special satisfaction that goes with being the primary user advocate on the project team. Developers You make the developer's job easier by providing good bug reports, as soon as possible. Strive to know your craft and know the product so you don't waste the developer’s time with mistaken or frivolous reports. Lessons Learned in Software Testing: Section 1.3. Lesson 3
- 21. Talk to Everyone “Talk to someone about themselves and they'll listen for hours.” Brush up your social skills Understand what is wrong Access morale Gather information uncritically Understand desire to change Gain constructive feedback “Any fool can criticize, condemn and complain--and most do.”
- 22. Management opinions Stay Neutral Don’t take sides Be open to discussion Constantly invite feedback Discuss opinions rigorously All opinions are weighed equally At this stage, they are Everyone has an opinion Be able to defend your arguments in a rational way. Otherwise, all you have is an opinion. - Marilyn vos Savant
- 23. To gain a fuller understanding Get you hands dirty and test Get familiar with the tools Get familiar with the processes This is essential To gain respect To gain direct experiences To understand the products Also get to understand your competitors 33% No – 30% Probably and 37% Definitely – That’s Scary
- 24. Delving Deeper • How do teams work together? Are they working closely or are they siloed? How does information flow between teams? Is it managed in any way? Are people expected to source the information themselves or does the project manager ensure the right people are kept in the loop at the right time? • Are people in teams listened to? Does one person in a team feel they are not listened to yet another has a totally different opinion? Why is that? • How have changes been implemented previously? Did they work, if so how did they work? If not, why not?
- 25. Delving Even Deeper • What is the current process? Is it documented? Does everyone know what the process is? • Is there a roadmap? Do people know what is happening in two weeks’ time? • Does test know when the next product is coming in to test? Are test consulted about the test schedule? Is there a test schedule? • Are test involved at the design stage? Are test involved in any stage of the development process? Is test only involved when the product is code complete?
- 26. Creating you plan for change Ideas are now growing roots Ideas for change based on: Experience with previous change Practical hands on experience Everyone's opinions Personal experience Change is now inevitable Not just an idea There’s a time line This shits now getting real, baby.
- 27. Argue and discuss changes: Convincingly Collaboratively Respectfully Understand objections: Is it practical Is it appropriate Be prepared to: Throw away plans Change plans drastically “No man is an Island” - John Donne James Bach
- 28. Summary: Assessing the Current Situation • Talk to everyone and Listen to everyone • Do not weigh any option above any other • Stay neutral, be uncritical and stay positive • Continuously gain feedback • Gauge desire for change • Gain support and buy-in • Keep your enthusiasm levels high • Understand current processes, what works and what doesn’t • Understand how to communicate effectively • Learn to argue in a Socratic style • Interact directly, do the job of the testers • Start planning early • Create, modify, scrap plans and constantly evaluate plans NOTE: I talk about measuring the success of new changes in the last section
- 29. Planning how to Deliver Improvements Plan for Success Think outside the box, while retaining what’s useful in the box
- 30. ALL! Changes for improvement identified Don’t be fooled Some ideas will be scraped Some ideas won’t get buy-in Continue to talk, discuss & collaborate It’s not about you It’s about improvement Keep management in the loop Create a realistic time line They don’t like surprises Don’t: “Over Promise and Under Deliver” or “Under Promise and Over Deliver”
- 31. Get buy-in To agree with; to accept an idea as worthwhile. To communicate a vision and create buy-in. From the supporters From opposes, this is essential From management From the ideas people From the people who desire change From the people who hate change “Deep and sustainable change... requires changes in behavior among those who do not welcome the change.” ― Douglas B. Reeves
- 32. Process, Process, Process! Avoid like the plague Process changes are needed : Yes Process for sake of process : No Processes needed : Yes But streamlined, not reams of docs The won’t be read anyway No Best Practices PLEASE! “A “best practice” is an idea that a consultant thinks he can sell to a lot of people. There is no assurance that this idea has ever succeeded in practice, and certainly no implication that it has been empirically tested and found superior (best) to competing ideas under general conditions.” Cem Kaner Lets not call them Best Practices, lets call them Good Practices Best: The highest quality, excellence, absolute qualifier, context independent Practice: Habitual or customary performance Therefore: Best Practice is: The highest quality of habitual performance with no context Lloyd Roden Definition
- 33. Summary: Planning how to Deliver Improvements • Keep talking • Get Buy-in from everyone and anyone you can • Be prepared to constantly evaluate plans for change • Collaborate • Keep your mind on the end game, Improving Test • Introduce small changes early • Always have you eye on the time line • Do not “Over Promise and Under Deliver” • Do not “Under Promise and Over Deliver” • Be realistic with your goals and time line • Don’t write numerous reams of documentation, they won’t be read • Don’t create numerous new processes, they will hate you for it • Don’t create ANY Best Practices • Get agreement
- 34. Inspire and Measure Deliver for Success Leadership comes in small acts as well as bold strokes - Carly Fiorina
- 35. Enable, Empower and Enthuse Lessons Learned in Software Testing: Read it, don’t expect others to though Lean Coffee sessions: Explore testing ideas Pair Testing: Learn together, challenge Challenge testing ideas: Challenge the status quo Challenge, push and drive: Question, question, question Encourage, nay demand feedback: Focus on those who don’t feedback Praise with sincerity: Don’t praise for sake of praise “Some men (sic) have thousands of reasons why they cannot do what they want to, when all they need is one reason why they can” ― Martha Graham
- 36. Measurement, ARGH! Measure for improvement Measure sparingly Be aware of error margins Don’t set targets to be achieved Metrics give you information It’s primarily about PEOPLE not metrics Gain feedback on metrics Ensure test are involved with metrics Recommended Reading: Measuring and Managing Performance Organisations : Robert D Austin. "When a measure becomes a target, it ceases to be a good measure“ - Dame Ann Marilyn Strathern
- 37. Did you take away these ideas from this talk? • Software development hasn’t advanced as far as people like to believe • Too many ideas, little consensus and no proven methodologies • No one software development methodology is the holy grail • Change is difficult, move forward slowly and throw away broken processes
- 38. Conclusion • There’s no such thing as best • There’s no such thing as perfection • Don’t create process after process • Don’t create any Best Practices • Don’t create reams of documentation • Change, challenge, and learn, together • It’s about people, not how to control people “If you meet the Buddha, kill him.” - Zen Master Linji – Blog: Adam Goucher on Heuristics
- 39. Questions? He must be very ignorant for he answers every question he is asked - Voltaire
Notas del editor
- Just like the old cliché, the delivery chain of software is only as strong as its weakest link.It is clearly a literal fact that a chain is only as strong as its weakest link. The conversion of that notion into a figurative phrase was established in the language by the 18th century. Thomas Reid's Essays on the Intellectual Powers of Man, 1786, included this line: "In every chain of reasoning, the evidence of the last conclusion can be no greater than that of the weakest link of the chain, whatever may be the strength of the rest."
- Answer: Countless. There are the methodologies you have just shouted out and listed, the ones with names that have been publish in a plethora of books on the topic.However; there are also the methodologies that you use, that are more than likely based on some of the recognised models, but have also been modified to suite your individual workspace, some of you might have a name for the ones you use, but I’d wager most of you don’t.That’s actually a good thing, as I don’t believe there is one model you can follow, this I will talk more about later.
- This talk was inspired by a phrase from a previous hiring manager “Stephen, QA is Broken, I want you to fix it”This sounds exciting I thought, and I got ready to immerse myself fully in to this new role with relish and desire. However I then started to think how do you though ensure you succeed? Is there a book, a course a clearly identified framework a check list of daily activities that will ensure this success?Short answer “No”
- There is of course plenty of material out there in all off its various guises, unfortunately how do you filter out the good from the bad and the downright barmy ideas from the snippets of wisdom?We all have a fantasy of how the ideal working environment should be, but how do we even try to achieve that goal. In the mean time we have to deal with all the current broken processes as best as we can. Unfortunately there is one way, ever organisation is different therefore the solutions will be different.
- TEST teams and the organisations they work in are varied in multitudinous ways and as such no one approach will work. You have to use a collection of resources that have the potential to work for you with your aim of improving TEST.
- At the end of this talk you will have some understanding of what is required to improve TEST. The main traits required are hard work, boldness, ability to change, a thirst to learn and most important of all a clear idea of the goal you are trying to achieve, not some hazy uncoordinated bullet point list of what needs to change, that’s the easy part.Ideas I want people to take away from my talk:Software development has not advanced as far as people like to believeTo many ideas, little consensus and no proven methodologiesNo one software development methodology is the holy grailChange is difficult, move forward slowly and throw away broken processes
- The title to this talk is catchy and succinct, but what is QA and what do I mean by broken.Obviously QA means strictly in this sense “Quality Assurance”, which on its own is misleading as in my opinion teams named QA are not assuring quality. What does “Quality” mean? A dictionary defines it as a “certain level of excellence”. “Excellence” meaning “Extremely good” and “Assurance” means “Keeping promises”.So Quality Assurance means:“Keeping promises to supply Products or Services to a certain level of excellence.”The problem with the word Quality is that it means different things to different people. What looks good to one person looks dreadful to another, it’s all a matter of personal choice.
- This exert is from a Paper Called: The On-going Revolution in Software Testing: CemKaner, J.D, Ph.D.: Software Test & Performance Conference, December 8, 2004A better word to use for me therefore is Test, as a test team’s responsibility is to reliably report on what has been tested, what hasn’t been tested and give a level of confidence on the stability of the product under test. They cannot assign any meaningful attribute as to the quality of a product or give an assurance.Some people don’t think it matters, personally I think it matters significantly as the term “Quality Assurance” has way to many connotation's and as such is meaningless, therefore I will continue to call it what it is “TEST”Everyone is involved with quality
- By broken, I could have used a different more provocative title such as.We are all doing it wrong, because no one knows how to do it right, including me.Now that is a bold statement, so let me take it apart and explain what I mean.“We are all doing it wrong” OK yes it’s a title to create a reaction and get you to think about what you are really doing in your test environments. I’m sure you all have differing ideas on what’s not working, that’s guaranteed and that’s the easy part. But where are the solutions, where do you get your solutions from?“Because no knows how to do it right, including me.”Do it right, what does “It” mean here? I am specifically talking about developing software in an efficient, rapid, consistent and streamlined manner that meets customer expectations both in requirements and schedule, and although there are defiantly some teams that are developing software following clearly defined agile practices from my experience it’s a bit hit and miss. Why is that?To give you some idea as to why we are were we are I’ll go through a Brief History of Software Development, where I will try to wring out why I think we’re all doing it wrong? And more importantly we are thinking we are doing it right.
- The problem as I see it here is that the software development community has so many different methodologies that it’s difficult to actually determine which (if any) is the right one for your development/test teams to follow.Secondly just to make the problem even worse there are methodologies that use several parts of several other methodologies, I’m not saying this is right or wrong but it does create a proverbial minefield that you have to navigate through to find the right path for your team/s.
- To take for instance Agile™: how many differing Agile™/agile practices are there? For one the term Agile™ or agile are often quoted as a development methodology when in actuality Agile™ it’s self is a statement a manifesto, it isn’t in anyway a structured outline of how to develop and test software.The Agile™ Manifesto states:We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: Individuals and interactions over processes and toolsWorking software over comprehensive documentationCustomer collaboration over contract negotiationResponding to change over following a plan That is, while there is value in the items on the right, we value the items on the left moreThe Manifesto is a statement of desire to prioritise a more positive personable approach to software development with less process and documentation based approach.Which I’m confident most of you here will at least have some familiarity with.
- So let’s start at the beginning:When I was doing the research for this talk; I had believed the evolution of software development had a linear flow and to have started with the Waterfall model then to the V model then to Agile practices, rather than what I actually found out which is that varying methodologies have been in use in one form or the other around the same time periods. The first known instance of using an agile methodology for software development actually dates back to 1957; yes that’s 1957!Although the term Agile wasn’t specifically defined, an iterative and incremental approach to software development was being recognised nearly 60 years ago, but didn’t really start getting any real traction until the mid-1990s’, when a plethora of books started being published even though there are many papers to the contrary.So knowing that fact why aren’t we all using iterative and incremental approaches to software development? Why is that we are still mostly using broken waterfalls with smatterings of iterative or incremental approaches?
- For starters the Waterfall Model was specifically created for manufacturing and construction industries not for software development. But as software development at the time had no known formal methodologies the Waterfall Model was adapted. The first known formal description of waterfall model was in 1970 by Winston W. Royce, in a paper titled "Managing the Development of Large Software Systems" although he didn’t coin the phrase “Waterfall Model” the article details why the waterfall model is broken, yet even though it was critiqued this methodology has been the model that has been adapted ever since in one guise or another.Royce in fact said that it was "risky and invites failure" and went on to describe Incremental development. He also went on to say that this top down approach was only really valid for long term large development activities.
- The big step towards popularity for the waterfall model came in 1984 when the American Department of Defense for System Software Development created DoD STD 2167, a military standard for Software Development, anecdotally stated that it was created from the classic picture of the waterfall model on page 2 of Joyce’s paper. It promoted a one-pass document-driven waterfall process.So why did the Waterfall Model become the most common approach to Software Development?I posit that Waterfall methodologies became the norm as they fitted well with the Hierarchical structures within organisations at the time and as such were and still are deemed a good fit for ensuring top down structured approaches that apply limitations and strict controls.So software development has been broken for a long time due to the need to create a formal approach, however the approach that has generally been adopted didn’t work and has never worked, but at least the people at the top had a modicum of control which created the illusion that everything was working fine.Software development has been around for a relatively short time and due to that there are a hundred and one ways of doing apparently the same thing, creating software. However compared to the sciences, software development isn’t yet out of its teens and as such there really isn’t an empirical evidenced based approach to software testing.So we just have to fumble along with the knowledge that we currently have and continue to improve.
- A good analogy to make here would be with Golf: There is so much information out there you really have no idea what the right training regimes are to follow and the best way of improving your handicap. You can only do this when you have tried many different methodologies until you eventually realise that ultimately there is no one way that’s right for all people. No one person is the same so a fix for one person’s hook won’t necessarily work for yours. And seeing as I’ve mentioned fix, there isn’t one, you just have to practice, practice and practice. Throughout this time I read so many conflicting ideas that it became a mind field of idea’s and not knowing which methods where correct for me. I had three different golf coaches; each one had a different method and different ideas on the basic principles, even down to the most basic aspects of golf as in how to grip your club, all three coaches had a very different approach of how to approach gripping the club and slightly different positing within the hand. I tried so many different things that I started to be disillusioned with the whole thing and started try to develop my own training regimes and how to play on the golf course. This happened slowly over time as I was randomly reading information and the end result was that from all the other books and information I naturally absorbed what worked for me and the way I play golf and my attitude towards golf.The same applies to software development methodologies you have to sort through a large amount of material to discover what will work for you and your development and testing teams.
- Choosing a model that works for your environment isn’t easy and it will probably may not even be one methodology that you eventually adopt, or at least I hope so. You have to gain an understanding of the numerous methodologies available and find a way to make them complement with your own working environment. Complement, not crowbar into an already broken process.Failure in implementing change can often be attributed to having the desire, but not really knowing…Not knowing how to changeNot knowing what change looks likeNot understanding the model chosen fullyThis can lead to other problems such as choosing the wrong model for starters then fearing owning up to failure and continuing to fix something that’s intrinsically broken.Not spending the time to implement the changeNo real desire for the change, just a general feeling that something needs to changeChoosing the wrong model or choosing one model, which is not a great idea as that will be too restrictiveThe most importantly attribute that is often missing is not having the people or person who is strong enough to instigate the change.So as you can see to try and improve development and testing processes you really need to understand what you’re getting yourself into and from the off you have to understand what your ultimate goal is. Attempting to instigate any change that doesn’t have a clearly defined end goal will be doomed to failure or at the very least never get to a software development and testing process that actually works for you and your business.
- So you’ve just started how do you go about finding out what’s wrong and where to make improvements? This part of the process is relatively easy in comparison to planning and ultimately implementing agreed changes.Firstly you need to know that if your staring somewhere new this immediately puts you at an advantage as no one knows you so you have a short period of time where people are learning about you and your personality and during this optimum period they are going to be (well sometimes at least) more open to discuss their ideas and listen to you suggestions. You can worry about appearing to be an interfering busy body later. Therefore it’s crucial during this short period, (optimally about two weeks) to make sure you get to speak to everyone within the business that you will be involved with, either directly or indirectly throughout the software development process.
- What is a problem? This seems like a simple concept, I have a spade the handles broke, that’s a problem, obvious.But is that the whole meaning of the word problem? No.Try this: From a Skype conversation: [06/05/2013 19:37:27] Michael Bolton: I bought a computer for my kids a few years ago.It had Windows Vista on it.I expected it to suck.It sucked.I assumed it would suck.It sucked.Is that a problem?Not to Microsoft or the other 10’s of millions of people who bought Vista and where happy with itBut it’s obviously a problem for meSo a problem is: “a difference between what is perceived and what is desired.”More specifically: A difference /according to some person/ between what is perceived /by some person/ and what is desired /by some person/Story:At work a manger talks to me and says Lisa is failing in her work, she’s keeps missing things, she’s not doing enough testing, we keep having to many problems on live, I don’t think she’s a good tester, we’ve got a problem. So we’ve got a problem, we have a poor performing tester in this managers opinion. I then spent an hour with this tester not from a let me find out the problem perspective, but from the point of view, let me just see what she is doing, so we pair tested for an hour. During the session she wrote lots of things down, we both learned some things, tried to figure out better ways we could test, I had no problems with this tester’s abilities and willingness to learn at all, so where was the problem? The next day the same manger comes over to me and says brilliant work Stephen, she’s great now. What had I fixed? The only thing that was missing was the detail of the information that she was giving in the testing tasks. But during the paired testing session, she naturally wrote more details down and I said put that in your report, specifically what you have tested but more importantly what you haven’t tested. Problem solved, or more specifically, perception that there was a problem changed.What can we do about a problem?So when you’ve recognised what the problem is and who perceives it to be a problem and for what reason they perceive it to be a problem, then you need to decide what to do about the problem.Change the differenceChange the perceptionChange the desire.The difference between I want this to do "a" but it does "b", what is "b", why was "b" unexpected, how are "a" and "b" different, how much difference is there between "a" and "b“Second Definition of a problem: This is for problems that are very hard, if not impossible to solve.Is Death a problem? Its undesirable, but we don’t call it a problemIs a meteorite that’s about to hit your house a problem? WE can’t do anything about it, so it’s not a problem, the aftermath is different.The point being that not all perceived problems are problems. Too often people rush for *process improvement* or some other buzzword--addressing a difference--instead of addressing the perception OR the desire. In fact, too often they rush for process improvement without understanding the problem first.I prefer to think in terms of problem solving, as improvement of people’s perception of what a problem is, THAT you could do IF you understand the problem.Jerry Weinberg calls a "solution problemmer". Not a problem solver, but a solution problemmer. That’s the common mentality.Basically: "If our solution doesn't work, *you should change your problem*.“[06/05/2013 20:25:36] Michael Bolton: The point being that sometimes it takes a very small amount of work to effect a very large PERCEIVED change.
- Test serves the whole business and a great way of understanding what tests role is within the business is discussed in “Lessons Learned in Software Testing: Section 1.3. Lesson 3: You serve many clients” Knowing this and acting on its implications will help to dispel age old opinions of the test team, that there blockers, they slow down the release process, there negative, the just like to brag about finding the latest’s bug, they find all the really important bugs at the 11th hour. All of these views are negative and it is your role to change that viewpoint and create a team that is no longer perceived as a hindrance, but a team of skilled professionals willing to help and not hinder.This to me is off primary importance without having a team that is seen and acts positively to everyone within the development cycle from end to end will only hinder your plans to implement change and create a Test Strategy that others see as beneficial and not another list of hoops that people have to jump through to gain sign-off.Story:Change that impression, call it Test.This I failed to do in my current role, for now, even though I used the explanations of what QA means, changing the name to Test seemed to suggest to the team negative connotations. QA sounds better and more professional, Test sounds demeaning and as such even though they understood the difference, they didn’t want a name change. So we moved on, for now, I hope in time with more persistence from me they will come to see that Test is a more appropriate name for the team.Introducing the outline aboveThe Outline here of what Tests role in the business should be, was also looked on as negative, someone even said it makes us look like slaves. However when I went through it in more detail, that negativity diminished somewhat and it became to be seen as a more positive way of looking at Test, outside of test. So it eventually got stuck on the wall, with a few residual objections
- Although you’ve been hired to improve the test team you are going to need everyone’s input, yes you can go about just putting in to place plans that are specific to the test team and don’t require any input or actions from other departments. However going down that route is not going to have the biggest impact; you need to look at the whole deliver life cycle.Something you are 100% without a shadow of a doubt going to have to do and learn to do well is know how to gain the information you need to identify where improvements and process changes are needed. To be able to do this as efficiently and effectively as possible then you are going to need to brush up on your social skills with that in mind I highly recommend that you read and put in to practice.“How to Win Friends and Influence People” Written by Dale Carnegie first published in 1936, and is still the foremost book on the topic and as such of essential reading for anyone who wants to become a leader and influence change. If you haven’t read this book, then regardless of your current position I strongly advise you do.So your first role is to get to know everybody, talk to them, find out what they think about the current situation and processes in place. Do not at this time (or at any time for that matter) make any disparaging comments, especially if the person you are currently talking to has very negative opinions on the current state of the development process. By all means be sympathetic with their frustrations but try and try real hard to enable them to see that things can change especially with their help and constructive feedback.Story:I’ve been a fairly strict and over bearing micro managing kind of manager previously. Not easily approachable, can be over bearing and not very personable. I’ve had to put a lot work in to improving this area, and during the process I’ve made a tonne of mistakes, I still make mistakes but I’m getting better. Previously if something I had asked to be done wasn’t done the way I had asked, then I’d get moody and probably sulk, now that’s not very professional or constructive. The reasons for this failure was my own fears of failing so I put the pressure on others and consequently they failed to delivery to my high expectations. They didn’t actually fail, I failed to first communicate the task properly, ensure they understood the task, worked with them directly to get the task moving and provide feedback whilst undertaking that task. All sounds easy, and it is really, but I constantly failed to do any of these things or some of them constantly. I’ve read other books of this ilk but this one for me has more clarity and is more directly related to aspects of my communication and managerial skills than others.
- Story:I’m very opinionated, I’ve got one about almost anything, regardless of how little I know about the whole subject. So I actually use that sometimes, to make an opinion on how I see something should be done, but in a challenging and open way, rather than a statement of fact. So for instance: I would say: Where I last worked I found that:Test and Dev worked together better when test wrote detailed bug reportsThis is said to testers (but not exclusively) Dev and Test worked together better when Dev understood test scripts used to testThis is said to developers (but not exclusively) Obviously this is true, well it has been where ever I’ve worked, but it does imply extra work, and process, it also implies a better working relationship. The main thing it does though it gets a conversation going. In this conversation I find it easy to wean out how things are working between test and dev without actually critiquing anything. As that in the main meat of that question, it’s a starting point.I also write everything down against the questions asked, I don’t tabulate against names though, that ensures I stay neutral. I have a template of questions which can and often does get expanded upon. The basis of these questions is to get to the facts and steer away from assumptions.Another one I’ve used:I love/hate agile, what’s your thoughts on it?I say love or hate depending upon the kind of information they have already given me as the last question often highlights there thoughts on Testing or Development process models.So I gather opinions in this way, more conversational, even though I still writing stuff down. All opinions are not equal, but all opinions are worth getting gathering at this stage.
- Story:The last two places I’ve worked I’ve had little to no experience with the products they produce. One was a betting and gambling organisation and the other was working with source/version control replication products. Now I’ve not actually been hired in to either of these roles to specifically test, I’ve been brought in as a test manager, so it would be very easy for me not to not actually do any testing what’s so ever. Would you respect your manager if they couldn’t test? Would you respect their opinions on testing matters if they couldn’t test? Would you respect their opinions on risks, testing schedules and testing priorities if they couldn’t test? Well I for one wouldn’t. So for me it’s essential that you know how to test the products in your organisation that you team/s are testing, only in this way will you be respected and only in this way will your opinions matter and have influence on the team/s. I find this latest stat from the Software Planet’s latest publication rather worrying. 33% have indicated that it’s not important. Even scarier is that 30% of people only think you PROBABLY should, a measly 37% think definitely, something’s wrong there I believe.
- Now you starting to gain direct and indirect information on how the testing team works and how the development life cycle works so now you need to get down deeper in to this understanding.How do teams work together? Are they working closely or are they siloed? How does information flow between teams? Is it managed in any way? Are people expected to source the information themselves or does the project manager ensure the right people are kept in the loop at the right time? Are people in teams listened to? Does one person in a team feel they are not listened to yet another has a totally different opinion? Why is that? How have changes been implemented previously? Did they work, if so how did they work? If not, why not?
- As you can see there are a hell of a lot of question I need to find answers to and these lists are in no way exhaustive.And many more questions, all of these questions have to have definitive answers to, then in that way I’ll be able to evaluate what changes can be implemented that will give the greatest improvement in the shortest period of time, backed up by longer term goals and plans.
- Story:In any relationship it’s always easy at the start until they start noticing annoying habits. The conversation flows as you learn about each other. This is no different. I find this can be difficult, especially when I’m asking people, “So how long do you think this will take?” It’s quite amazing when I ask this and there isn’t really a real target to aim for the replies are usually “Oh, not long” or “a few weeks” to “a couple of days.” Then I talk to Mike and go through the details and then a sudden realisation seems to occur and any estimate I got before is now, way different to the one I’m getting now, let alone the sudden objections and misunderstandings, like “oh I didn’t realise that was going to happen so soon in that way.”
- This is now when you need to be able to argue with each other as do other professional communities, scientists, academics, engineers etc. and not run away crying when people do not agree with you.
- Story:One place where I worked as a consultant I noticed one area where change could be implemented quickly and would provide huge benefits, however the feedback I got towards this idea was extremely negative. People agreed that it would be great if we did it but not one person thought it would ever be agreed to and therefore would not ever happen. This is a classic example of just making a huge assumption that something will not change and completely believing that this was a fact and thereby not even getting in to a conversation regarding that change.The specific change was not that controversial I thought at first nor was it that hard to implement. All it was, was to create a clear development sprint period of two weeks and then a test period of one week. Presently what was happening was that tasks were put in a sprint worked on until all tasks where completed regardless of the time taken, this could be up to a month. Then test where given a short period of time to test, say 10 days. The quality of this testing was poor and test where being seen as being inefficient, missing too many problems and basically a hindrance. The relationship had broken down, test where just being treated as the end game, not being integrated with the development process at all.So regardless of people agreeing that this change would be beneficial, no one had attempted even suggesting it due to believing it was pointless to even suggest as it would be rejected. And by the way it was agreed to and implemented with ease.
- As this topic is overarching it would be impossible for me to state exactly how this is done in an hour, and as the tenure of my talk has been that there is not one development/testing model to follow, nor is there one Planning for Change model to follow. Yes indeed you are going to have to have a detailed plan to deliver to management for them to see how change is going to happen over time, but this does not have to be a detailed step by step process to follow and defiantly should not be restrictive as in do A then do B then do C etc. That is way too restrictive.One point I need to clarify, is that although I’m talking about process, investigation, planning and delivery, all of these will be happening simultaneously. So small changes can be identified very early on, planned and commenced whilst your overall plan is still being crafted.Therefore I propose approaching planning for change using a Meta process. Opps that distinctly sounds like I am proposing one model, I suppose I am but based on my personal experience using a model to implement changes that allows flexibility is distinctly more desirable than having to follow strictly defined incremental steps.I personally believe that the most affective changes are ones that use the current process models in place and evolve them in the direction of the changes I want to implement. I’ve read that it takes 66 days to develop a good habit but only half that time to develop a bad habit. Yes it’s often quoted that it takes 21 days, but that data has no founding in evidence. Also every change regardless of how small, say deciding to eat a piece of fruit everyday still takes on average 66 days to become a good habit. This needs to be borne in mind when planning for change. Start small and incrementally add new changes over time.
- Story:“Under Promise and Over Deliver”, invites low quality and low performance and is downright dishonest.Do not “Over Promise and Under Deliver” or “Under Promise and Over Deliver” (the latter is being obvious), be realistic with setting timelines. Do not try and be sneaky, knowing that I can get something done in one week but then saying it will take two weeks. If I think something is going to take one week, then I’ll say one week. Taking the “Under Promise and Over Deliver” route will eventually be found out then any trust I had will have been lost.
- Story:How you can instigate change if no one wants it? Well you can’t, that’s why you need, must have buy-in. Even if I had the greatest idea ever, if no one can see how it’s going to work and help and improve what they are doing, then they won’t be doing it, regardless. I’ve found that you have three choices. 1: Scrap it all together. 2: Repackage the idea and try to get understanding and buy-in. 3: Do a pilot, and then let the results speak for themselves. I’ve had difficulty before with someone who due to a misunderstanding with how change happens literally asked me “Do I have a Vito on changes?” Mainly due to a strong difference of opinion on how test cases should be managed. The situation was that the team wanted to have check boxes, or pass, fail dropdowns next to each test step in a test case. I vigorously objected to this, explaining this would add unnecessary admin tasks.” But some test cases have 20 to 30 steps” “Then those test cases need to be broken down”. After the debate, I said I’ll change the test tool to allow you to do this temporally, but I guarantee you won’t want it. Did they want it when they saw how it would impact there work, no. That’s an example of how quickly you can do a pilot and get results almost immediately; change isn’t always about massive amounts of planning.
- They will not even be read, as The Social Life of Information: has shown that most of the documentation written for business practices is never ever read.The Social Life of Information by. John Seely Brown and Paul Duguid. Harvard Business School Press, February 2000. ISBN: 0875847625Story: Lloyd Roden:Most documentation isn’t read, so when you do write documentation, make it interesting, make it unique, and make it focused to its intended readers. How many here write test plans? How many here know that these test plans are read? I bet most of the test plans you have written are mostly a copy and paste of the last test plan with a few small changes for the new project. That’s boring, who’s going to read that, so why are we doing it? Because we’ve always done it, the Best Practice, procedure says we should do it. Well stop it, that’s not testing, only write was absolutely necessary and allow testers to think, not just tick boxes, challenge the status quo.
- Do not become a process or Best Practice junkie, what you should really be doing is actually instigating improvements that work and will have a positive impact on the development of your software and encouraging the teams buy-in and assistance. You are not an island, you are not there to prove anything, you are there to help, advise and put your mouth were your money is and make lasting improvements.
- So we have a comprehensive list of changes that have been agreed to that we now have to implement within an agreed time line, now what? Do you just hand out your plan, put your feet up and wait for all the plaudits to come in with how wonderful everything is now. Thanks Stephen you are super. I wish if you devise a plan then you need to see it through to the end, obviously, you would think. The really hard part is now going to start and you are going to have to get the wheels turning and keep them turning until you no longer need to be the driver and someone else can take the reins. Part of the process of implementing the agreed changes will be to change people’s perceptions of what Test is remember your remit, QA is Broken, Fix it! It is obviously perceived that something is wrong in Test therefore as part of implementing changes a key focus has to be improving the perception of test.
- The best place to start is with test themselves, they will obviously be aware of the opinion that others may have of the team so it is your role to change their way of thinking. To do this requires continual support, feedback and encouragement especially when the shit hits the fan and Test are once again blamed for something going wrong it is your job in these situations to take the hit, constructively.Lessons Learned in Software Testing:Don’t expect anyone else to read this, but do read it yourself and use lessons to help inspire enthusiasm within testLean Coffee sessions:This is a great way to get away from the work and talk about testing and development issues. Can be directly related to the work environment, but ensure negatives expressed don’t turn in to long drawn out moaning sessions. Direct sessions with positivity and enthusiasmCoaching:Can be difficult if you have never done it before, so learn the process together, share materials regarding coaching sessions. Explain I detail that these are session to challenge ideas on how we are testing and try to think away from scriptsChallenge with testing ideasCreate discussion, in team meetings, internal forums or random conversations regarding testing topics, talk about blogs and articles and tweets. Challenge the status quoEncourage, nay demand feedbackI say demand as getting feedback is never easy. There will always be a one or few people who always give feedback, encourage them to encourage others to feedback. Praise there feedback but challenge their opinions and make a point that you want more from those who contribute little. Don’t force, just strongly encourage, some people will never provide any feedback, live with it.Praise with sincerityDon't praise if they haven't done anything that great, this will destroy trust by giving false praise. Give praise when it’s worthy of praise, always praise when in a group, not just one to one. Challenge, push and driveWhen feedback is given, talk fully about it, try to wean out the thinking behind the idea, how would that be implemented, who’s going to make it happen, what’s required to make it a success, don’t just say great or bad idea and then move on.Provide feedback to the people who have been negative towards test as to the reasons why this has happened and provide assurances that this is the reason why a plan has been put in place to reduce the risk of future occurrences. I believe it is OK to fail, but you have to learn from failure and do everything in your power to prevent the same thing happening again in the future, or at the very least mitigate the severity of a similar problem occurring.When things go wrong and they will, people are going to want reassurances that your plan will stop these future occurrences. These are the people that need to be fully kept in the loop with your plan so encourage them to become more involved and that you are happy for them to provide any assistance or advice to help test move forward and improve. Encourage them to actively be involved in the plan for change and ensure they know at each step along the way they know what is happening and how long it will take to see positive results, change is rarely instant.
- "When a measure becomes a target, it ceases to be a good measure,“Which is known as Goodhart's law and is named after the banker who originated it, Charles Goodhart, although it’s most popular formulation was created by British feminist anthropologist Dame Ann Marilyn Strathern. We need help setting direction and measuring progress. But maybe there's a better way to achieve those things while sidestepping goals' negative side effects. I want to propose one: Instead of identifying goals, consider identifying areas of focus. - Michael BoltonStory:Saying in a two week testing cycle after one week that we have 100 tests to run in total, out of that we’ve run 50 tests and found 30 bugs, of which 10 have been fixed. What does that really mean? Nothing! Does it mean in the next week we’ll have the last 50 tests completed and find another 30 bugs and still have 40 bugs open? If testing worked linearly, then possibly yes, but it doesn’t. Testing is unpredictable; you can predict what is likely to happen at an early stage, which gives you the basis for your schedule, but you have to analyse the real data and make correlation between the actually data to be able to predict the likely outcome of still meeting the testing deadline, given current progress. You won’t always get it right, you won’t always be liked, but you have to continually adapt and change and update your predictions based on new evidence. Weather forecasters get it wrong often; they are getting better with their predictions by continually fine tuning there models and collecting more data. But it’s not their fault if there predictions are wrong. Was Michael Fish wrong in 1987 when he quashed rumours about a Hurricane hitting the UK? The next day in the UK the worst storm since 1703 unleashed its fury. No he wasn’t wrong, as his prediction was based on the evidence that was available to him at that time. So nor are you at fault if your predictions on testing progress are wrong, it’s a prediction not a statement of fact.
- The process of change never stops, continuous improvements will always be happening. You have set the process in motion and successfully implemented changes that have worked. Don’t fail now, now that your plan has been achieved and think, finished. Hopefully you will not and throughout the process of implementing the changes you will constantly be devising new ideas for change and going through the whole process from end to end continually.No one software development methodology is the Holy Grail, be cognisant of this at all times“If you meet the Buddha, kill him.” - Zen Master LinjiThis Buddhist Koan does not of course mean that you should literally commit murder. What it is saying is that if you believe you have a correct image of the Buddha or the nature of Enlightenment then you should kill that idea, as it is false. Killing false ideas is important in all testing, including automation. Recall that both activities are heuristic based. This means that any “best practices” or “expert advice” is fallible. It is therefore important to constantly look at the path you are on and determine whether the practice is one that you should continue, or should kill. And that includes every single heuristic mentioned in this this talk. But of course, even what I have talked about is a heuristics - Adam GoucherThere are three main stages to the change process, as mentioned previously some changes can be made quickly and delivered almost immediately, such as why are we not having a daily stand up? It may not be required, I don’t subscribe that it’s a must, but I do want to know if it would be useful.You don’t go through this whole process for each and every change independently before you implement the first change but you will assess, plan then deliver for each change.Assess – What the current problems are and identify areas to focus on for improvementPlan – How are changes going to be delivered and over what time lineDeliver – Put your ideas in to action and guide people down through the whole delivery process, don’t just say what is changing and then let them get on with it.Change is difficult and fraught with barriers, move forward slowly and throw away broken and unnecessary processes and continue to improve. Change never sleeps good luck.