My testing presentation on visual testing - more on the things that we miss when we pay too close attention or fall prey to "inattentional blindness". Discussion on how the brain plays tricks on testers.
2. About Me - @mikelyles
QA Director with over 24+ years in IT:
• Development
• Project Management Office
• Development Manager
• Testing
• Functional Testing
• Test Environments
• Software Configuration Management
• Test Data Management
• Performance Testing
• Test Automation
• Service Virtualization
• Building Testing Organizations
• Defining Processes, Methodologies, Measurement
2
3. Recap of Session Objectives
• An understanding that no matter how good we believe we are as
testers, we have to realize that there is the possibility of being so
familiar with a product that our eyes do not notice changes that
sneak in.
• Tips to recognizing patterns and potential gaps that many visual
testing activities may miss.
• Techniques that can be used in becoming a better visual tester.
3
4. It’s Not the Car, It’s the Driver!
How many times have
you driven somewhere
only to find that you
don’t remember much
that happened along
the way?
4
9. QA Director with over 24+ years in IT:
• Development
• Project Management Office
• Development Manager
• Testing
• Functional Testing
• Test Environments
• Software Configuration Management
• Test Data Management
• Performance Testing
• Test Automation
• Service Virtualization
• Building Testing Organizations
• Defining Processes, Methodologies, Measurement
Mike Lyles
About Me - @mikelyles
9
11. Vision verses the Brain
• If you tell your brain it has
comparisons to make, it will get busy
making them, even if they don’t
really exist
• The brain will sacrifice facts over
efficiency
• Naturally wired to use mental
shortcuts to save time and energy
12
13. Every Brain isWired Differently
• The brain is constantly
evolving
• What you do and learn in life
physically changes what your
brain looks like
• Brains of school children –
unevenly developed as their
physical body
15
14. We Do Not Pay Attention to BoringThings
• Avoiding boredom by multi-
tasking
• The brain is incapable of multi-
tasking
• Everyone today is trying to
juggle emails, phone, tablets,
social networks
• Error rate goes up 50%
16
15. Speaking of BoringThings….
How quickly can you find out what is so unusual about
this paragraph? It looks so ordinary that you would think
that nothing is wrong with it at all, and, in fact, nothing
is. But it is unusual. Why? If you study it and think about
it, you may find out, but I am not going to assist you in
any way.You must do it without coaching. No doubt, if
you work at it for long, it will dawn on you. Who knows?
Go to work and try your skill. Par is about half an hour.
17
16. Let’s Play A Card Game
18
YouTube: Colour
Changing CardTrick
18. We Starve for Relevance
We must do something relevant every 10 minutes to reset our attention
21
19. Stressed Brains Don’t Learn the SameWay
• Brains are wired to deal with
stress lasting <= 30 seconds
• Whether the lion eats you or you
get away – the stress lasts <= few
mins
• Stress of a bad boss may last
years
– A great deal is expected AND
– No control over performing well
22
28. We Are Natural Explorers
• Babies learn by exploring
• The desire to explore never
leaves us
• Good testers will learn by
experimenting like babies
31
30. Your BrainWill FillThe Gaps…
fi yuo cna raed tihs, yuo hvae a sgtrane mnid too. i cdnuolt
blveiee taht I cluod aulaclty uesdnatnrd waht I was rdanieg.The
phaonmneal pweor of the hmuan mnid, aoccdrnig to a rscheearch
at Cmabrigde Uinervtisy, it dseno’t mtaetr in waht oerdr the
ltteres in a wrod are, the olny iproamtnt tihng is taht the frsit and
lsat ltteer be in the rghi t pclae.The rset can be a taotl mses and
you can sitll raed it whotuit a pboerlm.Tihs is bcuseae the huamn
mnid deos not raed ervey lteter by istlef, but the wrod as a wlohe.
Azanmig huh? yaeh and I awlyas tghuhot slpeling was ipmorantt!
33
31. The Gas Pump Dilemna
Which button do you press?
??
??
??
??
??
??
??
??
34
36. Change Blindness
a surprising perceptual phenomenon that occurs when a change in a
visual stimulus is introduced and the observer does not notice it
39
52. None of us are as good
as ALL of us!
Can you guess the number of gumballs? 59
53. Leverage theWisdom of the Crowd
• Same number of people UNDER-estimate that OVER-estimate
• The miscalculations cancel each other out
• Use the crowd to increase accuracy and quality
• Crowdtesting / Bug bashes – increases chances of finding more defects
60
54. Inattentional Blindness
• AKA - Perpetual Blindness
• Psychological lack of attention
• Not associated with any vision
defects or deficits
• An event where individual fails to
recognize an unexpected stimulus
that is in plain sight
Conscious perception is the abnormality
61
55. Inattentional Blindness
• The visual world is abundant
and complex. The brain’s
complexity is not
• Visual stimuli compete for
processing resources
• When important information
loses out, the consequences
can be lethal
62
56. Inattentional Blindness - Impacts
A nurse pulls a vial from a medication cart. She
looks at the label, fills the syringe and then injects
the patient.The patient receives the wrong drug
and dies.
65
57. Inattentional Blindness - Impacts
An Eastern airlines pilot and his fellow officers see a
bulb flash on the control panel.They become so
concerned with the cause, that they don't notice
the plane approaching the ground or hear the
alarm.The crash kills over 100 people.
67
58. How DoWe Cure Inattentional Blindness
• Avoid distractions – pay
attention
• Looking is not the same as
seeing
• Eyes open ! = seeing
something
68
There is no cure….but we can…
60. Anamorphic Art
A distorted projection or perspective requiring a specific vantage point
to reconstruct the image
70
61. Choice Blindness
• Blind to choices and preferences
• People often fail to notice glaring
mismatches between their intention
and outcomes
• Prepared to offer introspectively
derived reasons
• Fail to notice when presented with
something different than what we
really wanted
71
62. Motion
• Visual cortex designed to see
motion whether it is there or not
• Brain sees apparent motion as a
survival technique
• Orienting response – if anything
going on in field of vision we see it
• Peripheral vision – not as strong
72
63. Automation
• Use automation to test things
that are not as easily visible with
the human eye
– A/BTesting
– UsabilityTesting
– GUITesting
• Assists in testing things that the
human brain may find boring to
test after a few rounds
The burning question of the near future will not be how much work a man can do
safely, but how little" (Human Factors, 1980)
73
66. Primacy Effect
• When given a list of information
and later asked to recall that
information, the items at the
beginning (primacy) and the
items at the end (recency) are
more likely to be recalled than
the items in the middle.
• You never get a 2nd chance to
make a 1st impression
84
71. Neuroplasticity
• Brain’s potential to reorganize
by creating new neural pathways
to adapt as it needs
• The more you focus and practice
something the better you
become at the new skill
• Synapses that don’t usually fire
together now sharpen the skills
90
72. What DoWe Know Now?
• The human brain is very complex
• Vision is very critical to us
• We don’t always SEE when we are LOOKing
• Understanding our own brains, its weaknesses and
strengths, helps us design our testing strategies,
methodologies and the tools for which we will use
91
75. References - 1
Brain Rules: 12 Principles for Surviving andThriving at Work, Home, and School:
http://theweek.com/articles/460769/12-things-know-about-how-brain-works
Fooled byYour Own Brain:
http://nautil.us/issue/2/uncertainty/fooled-by-your-own-brain
Video for the DoorTheory
http://www.simonslab.com/videos.html
Where Science and Story Meet
http://nautil.us/issue/0/the-story-of-nautilus/where-science-and-story-meet
Failures of Awareness: The Case of Inattentional Blindness
http://nobaproject.com/modules/failures-of-awareness-the-case-of-inattentional-blindness
Reference Guide: Inattentional Blindness
http://www.yale.edu/perception/Brian/refGuides/IB.html
94
77. References - 3
Video Credits
https://www.youtube.com/watch?v=VT9i99D_9gI
https://www.youtube.com/watch?v=vF2f__k0Pnw
https://www.youtube.com/watch?v=bgTbvyKyPCE
Choice Blindness Research
http://www.lucs.lu.se/wp-content/uploads/2011/01/Johansson-et-al-2013-Choice-Blindness-and-
Preference-Change.pdf
96
Notas del editor
----- Meeting Notes (3/20/16 19:35) -----
1:00
We have all done it….we get in the car, we are listening to music, or we are on a call, or passengers are talking to us…
We get so engrossed in everything that is going on around us, that we don’t even pay attention to our surroundings
Before we know it, we have reached our destination and a light goes off above our head saying “I don’t’ recall anything that went on around me while I was driving”
How were we able to operate? How were we able to know to turn left, turn right, stop, yield, slow down? Are we on autopilot?
Are you amazed at how you were able to do this so many times without having an accident?
----- Meeting Notes (3/20/16 19:35) -----
1:30
I would bet that each of you saw “Mike Lyles is Jumping to Conclusions”, however, when you see the text fully uncovered, it says nothing like that.
This is the first of many examples of how that we can never assume that just because something looks familiar, it doesn’t always match our first impressions
Sometimes you have to dig deep and see behind the initial look and feel of things to know for sure that what you are assuming is correct (or incorrect)
----- Meeting Notes (3/20/16 19:35) -----
1:30
If you have not seen this before on Brain Games or online, you will likely read this as “New York in the Spring”.
Almost all of the people who were interviewed for this picture stated it without the two “THE” words together.
Our brains filter our the 2nd “The”
This is a direct opposite of what we had in the last slide – where we had to make assumptions on what was behind the boxes.
In this example, the two words “THE” are right in front of us – in plain sight, but we do not see it duplicated.
In the testing world, some of the most difficult things to test for and find are ‘hidden’.
But many are right there in plain view.
It is critical that we look closely for them when we are testing.
----- Meeting Notes (3/20/16 19:35) -----
1:00
----- Meeting Notes (3/20/16 19:35) -----
1:00
No, I didn’t accidentally go back to my “About me page”
And no, I’m not a narcissist that just wants to talk about ME ME ME all the time
I brought this back to see if you noticed something in my biography that was subtle
I’m wondering how many of you assumed that the name tag “Hello My Name” was a regular nametag that we have seen many times.
Did you notice that instead of saying “My name IS Mike Lyles” it says “My name ISN’T Mike Lyles”
----- Meeting Notes (3/20/16 19:35) -----
1:00
We have so much to discuss today
I want to share with you some of the definitions we will explore are we go through this session
Each of these definitions have a story to tell
Each of these relate to how we interpret, see, and evaluate what we see
And each of these have a correlation to visual testing
Inattentional Blindness is one that we will spend the most time with today – as it relates to what things we miss visually and why
----- Meeting Notes (3/20/16 19:35) -----
0:30
To have VISION, you have to have a brain.
And I know we can probably name a few people that we know that having a brain may be QUESTIONABLE.
When we look at VISION, it’s behind everything else that the brain is doing
Your eyes are up front, but the information for that vision has to pass all the way through the brain into the back to the Occipital Lobe
When you use mental shortcuts this allows your brain to answer quickly and with confidence
The brain is a complex organWith the complexity comes a constant need for survival
This is developed early in our years
Our environments limit us many times
Your responsibility is to think outside the CUBE / BOX and keep growing your brain
Many years ago it was believed that the brain slowed growth at age 20 and completely stopped growing at 40
We know this is not true today
Each and every activity that we partake in, everything we experience, has an opportunity to shape and remold our brains
School children are in the early stages of brain development – and their brains are growing much like their bodies
Our schools don’t always recognize this – and they expect children to grow at the same pace – generically
It is our responsibility to train ourselves and continue to grow our brains and knowledge daily
Just doing one thing at a time is so 1980’s
Everyone is moving at such a fast pace that they constantly have to be doing something
Just get on a transit system and look around – no one is looking at others, no one is talking – everyone is on their phones or tablets
Go to a restaurant and watch families sit – everyone using their tablets, phones, or doing anything but talking
We try to convince ourselves that we can multi-task – that we are good at it- and that we are productive
Research shows that error rates increase 50% when multi-tasking takes place
In the testing world, it is important that you don’t try to do multiple things at once – to the point where your success is impacted
The best way to be a visual tester, focused and accurate, is to focus – and to not multi-task
----- Meeting Notes (3/20/16 19:35) -----
2:44 for video
1:00 for discussion
----- Meeting Notes (3/20/16 19:35) -----
1:44 video
1:00 discussion
The human brain is wired to look for something relevant every 10 mins
At the time of this presentation – I am challenged to give you something relevant and interesting every 10 mins or it’s just me and powerpoint
We must challenge ourselves to find something interesting in the testing we are doing – every 10 mins
Stress impacts the human brain more than anything else
Whether its your workload, your boss, personal issues, or unfulfilled goals – they can weigh on you and take you down
Stress damages every type of cognition that exists
It can hurt your motor skills
Over a long period of time, disrupts your motor skills
People who are under stress are usually sick more often, can’t sleep, and are depressed
It can actually shrink the brain, cause you to lose sleep, and ultimately affect you for you life
Manage your stress – prioritize the important things – focus on the positives – seek accomplishment
We are here to focus on VISUAL testing
But it is important to understand that if we combine other senses with vision, we increase our learning and memory
These effects have been proven to last more than 20 yrs
I am sorry to be using the word TRUMP in the United States right now!!
There is a reason that childrens books have mostly pictures and not many words
Having a picture associated with text gives us a mental image to store and increases the memory of the situation
So I stopped at a gas station, swiped my credit card at the pump and this prompt came up
“DEBIT Y/N”
The issue was there was no arrows to show me which button was yes or no.
I assumed that the 2nd one down on the left would be NO and top would be YES – most all pumps are designed this way
However, I could not be sure.
But to move on and get the gas started, I pressed the 2nd one down and nothing happened
So then I pressed first one, then 3rd, then ALL of them – still nothing
So without any instruction on the top screen, it was up to me to decide that the keypad down below (normally used only for PIN for Debit Cards) would be the one where you would press YES or NO.
I will tell you that 90%+ school kids will get this right every time
But as adults, we have built our own theories on how to solve things – and to be honest, we have complicated the simple solution at times
If I were to ask you how does a zipper work, or how does a ceiling fan work, or how does a starter work on a car – you likely will try to answer
It’s the fact that we, as humans, feel the need to HAVE the answer, and sometimes we complicate it with overthinking
The problem with answering based on past experiences using common sense, leaves open the possiblity that you’re using incorrect facts – especially if the information is falsely used or not up to date (out of context!)
The answer to this depends on what country you are in --- kids in the USA would say this bus is going left. Because cars drive on the right side of the road, and the door opens on the right side front for kids to get off the bus. Since the door is not visible, this means the bus is travelling left and the door is on the opposite site of the far left of the screen. If you’re in a country where cars drive on the left side of the road, the bus would have a door on the left front, meaning this bus would be travelling to the right.
While things like buses are not universally designed the same way…..there are may things that are designed the same way everywhere…
So many things in our every day life require a design that matches the consumer’s expectations
No matter how many buttons and options a TV controller has, you will always find the POWER button at the top. It may be on left or center or right, but it’s always at the TOP.
Light switches are almost always AT the DOOR. When you walk in you instinctively start feeling around a dark room near the doorway to find the switch.
Placing these buttons/switches in other locations would not only confuse, but irritate, the users
And then we come to this hotel and eveyrtime I press the top buttons I get either menu or ORDER. Well done, Sheraton….
If someone were to ask me to draw a website that looked like ebay (before I knew what I now KNOW), I would have colored it yellow
Ever since I met Ebay, used ebay, sold on ebay, and bought on ebay through the years, I remember it being yellow
But there is a history --- because if you look at Ebay.com now, you will find that is all white – there is no yellow
Ebay didn’t do this overnight….well they DID, but they retracted it.
They chose to go from yellow to white in one update, and customers complained everywhere
So what did Ebay do? They gradually modified the yellow, to a lighter color, and then more, and then more, until it was all white.
This is a perfect example of how we sometimes have to work in UI/UX and in our testing of the product. You have to be aware that users/stakeholders may not be able to handle a drastic change.
In 1987, a movie titled “3 Men and a Baby” came out.
It was a major hit in the USA
But then an urban legend was born…..people began talking about this scene, about the image of a young boy in the window
The rumor was that this boy had died and it was his ghost
Everyone around watched this movie and it took one person calling it out before people noticed it in the window
For sure your card is gone.
But regardless of what card you pick, it would be.
This is because ALL cards changed – the color stayed the same but the suit was changed on all.
This gives the person focusing on ONE specific CARD the illusion that their card was removed.
In testing, we have to be sure not to focus so closely in one area that we miss changes in other areas.
Brains are wired to recognize the human face or anything that resembles it.
When tested, babies preferred the image on the left to the image on the right
We are born with the ability to recognize things that are familiar to us – and faces are one of those.
When we recognize faces, we also recognize gender
A face with more contrasts looks female
In 2009, Harvard Psychologist Richard Russell experimented by showing the same face with more contrast.
The one on the left appeared to be a female to the left.
We can recognize faces even if they are blurred
It’s our tendency to recognize features – there are times when we CAN visualize without seeing.
Face Pareidolia – tendency to perceive something meaninful in an ambiguous stimulous
Virgin Mary on grilled cheese 10 yrs old - $28k
Names of people:
Michael Jordan
Woody Allen
Goldie Hawn
Bill Clinton
Tom Hanks
Sadaam Hussein
Elvis
Jay Leno
Dustin Hoffman
Prince Charles
Cher
Richard Nixon
We can recognize faces even if they are blurred
It’s our tendency to recognize features – there are times when we CAN visualize without seeing.
Face Pareidolia – tendency to perceive something meaninful in an ambiguous stimulous
Virgin Mary on grilled cheese 10 yrs old - $28k
Names of people:
Michael Jordan
Woody Allen
Goldie Hawn
Bill Clinton
Tom Hanks
Sadaam Hussein
Elvis
Jay Leno
Dustin Hoffman
Prince Charles
Cher
Richard Nixon
We can recognize faces even if they are blurred
It’s our tendency to recognize features – there are times when we CAN visualize without seeing.
Face Pareidolia – tendency to perceive something meaninful in an ambiguous stimulous
Virgin Mary on grilled cheese 10 yrs old - $28k
Names of people:
Michael Jordan
Woody Allen
Goldie Hawn
Bill Clinton
Tom Hanks
Sadaam Hussein
Elvis
Jay Leno
Dustin Hoffman
Prince Charles
Cher
Richard Nixon
2011 Cognitive scientist Matthew Thompson shared a distubring face illusion.
Nothing happens if you look at the faces directly
But if you look in the middle – they become grossly deformed.
Canadian Psychologist Donald O. Hebb (1904-1985) studied how people learn
Less and less input is needed once we draw on what we are seeing
He called it Pattern Recognition
A fundamental prerequisite for PR is the ability to distinguish between similar but not identical inputs
Because we are compelled to take incomplete stories and run with them – with a half story in our minds, we earn dopamine reward every time it helps us understand something
In 2003, Stephen Macknik discovered flickers in eyes that he called Illusion of Flowing Motion
When eyes are fixated, about twice a second they make movements called MICROSACADES that are too small to be noticed
Eyes move at most 1/3 of a degree
When we perceive flow, the MICROSACADES rate is increased
Researchers argue that our perceptual system essentially interprets the unmoving dots as SCOTOMA (a visual blemish caused by eye damage) and it “expunges it from awareness”
In 2003, Stephen Macknik discovered flickers in eyes that he called Illusion of Flowing Motion
When eyes are fixated, about twice a second they make movements called MICROSACADES that are too small to be noticed
Eyes move at most 1/3 of a degree
When we perceive flow, the MICROSACADES rate is increased
Researchers argue that our perceptual system essentially interprets the unmoving dots as SCOTOMA (a visual blemish caused by eye damage) and it “expunges it from awareness”
Let’s try an experiment…..
Everyone in the room take out a sheet of paper and see if you can guess how many gumballs are in the container.
Everyone gets a vote (unless there are > 30 people, then we go with 20-30)
Average the total of the gumball guesses
We are hoping for a number within the range of 10% of the actual
The total number of gumballs is ______
The wisdom of the crowd happens because there are the same amount of people that will underestimate that overestimate
These miscalculations cancel each other out
This takes into account the MISSES and comes close to the actual number
IF we are accurate and close the true # of gumballs – we can say it works
If we are NOT close, then we can discuss how that we were surely closer than the person who was the furthest from the actual
Crowdtesting and bugbashes are examples of using GROUPS of people to increase your accuracy and quality
A team of diverse testers may contain members that each find something particular that the other may not
This increases your chances of finding defects before production
Check out crowdsourcing.org for more
Our focus today is going to be mostly around inattentional blindness
This is when a person is focused on one area so much that they don’t see whats going on around them
Unless we pay close attention, we can miss even the most conspicuous events
CONSCIOUS PERCEPTION IS THE ABNORMALITY
Our brains are often “tricked”
Challenge your visual images
Stop accepting NO CHANGES
Example: driving around looking for yellow arrows or blue arrows – ran into bikes of the opposite color
Imagine your brain is like a computer that cannot process information as fast as you are feeding it
How many times have we been using a computer or device that asks us to wait while it processes
How many videos have we waited for the buffering to catch up
How many times have we waited and watched the % download and # of minutes/seconds left to complete
It is important for us to realize that our eyes can see more than our brain can comprehend at one time
Focused attention gets us into trouble
If we are searching for an earring on the floor – everything around us is filtered out
Party discussion – we tone out all other talking
"Only those items which I notice shape my mind - without selective interest, experience is utter chaos"; "The function of ignoring, of inattention, is as vital a factor in mental progress as the function of attention itself.” (William James, 1890).
Gaming – loss of business or errors in the game
Alibaba uses facial recognition for online payments
Gaming – loss of business or errors in the game
Alibaba uses facial recognition for online payments
Gaming – loss of business or errors in the game
Alibaba uses facial recognition for online payments
Gaming – loss of business or errors in the game
Alibaba uses facial recognition for online payments
When you have 2 choices you can quickly take a call
When you narrow down to the one you chose, you don’t always remember what you picked and why
You just CHOSE one.
First bullet = (slight changes in color, slight movement of fields on screen)
A/B – tests design appeals - monitor 2 different designs – tools available to capture user clicks – tons of tools to looka t analytics
Usability Testing – tools available to test different aspect of usability – algorithms to simulate a real user gnerated eye tracking
UI testing – recording actions and location of items on the screen – validating screen properties of the text boxes
Neuroplasticity allows neurons in the brain (nerve cells) to compensate for injury or disease and to adjust activites to new situations or change in their environments
The brains of blind people will compensate by being stronger in other sensory areas
Be conscious of the fact that if you work hard at being a better tester, if you focus more on what you are seeing and not just what you’re looking at – then you can improve
Asymmetric Dominance
In marketing the decoy effect is the phenomenon where customers will tend to have a specific change in preference between two options when presented with a third option that is asymmetrically dominant to the others