Real World Lessons Using Lean UX (Workshop)

real world lessonsmoving to leanux at paypalbill scott (@billwscott)sr. director, user interface engineering, paypalwebvisions workshopmay 22, 2012

the schedule2:00-2:30 the problem (30)2:30-3:00 the pattern (30)3:00-3:15 q & a (15)3:15-3:30 break (15)3:30-3:45 the solution (15)3:45-5:15 the lessons learned (90)5:15-5:30 wrap up + q & a (15)

the purposehelp you identify the common problems encounteredput before you a clear pattern of how it should worktalk speciﬁc ways we solved our problemsleave you with a number of lessons learned you canapply

the problema look at where paypal has been. can you relate?

team breakdownstandard process creates distinct work phasesboundaries are the hand-off points between rolesproduct(business)design engineering

productproduct manager, businessanalystowns the features. doesn’tdo design. drives byhypothesis.typically produces PRD

designUED, UX, ID, IA, VizDe,Content, designernot responsible forengineering the experience,but designing theexperience.typically consumes PRD andproduces design specs.

engineeringfront-end engineer, userinterface engineer, webdeveloper.not the designer, but theengineer that creates theexperience.typically consumes UI specsand PRDs (for context).

typical product life cycleproduct(business)designengineering(agile team)PRD UX spec(wall) (wall)customerdeliveryupon delivery, team disbands and forms into new teams

what was broken in design?late 2011/early 2012

deep silositeration planning done by developers withoutdesigner’s involveddesigners hand off specs without prior involvement ofdevelopersdeveloper days (“dev days”) valued over design timefrequent WFH days created low energy and lesscollaboration timehyper-segmentation of products

broad team distributiongeographic distribution created wedges, duplicationsand blocked collaborationlack of alignment with UED partners (not uncommonto have designers & engineers in same region to beworking on different products)

lack of agile understandingwhile UED interfaced with agile teams they did notparticipate directly in agile planning, retrospectives,etc.agile machinery also did not account for experiencedesign

no sense of ownershipUED staff in a pooled/studio model instead of adedicated modelonce delivery happened the designers moved toanother projectoften engineers did not know who the designer wasfor a product to ask questions toteams not empowered to make decisions as agauntlet of other teams had to approve to go live

what was broken in product?late 2011/early 2012

no measurement/learn culturein several products there were no key performanceindicators to measure & learn againstsince a/b testing was hard to do, there was noconcept of an MVP (minimal viable product)

feature-itussince the organization rallied around projects insteadof products, product tended to try to put as much onthe train as possiblewithout kpis you guess more and more (F.O.G.)without measurement you never get rid of features

too many silosproduct was divided over 9 different organizations!mobile was also a separate business, product andengineering silo

what was broken in engineering?late 2011/early 2012

too many silosjust like our counterparts, we were broken into manydifferent organizationsmobile was a separate organization

too hard to go live37 tickets just to get a bug ﬁxed and pushed to liveevery organization was set up to say “no” to anythingthat might be innovative for fear of failure, risk,security issues, etc.

technology brokenno modern services architectureall solutions were built as silosui logic and business logic intertwinedtechnology platform assumed developers were not tobe trusted

agile way too granularone product had 20+ agile streams. 12 of these wereexperience streams. each stream was responsible forone small part of the experiencecreated nightmares of integrationcreated a fractured experience for users

paypal circa 2011roll your own.disconnected deliveryexperience. culture oflong shelf life. inwardfocus. risk averse.

the patternfollowing a build/measure/learn mindset

a good patterncontinuous customerfeedback (get out of thebuilding - GOOB)customer metrics driveeverythingthink it. build it. ship it. tweak itfail fast. learn fast.lots of experimentation...build/measure/learn

fourdifferent PS3experienceslaunchedon same daylaunching the ps3experience16 different test cells2 different tech blogs weresimultaneously reviewing differentexperiencesfocus was on build/measure/learn

the epiphanydesign for volatility

you have to engineerfor volatilitychange is the normthe ui layer is the experimentation layerexperimentation is not a one time eventlaunching a product is giving birth to theproduct. the product’s life just begins.design for throwaway-abilitymajority of theexperience codewritten is thrownaway in a year

lean startupfounded on build/measure/learnget out of the building (GOOB)invalidate your riskyassumptionsgo for the minimal viableproduct (MVP)fail fast, learn fastget to the pivot

lean uxdesigning products for build/measure/learn (lean startup)requires 3 rules to be followed atall timesget to & maintain a sharedunderstandingform deep collaborationacross disciplineskeep continuous customerfeedback ﬂowing

engineering focusedon learningengineering the build/measure/learncycleshift the focus to minimal viableeverything (MV*)follows the key rules of lean ux:shared understandingdeep collaborationcontinuous customer feedbackLEANENGINEERINGApplyingLeanStartupPrinciplestoBringDesigntoLife

sharedunderstandingthe more understanding theless documentationbut this doesn’t mean NOdocumentationyou need whatever isneeded to gain a sharedunderstanding

deepcollaborationstrong belief that ideas comefrom many different voicestrust is essentialall efforts never stray far fromcollaborative efforts

continuouscustomerfeedbackthis is the lifeblood of theteamgets rid of politicsturns a team outside-in

getting to mvpmvp is a key tenant of leanstartupapplied to engineering weshould think: minimal viableeverything. mv*minimal viable processminimal viable team sizeminimal viable technologyminimal viable toolswhat are startups using?

getting to mvpmvp is a key tenant of leanstartupapplied to engineering weshould think: minimal viableeverything. mv*minimal viable processminimal viable team sizeminimal viable technologyminimal viable toolswhat are startups using?zuck’s rule: how will the startup down thestreet do this tomorrow? do it like that today.

healthy product life cycleDiscoverCustomerInsightsDeﬁneCustomerProblemsDeﬁneSolutionConceptsDeliver &TestSolutionsCDI

the solutionrefactoring your way to the pattern

paypal vs netﬂix

new dna @paypaljan 2012ﬂeshed out ui layer that couldsupport rapid experimentationmarch 2012david Marcus becomes presidentof PayPal

in the midst of transformation

hermes projectre-inventing checkout with lean ux

hermes projectwhiteboardto codecode to usabilityproduct/design/engineering teams usability/customerslean ux in action

free to iterate independent of agileuser interface engineering - agile scrum teamlean ux - lean team trackengineering - agile scrum teamsprint 0usability usability usability usability usabilirelease release release release{agilelean ux can provide a brain for agile

the lessons learnedwhat you can apply to your teams

create a sandboxIMVU allows every engineerto put a test out to 1% ofusersat netﬂix we often createdadditional tests thatdesigners or engineersindependently wanted to tryas a solution to a hypothesis1

hotwire case studyhow do you protect the parent organization from theinternal startup? create a sandboxSource: “Lean Startup in the Hotwire Enterprise” by Kristen Mirenda & Karl Shultz

hotwire case study: feedback“hate it - cant evensort anymore”“I dont like it because youcannot filter the results oreven sort them.. Whatwere you thinking?”“absolutely blows...puregarbage. need to be able to sortasap. ill come work for you andhelp you figure it out. wtf.”Source: “Lean Startup in the Hotwire Enterprise” by Kristen Mirenda & Karl Shultz

hotwire case study: dataSource: “Lean Startup in the Hotwire Enterprise” by Kristen Mirenda & Karl Shultz

move to a“living spec”break down barriersbetween prototyping andproductionuse developers forprototyping as forcingfunctionembrace RITEavoid tools/process that getaway from collaboration2

make the spec realthere are many, many prototyping tools available nowyou can create a livingspec with thesehowever the ﬁdelityis never the same asreal coderecommend HTMLprototyping(more on this later)

but what about docs?watch out for “predictive documentation”watch out for documentation that replacescollaboration or is a band-aid for bad processgood documentation will enhance collaboration,shared understanding and disseminate learnings

use a prototype stackwhiteboardto code code to usabilityproduct/designteamuser interfaceengineersusability/customersto enable learning

use a prototype stackwhiteboardto code code to usabilityproduct/designteamuser interfaceengineersusability/customersto enable learningnodejsJS librariesJS Templating(dustjs)less -> CSS images

enables sketch to codeforcing functionit brings about a close collaboration between engineeringand designit creates a bridge for shared understandingrequires a lot of conﬁdence and transparency

how lean & agile can play togetherfuller integration with services, unhappy paths & hardening error handlingprototyping happens in rapid succession (lean ux track)services agile scrum teamsusability usability usability usability usabilirelease release release release{agilelean ux can provide a brain for agilestories & code sharedlean & agile teams should blend together

code prototypes vs tools?use the right tool at the right timeas you get closer to agileaxure, proto.io, POP and a host of other prototypingtools are amazing -- especially early in the learningcyclecode prototypesimportant once you get close into the actual agile sprintsprovide high ﬁdelity to the user testingfaster cycle from “learning to live”

suggestions for code prototypingbootstrap is one of the quickest to get going withwe use it on our production stack as welljetstrap allows you to drag and drop a bootstrappage to get a quick startnodejs is really powerful for prototyping your fullapplication (web, tablet, desktop)

prototyping toolssee:list of prototyping tools on my blog: http://bit.ly/SfWygkfew that we also use:Axure RPInVisionPOP

engineer forexperimentationlong shelf life to rapidexperimentationfocus on learning not ondeliverydesign for volatilityrefactor the tech stack withlearning in mind3

experiences must learnAll buildings are predictions.All predictions are wrong.Theres no escape from this grimsyllogism, but it can be softened.Stewart BrandOur software is alwaystearing itself apart (orshould be)Recognize thatdifferent layers changeat different velocities

velocity changes by layerrecognize that different parts of tech stack change atdifferent velocities“any building is actually a hierarchy of pieces, each ofwhich inherently changes at different rates” - StewartBrand. How Buildings Learn.design for throwaway-ability (volatility)!“use before you reuse” (optimize for change)utilize packaging or paths to capture experiments

why start with experience?stay honest & pure by having experience be the driver(not what your boss thinks or what looks good onyour resume or what the loudest one in the roomthinks)rememberuse before you reuselet the experience drive the engineeringreuse is an engineering optimization. use is what usersdo. reuse is what engineers do.

experience vs components

build in rapid experimentationthink of the UI layer as “the experimentation layer”early rapid prototyping leads to learnings to get intothe right ballparkfollow with live A/B Testing. Lots of it.creates a healthy environment with constant customerfeedback loopscontrast this with “long shelf life”culture

requirements for lean Stackindependent of the backend languageﬂexible enough to run in either the server or in the clientequally good at building web sites as it is building web applicationspushable outside of the application stack (publish model)cleanly separated from the backend/app code (ideally by JSON)utilize what is common to developersquick & easy to build & tear down components

tangled uptechnologybig problem. technology andprocesses not geared to build/test/learn.refactor your way out oftechnical and experience debt

technical debtwe have to be on modern techstack to continuously innovatewe have to be on a continuouslyavailable stackcontinuously integratingcontinuously deploying

stack circa 2011/early 2012simple change couldtake minutes to seefollows an “enterpriseapplication” model. uigets built into the “app”javajsp***restrictedcapabilities*prototypingwas hard“ui bits”could onlylive here* assumed client developers were low-skill* required server side java eng for simple client changes** java server pages. server-side java templating solutionserver sidecomponents**clientserver

a tale of two stacks (c++ & java)two non-standard stacksnew stack tied to Java“one word” change couldtake 6 weeks to ﬁx on c++stackc++ javaxml jspproprietaryuiproprietaryuiold new’ishlongreleasecycles

decision was to move to javamigration was already inplace to leave the c++/xml stack behindsome odd choicespreceded this: writeeverything in java (html,css & js!!)c++ javaxml jspproprietaryuiproprietaryuiXold new’ish

old stack not designed for learningthis new stack was notconducive to prototypingfollowed an “enterpriseapplication” model. uigets built into the “app”ajax/components all doneserver-side (subclass javacontroller)javajspproprietaryuiprototypingwas hard“ui bits”could onlylive here

separate the ui bitscode = JS(backbone)templates = JS{dust}style = CSS(less)imagesre-engineered the userinterface stack so thatthe only artifacts are:• javascript• css• imagesditched the server-sidementality to creating UIs• no more server-side onlytemplates• no more server-sidecomponents• no more server-sidemanaging the ui

use javascript templatingtemplates get convertedto javascript<p>Hello {name}</p>we use dust.jscode = JS(backbone)templates = JS{dust}style = CSS(less)imagesJavaScriptcompiles to...javascriptexecutedto generate ui

ui bits now just natural web artifactsserver-side language independentserver/client agnosticCDN readycacheablerapid to createcode = JS(backbone)templates = JS{dust}style = CSS(less)images

portable in all directionsJS templating can be runin client browser or serveron the production stackwe can drag & drop theui bits from prototypingstack to the productionstackjava(rhinoscript)node.js{dust}JS templateprototypestackproductionstack{dust}JS templateclient ORservereither stack

embrace opensource fullystop rolling your ownsolutionsdemocratize softwaredevelopmentbuild software that can beshared (keeps architectureclean)4

use open source religiously

work in open source modelinternal github revolutionizedour internal developmentrapidly replaced centralizedplatform teamsinnovation democratizedevery developer encouragedto experiment and generate reposto share as well as to fork/pull request

give back to open sourcewe have a projects that we will open sourcenode bootstrap (similar to yeoman)we are contributing back to open sourcecontributions to bootstrap (for accessibility)contributions to bootstrap (for internationalization)core committer on dustjs project

using github for continuous *use github for continuous integrationstarting to use github repo model for continuousdeploymentmarketing pagesproduct pagescontent updates & triggers into i18n, l10n, adaptationcomponents

give agile a brainuse lean ux as the brain foragiledevelop a lean cadenceinvolve all members in leanux (balanced teams)5

free to test frequently with users

sprint fasterfocus on getting to customer as early and as often aspossibleremoves the politics in the team as this becomes thearbiteryou can slow down this cadence after you convergeon key hypotheses and potential solutions

example: spotifysquads run like lean startups

spotify: squadsimilar to scrum team. feelslike startuplong term mission: build &improve the product. staylong term on the product.apply lean startup principleslike MVP“think it, build it, ship it,tweak it”emphasis on greatworkspace

spotify: tribescollection of squads thatwork in a related areaincubators for tribeshold regular gatherings

spotify: chaptersand guildschapters representhorizontal practices within atribeguilds represent horizontalpractices across tribes

becomehypothesis drivenlearn to state design goals interms of solving speciﬁccustomer problemsdon’t be afraid to invalidatehypotheseslook for the pivot6

embrace theproblem not thesolutionengineering: don’t start withtechnology, start withexperiencedesign: get your ideas outearlytogether: get in front ofcustomers so problem is thefocus, not our currentsolution7

co-locate if atall possiblehigh bandwidth “meatspace”facilitates sharedunderstanding and deepcollaborationalso facilitates shared timewith the customer8

suggestionsat a minimum teams should come together for theﬁrst few weeks to build shared understanding, deepcollaboration and getting feedback from customersfor distributed members use high bandwidthcommunication where possible (skype, tele-presence)high bandwidthcommunicationnecessary.

github counterpointelectronic: discussion, planning and operationsprocess should be in high ﬁdelity electronics.available: work should be visible and expose process.work should have a URL. single source truth.asynchronous: almost nothing should require directinterruption.lock-free: avoid synchronization points.cooperation without coordinationhttp://tomayko.com/writings/adopt-an-open-source-process-constraints

tools that can help

team working agreementdecide who is the decision makerdefine your cadencedefine how you will work togetherdefine your hypotheses

sharedunderstandingthe more understanding theless documentationbut this doesn’t mean NOdocumentationyou need whatever isneeded to gain a sharedunderstanding

deepcollaborationstrong belief that ideas comefrom many different voicestrust is essentialall efforts never stray far fromcollaborative efforts

continuouscustomerfeedbackthis is the lifeblood of theteamgets rid of politicsturns a team outside-in

getting to mvpmvp is a key tenant of leanstartupapplied to engineering weshould think: minimal viableeverything. mv*minimal viable processminimal viable team sizeminimal viable technologyminimal viable toolswhat are startups using?

getting to mvpmvp is a key tenant of leanstartupapplied to engineering weshould think: minimal viableeverything. mv*minimal viable processminimal viable team sizeminimal viable technologyminimal viable toolswhat are startups using?zuck’s rule: how will the startup down thestreet do this tomorrow? do it like that today.

picture creditshttp://www.flickr.com/photos/wuschl2202/531914709/sizes/o/in/photostream/http://www.flickr.com/photos/a_ninjamonkey/3565672226/sizes/z/in/photostream/http://www.flickr.com/photos/funky64/4367871917/sizes/z/in/photostream/http://www.flickr.com/photos/emdot/9938521/sizes/o/in/photostream/http://www.flickr.com/photos/gregory_bastien/2565132371/sizes/z/in/photostream/http://www.flickr.com/photos/trvr3307/3703648270/sizes/z/in/photostream/http://www.flickr.com/photos/legofenris/5426012042/sizes/l/in/photostream/http://www.flickr.com/photos/cleaneugene/6866436746/sizes/c/in/photostream/http://www.flickr.com/photos/66309414@N04/6172219058/sizes/l/in/photostream/http://www.flickr.com/photos/nicmcphee/2954167050/sizes/l/in/photostream/http://www.flickr.com/photos/pasukaru76/6151366656/sizes/l/in/photostream/http://www.flickr.com/photos/brianmitchell/2113553867/sizes/o/in/photostream/http://www.flickr.com/photos/ciscel/422253425/sizes/z/in/photostream/http://www.flickr.com/photos/zebble/6817861/sizes/l/in/photostream/http://www.flickr.com/photos/nicasaurusrex/3069602246/sizes/l/in/photostream/http://www.flickr.com/photos/nathangibbs/98592171/sizes/z/in/photostream/http://www.flickr.com/photos/neilsingapore/4047105116/sizes/l/http://www.flickr.com/photos/smb_flickr/439040132/http://www.flickr.com/photos/therevsteve/3104267109/sizes/o/http://www.flickr.com/photos/st3f4n/4193370268/sizes/l/http://www.flickr.com/photos/eole/380316678/sizes/z/http://www.flickr.com/photos/cobalt/3035453914/sizes/z/http://www.flickr.com/photos/mbiskoping/6075387388/http://www.flickr.com/photos/fragglerawker/2370316759/sizes/z/http://www.flickr.com/photos/soldiersmediacenter/4685688778/sizes/z/

picture credits (continued)http://www.flickr.com/photos/dahlstroms/4083220012/sizes/l/http://www.flickr.com/photos/don2/53874580/sizes/z/http://www.flickr.com/photos/hao_nguyen/3634552812/sizes/z/http://www.flickr.com/photos/42573918@N00/8194636033/http://www.flickr.com/photos/pagedooley/2420194539/sizes/z/http://www.flickr.com/photos/neilsingapore/4047105116/sizes/l/http://www.flickr.com/photos/smb_flickr/439040132/http://www.flickr.com/photos/therevsteve/3104267109/sizes/o/http://www.flickr.com/photos/st3f4n/4193370268/sizes/l/http://www.flickr.com/photos/eole/380316678/sizes/z/http://www.flickr.com/photos/cobalt/3035453914/sizes/z/http://www.flickr.com/photos/mbiskoping/6075387388/http://www.flickr.com/photos/fragglerawker/2370316759/sizes/z/http://www.flickr.com/photos/soldiersmediacenter/4685688778/sizes/z/http://www.flickr.com/photos/janed42/5033842895/sizes/z/http://www.flickr.com/photos/9619972@N08/1350940605/http://www.flickr.com/photos/alanenglish/483251259/sizes/z/thanks flickr!

Real World Lessons Using Lean UX (Workshop)

Bill Scott

Real World Lessons Using Lean UX (Workshop)