Select material used during product development orientation.

1. Lean/Agile Product Development
Methodology, Practices, Terminology
1

2. Session Goals
• Raise awareness within the team of Lean-Agile Product Development
methodology fundamentals, practices, and terminology.
• Provide a common starting point for discussion of how we will work day to
day: Processes, Practices, and Tools.
• Encourage further learning by individuals and their teams.
2

3. Methodology Fundamentals
3

4. What is Agile Development?
Agile development is a group of software development methodologies,
including Lean-Agile, based on adaptive planning, and iterative and incremental
development, where requirements and solutions evolve concurrently through
collaboration between self-organizing, cross-functional teams.
Core Agile Values:
• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan
While there is value in the items on the right, we value the items on the
left more.
4

5. Introducing Lean-Agile Product Development
A disciplined end-to-end Product Development methodology, with Agile values
and practices infused with “Lean Thinking”, that helps an organization achieve
rapid and reliable delivery of value, with an emphasis on continuous learning
and improvement.
Principles:
 Eliminate Waste: spend time only on what adds real customer value;
remove demands on capacity that do not add value.
 Amplify Learning: build knowledge; when you have tough problems or
uncertainty, increase feedback to improve. “Fail fast, fail cheap”.
 Defer Commitment: keep your options open as long as practical, but no
longer.
 Deliver as Fast as Possible: deliver value to customers as soon as they ask
for it; automate or remove impediments to rapid delivery.
5

6. Lean-Agile Principles (cont.)
 Trust and Empower the Team: let the people who add value use their full
potential to meet the business challenges.
 Build Quality In: don’t try to tack on quality after the fact; More testing, less
debugging.
 Systems-Thinking to “See the Whole”: resist the temptation to optimize
parts at the expense of the whole. Look at entire system’s value flow.
 Technical Excellence: an adaptive low-dependency architecture and quality
code through test driven development are mandatory to sustain rapid and
reliable delivery.
6

7. Lean-Agile is Adaptive
Traditional Approach is Prescriptive: a plan is a
commitment
• Beak project into identifiable work products
that are defined and built to meet specific
fixed goals.
• Carefully plan and then controlled to meet
that plan.
Lean-Agile is Adaptive: planning is indispensable
but plans are useless
• Look at the whole system’s ability to deliver
value rather than focusing on optimizing and
delivering individual parts.
• Value learning effectively through feedback
and testing and empower the people who do
the work.
7

8. Lean-Agile is Concurrent
Traditional Approach: deliverables are "tossed over the wall" from department to
department which results in lost information, defects, delays, and lower value
outcomes.
Lean-Agile Approach: tasks are integrated to reduce the elapsed time required to
bring a new product to the market by minimizes the inefficiency and defects that
arise from hand-offs.
8

9. Lean-Agile is Hard to Do
• There is no easy way, no "free lunch” – you must do the hard parts.
• Need to have strong engineering practices, high-bandwidth communication,
concurrent product development, continuous process improvement,
motivated cross-functional teams, and engaged Product Owner.
• Backsliding into old ways is easy: success requires sustained commitment
from all levels of the organization.
• Worth the effort - build great products and services rapidly and reliably that
delight customers and compete to win in the marketplace.
9

10. Practices and Terminology
10

11. How does a Product Request get Prioritized?
Portfolio Management
• Priorities are set at the portfolio level and
informs the individual team priorities
• Central oversight of budget, risk management,
strategic alignment of investments, demand and • Portfolio
investment management along with Portfolio
standardization of investment procedure, rules Priorities
Priorities
and plans. • Portfolio
success
Product Backlog metrics
• A prioritized list of things needed to be done:
Themes, Epics, Products, Features, and Stories.
Team Product • Team priorities
• Final requirement details are figured out during
implementation. • Team success
Backlogs
• New items can be added to the Product Backlog metrics
at any time (unlike Scrum where items can only
be added during Sprint planning).
• “Backlog Grooming” is performed by the team
on a regular basis.
• The GM and Product Manager regularly make
priority and sequencing decisions based on
expected value of items in the Backlog

12. Three Variables Product Considers When
Weighing Priorities
• Business Score
– What is the business value of the initiative (ROI, PV’s, etc.)?
• Project brief
• Level of Effort
– How many resources will it take to realize the business value?
• Step one: “t-shirt” sizing (S,M,L)
• Step two: Grooming meetings
• Team Capacity
– What else are people working on and what would need to be
de-prioritized to work on this initiative
• Working on standard measure to communicate this at the Portfolio
and Team levels

13. Themes, Epics and Stories
• Theme or “tent-pole”: a top-level objective
or vision.
• Epic: a group of related Stories that
describes a particular higher level capability
or functionality.
• Story*: an Independent, Negotiable,
Valuable, Estimatable, Small, Testable
candidate requirement (“INVEST”).
• Task: Stories will often be broken down into
specific work sub-tasks.
The Product Manager is responsible for making sure that work is broken down
appropriately at each stage. For example, Themes and even Epics are okay during
Portfolio reviews but all Epics should be broken down to the Story level once
Grooming meetings are complete.
13

14. MVP: Minimum Viable Product (or Feature)
• Has just those user stories that allow the
product/feature to be deployed that still achieves the
desired business goal, and no more.
• Avoid building products that customers do not want,
maximize the information learned about user
preferences per dollar spent.
• The process is iterated until a desirable product-
market fit is obtained, or until the product is deemed
to be non-viable.
“Over 70 years, this (Silicon) Valley has developed a culture that does not personalize failure,” said Mr.
Komisar, of Kleiner Perkins. “If you’re not corrupt, stupid or lazy, we see failure as learning — learn from it,
and reapply it.” NY Times
14

15. Kanban
• Kanban is a method for developing and releasing products with
an emphasis on moving small batches of work through the
product development system with a more continuous, even
flow.
• Analysts, developers & testers pull from a queue of tasks that
are ready for work.
• People only work on one or two tasks at a time (limit WIP).
• The work in progress is displayed for participants to see on a
physical or virtual wall-board – making it easy to see status and
for bottlenecks to be identified and fixed.
15

16. Kanban Wallboard Example
16

17. Types of Testing Used in Agile Processes
• Test Driven Development (TDD): Developers create automated or manual unit
tests that define code specification (immediately) before writing the code itself.
• Integration Testing: A phase in software testing in which individual software
modules are combined and tested as a group. It occurs after unit testing and
before acceptance testing.
• Acceptance Testing: Requirements written as tests by Product Manager before
development starts, used by developers in unit tests, and by QA for testing Stories
in QA environments. Acceptance tests are how you know when you are “done”.
• User Acceptance Testing (UAT): Performed by actual product manager or users
during QA, or earlier to get feedback as early as possible.
• Regression Testing (end to end): Continuous automated regression tests give
rapid feedback during development, also performed by QA before release.
17

18. Agile Meetings: Feedback & Learning
• Daily Stand-up
– Mandatory for all team members
– Keep to 15 min or less
– What you did yesterday, doing today, and blocking issues
– Take technical discussions offline
• Demo
– Alpha/beta/feedback
• Retrospective
– What worked well last iteration that we should continue
doing?
– What didn’t work well last iteration that we should stop
doing?
– What should we start doing?
18

19. Deployment On Demand
• “Deployment on Demand” is the ability to choose when
a particular product, feature, or bug fix should be
deployed into an integration environment, into a QA
environment, or into production.
• As each Story is completed and release ready, the goal is
to move that Story to production in order to begin
gaining the value (and learning) expected from it.
• If a Story is part of an MVP/MVF, then the completed
Story may be held until the other Stories are completed.
19

20. Documentation in the Agile World
• Where should Product documentation exist?
– Discovery Wiki: product docs, architecture docs, etc
– JIRA ticket: story specific acceptance criteria, notes,
etc.
• When to document?
– Project stakeholders require it
– To define a visual model (design comps)
– To define a contract model (external interfaces)
– To support communication with an external group
– To support organizational memory
– To think something through
20

21. Process Improvement Basics
• Start with what you do now.
• Initially, respect current roles, job titles and
responsibilities.
• Make Process Explicit.
• Improve Collaboratively.
21

22. Agile Estimation – “t-shirt sizing”
• In Lean-Agile process, we assign work to the team, not the
individual
• In the Grooming Meetings, the team sits down to estimate
its effort for the Epics and Stories in the backlog
• T-shirt sizes (XS, S, M, L, XL, XXL) are common estimating
methods for story items
• The Product Manager needs these estimates, so that he
can effectively prioritize items in the backlog and, as a
result, forecast releases based on the team's throughput
22

23. New Process
Vertical/Central Services teams
Monitoring/
Analysis
Vertical
MVP Acceptance
Request Development Production
Vertical Process
Review
Team
Request
Central Vertical
support support
Central
Service
Portfolio MVP Central Acceptance Production
Review Process Dev
Monitoring/
Analysis

24. Intake/Review
• Route new requests to Product Managers*
• Project Brief
• Scoring
– Business value
– Level of effort (t-shirt size: S, M, L, XL)
– Capacity
• Green lighting
• Backlog prioritization
*Maintenance/bugs – continue to do what you are doing.

25. Minimum Viable Product
• Epics/User Stories
• Team review
• Initial estimates (story points)
• Team negotiation
• MVP defined
• User Stories refined w/ acceptance tests

26. Request Checklist
Project Manager creates a Checklist for each request: Theme, Epic,
and large User Story. Input is solicited from team members.
 Does this require a Technical Approach Document (TAP) to capture high-level
architectural approach.
 If a VAP project, does this need Central architectural review or input?
 How much requirements documentation is needed? This is usually based on
risk and stakeholder needs.
 User/Tech Stories with Acceptance Test Criteria (required)
 Wireframes?
 Design Comps?
 Full functional specification?
 API or interface specification?
 Do other stakeholders need to be included in the requirement analysis,
demos, reviews, etc.
 AdOps
 Ad Sales
 Analytics
 Publishing
26

27. Development/Acceptance - very simplified…
• Developers pick up User Stories that are “ready for development”
(i.e. refined with acceptance criteria and estimate) as they have
capacity.
• Developer codes & unit tests components, then integrates and tests
User Story end-to-end.
• Developer frequently demos results to Product Manager before and
after integration testing to get feedback.
• QA tester picks up and verifies “integrated” stories against
Acceptance Tests, then UAT by any external stakeholder (e.g. AdOps)
is performed.
• When all User Stories for MVP have been passed Acceptance Testing,
and regression testing has been completed, the software is “ready for
deployment”.
• Software is deployed and verified in production by QA.

28. Standard Daily Meetings
• Daily Stand-up (All Teams: VAT, Central, Mobile, etc)
– Daily check-in with team members
– Mandatory for core team members: Product, Dev, QA, PM, Design/UX
– Each team to pick a non-overlapping 15 minute slot between 9:30 and 11:30. PMs
to coordinate
• Daily Change Control
– Every morning at 9:15-9:30
– PMs, Product Managers, Central tech reps.
– Determine what do with new/timely issues – particularly from VATs. Cancel if no
items for change control.
28

29. Other Weekly or Ad-Hoc Meetings
• Grooming Meetings (Each Team)
– Product Manager/Owner collaboratively reviews Backlog, discuss and refine Stories,
etc. with the team. Usually done in the context of a particular MVP/MVF
– Story point estimates provided by team
– Most, if not all, team members should participate as this is a critical information
sharing session
• Prioritization Meetings (Each Team)
– Product Managers and Product Owners will meet on an as-needed basis to review
priorities.
– Program Manager will coordinate any portfolio prioritization meetings on an as-
needed basis.
• Retrospectives “Lessons learned” (Each Team)
– Organized by PM after an iteration or every few weeks
29

30. Theme, Epic, User Story Examples
30

31. Recap: Themes, Epics and Stories
• Theme or “tent-pole”: a top-level objective or vision.
• Epic: a group of related Stories that describes a particular higher level capability or
functionality, or more generally an epic is a high level story used as a placeholder.
• Story: an Independent, Negotiable, Valuable, Estimatable, Small, Testable candidate
requirement (“INVEST”). User stories are written from a user point of view.
• Acceptance Test: Specific user scenarios, provided by Product Manager, that are used to
determine when a user story has been correctly implemented.
• Sub-Task: Stories will often be broken down into specific work sub-tasks.
• MVP/MVF: Has just those user stories that allow the product/feature to achieve the
desired business goal, and no more. Iterate and learn until a desirable product-market
fit is obtained, or until the product is deemed to be non-viable.
Note: this “user story” based process is very deliberately intended to minimize the use of
big requirements documents and focus more on verbal conversation to achieve mutual
understanding between customer/user representative and the developer. 31

32. Example: Website Redesign
Theme: Website Redesign 2011
• Relaunch XYZ website as a destination for men between age of 18-45, double
audience in 18 months.
Sample Epics:
• New Blog oriented verticals for Adventure, Gear & Gadgets, Cars & Bikes
• Producers can publish and feature News blog content
• Visitors can easily navigate to featured videos, topics, sub-topics, ad sales
packages, and Commerce promotions within a streamlined top navigation
• Producers can feature new content within an updated Homepage design
Sample “Homepage” Stories:
• As a Producer I want to select the featured content so visitors can easily find
curated content.
• As a Visitor I want to view the Facebook like count and Twitter follower number
so I can easy like or follow if I want to.
• As AdOps user I want to be able serve Pushdown, Tile, and Entitlement ads on
the homepage so we can monetize the content.
• As a Visitor I want to see the most recent articles 32

33. Example: MVP
Homepage MVP
1. As a Producer I want to select the featured content so visitors can easily find
curated content
2. As a Visitor I want to view the Facebook like count and Twitter follower
number so I can easy like or follow if I want to.
3. As AdOps user I want to be able serve Pushdown, Tile, and Entitlement ads on
the homepage so we can monetize the content.
4. Etc…
Deferred – non-MVP
1. As a Visitor I want to see the most recent articles
2. As a Commerce Producer I want to promote a list featured products in the
Right Rail in order for Visitors to see different kinds of products.
3. Etc…
33

34. Example: Acceptance Test Criteria
User Story: As a Producer I want to select the featured content so visitors can easily find
curated content.
Acceptance Test Criteria
• Featured content consists of a minimum of 6 items and a maximum of 10 items.
• Content in the featured stream can come from all articles and blog posts.
• Producers control what appears in the featured stream and the position in which it
appears.
• Featured Content Displays [title], [image], [topic], [description], [read more link].
• [title], [image] and [read more link] link to the individual asset.
• [topic] rules are as follows:
1. Display [topic] which the asset belongs to
2. IF [topic] is "Show News", THEN display show tag instead. (e.g. Mythbusters or Shark
Week)
3. IF [topic] is not "Show News" and the asset has a Supertag, THEN display the
Supertag
instead.
4. [topic] links to topic page, showsite or tag listing page accordingly.
34

35. JIRA Request Tracking
35

36. Backlog vs. Work In Progress
• Backlog
– ___BCK-###
– Backlog is upcoming work, may be some early discussions
– Has not been fully prioritized/sequenced
– The items to be worked on next should be more refined
• Work In Progress
– ___WIP-###
– Prioritized
– Meets “Ready For Development” criteria
– In the hands of engineering
• All teams have both types of queues

37. JIRA Queues or “Projects”
• Lessons learned
– Jira only allows a certain level of granularity
because of hierarchy restrictions in the software
– We need the granularity to provide vertical
specific reporting
– We need the granularity to view backlog (new
stuff) separately from work in progress
• Separate queues provides better granularity

38. Simplified Ticket Types
• Backlog Ticket Types • Subtask Ticket Types
– Theme/Epic – Design / Development
– User Story – Documentation
– Tech Story – Deployment / SysAdmin
– Production Defect – QA Defect
– Research/Prototype • Support Ticket Types
• WIP Ticket Types – Production Support
– User Story – Publishing Support
– Tech Story
– Production Support
– Production Defect
– Research/Prototype

39. Customized Workflows Based on Ticket Type
• New Jira feature for the team
• Tickets follow a particular path and change
status by clicking a “workflow button”
• Software Limited choices
– Simplifies moving ticket from one status to
another
• Business Criteria / Rules to change status
– Clarifies what a particular status really means

41. Lean/Agile Principles – More Info
41

42. Eliminate Waste
Look for policies, processes, or practices that take time, energy, and resources
that do not add value and streamline, automate, or stop doing them.
• Work in Progress
• Extra Processes
• Extra Documentation
• Extra Features
• Task Switching
• Design Loop-backs
• Significant support burden
• Waiting
• Hand-offs
• Defects
• Management Activities
42

43. Technical Excellence
Ability to respond to change and rapid delivery requires strong engineering
practices
• More testing, less debugging
• Low-dependency architectures
• Continuous integration
• Evolutionary design and development
• Refactoring
• Good coding practices
• Code reviews
See wiki for more details:
https://wiki.discovery.com:8443/display/DMS/Technical+Discipline+and+Tools
43

44. Amplify Learning
• “Fail Fast, Fail Cheap”
• Use set based decision making to identify and evaluate several options.
• Share requirements, designs, and code (demos) early and often.
• Use feedback loops: demos, reviews, postmortems, metrics, prototypes, etc.
• “Information Radiators” that are quick ways to get info to the team.
• Relentless improvement: seeing and solving problems, sharing the
knowledge.
44

45. Defer Commitment
• Leave options open and accommodate evolving requirements.
• Let the solution emerge: do not wait to get all of the details worked out
before starting design and development.
• Define scope at a high level, the details are negotiable.
• Do not waste time developing detailed requirements for work to be done in
the future.
• Multiple options of good+safe, better+risky, optimal+high-risk approach.
• Delaying decisions until they need to be made reduces rework.
45

46. Deliver as Fast as Possible
• Deliver value early rather than wait longer for a “complete” product.
• Short iterations with continuous delivery encourage a “Try-it, test-it, fix-it”
approach instead of “Do it right the first time”.
• Use the “pull-system” principle of Kanban and "Just in Time“ – no large
batches that cause bottlenecks and delays.
• Releases to Production in small increments when ready.
• The goal is reliable and repeatable results, not a repeatable process.
46

47. Build Trust and Empower the Team
• Excellent, focused, disciplined, and empowered people are key to success.
• Trust and respect between team members and between managers and
employees is essential.
• Find and grow professionals with purpose, passion, persistence, and pride.
• Keep teams as small and focused as possible.
• Build cross-functional teams that mixes domain expertise with technical
expertise.
• Ensure there is a clear and compelling purpose for the work.
• Give the technical team access to the customers, users, and decision
makers.
• Let the team make its own commitments and be accountable for outcomes.
• Analysts are not be a substitute for developer-customer communication but
should facilitate understanding on both sides.
47

48. Build Quality In
• Quality must be built-in, not tested for after the fact.
• Technical Excellence and solid engineering practices are mandatory for Agile
development to be successful.
• Quality means realization of purpose or fitness for use rather than
conformance to requirements or a set plan.
• There are two kinds of Quality:
 Perceived or External Quality-- the beauty, elegance, and delight of the
product to the end user.
 Conceptual or Internal Quality-- the integrity of the product’s
architecture as an integrated whole as it balances between flexibility,
maintainability, efficiency, and responsiveness.
• Model driven design – based on end user view of the system. This will help
ensure that we "build the right thing".
• Everyone is responsible for quality. The developer in particular has a very
active role in ensuring quality - this must not be thought of as "QA's job".
48

49. Systems-Thinking: “See the Whole”
• Look at customer outcomes (increased audience and engagement for
example) rather than internal measures such as productivity, features
developed, or sticking to a plan.
• Value output: quality of output is more important than quantity of output
when measuring productivity of knowledge workers. Delivering fast without
quality is wasteful.
• Look for ways to bring predictability to value demand, by eliminating
“failure demand” and filtering value demand in a more disciplined way to
match up with capacity.
• Instead of optimizing parts the system or organization to increase growth
such as productivity, or speed, look for and remove what is a constraint to
growth, productivity, or speed.
• Beware of optimizing a part, such as ensuring high utilization, at the
expense of the whole. Trying to maximize utilization invariably leads to
delays and other adverse results.
• Look for and fix root causes, rather than symptoms.
49