1. Knowledge management is important for analytic teams to avoid common pitfalls like work being hard to understand, impossible to verify, flawed, or inefficient. It helps by establishing standards, sharing lessons learned, and avoiding duplicating work. 2. At Bundle, knowledge management supports their workflow by standardizing definitions, algorithms, and processes through a wiki and persistent code. This helps onboarding and allows progress to build over time. 3. Effective knowledge management is essential for technical work since it draws on more dimensions of knowledge than can be managed informally. It must be customized to each team's workflow and thought processes.

1. Knowledge Management for
Analytic Professionals and Teams
Jaime Fitzgerald, President, Fitzgerald Analytics, Inc.
Alex Hasha, Chief Data Scientist, Bundle Corporation
March 2011
Architects of Fact-Based Decisions™

2. Introduction
Alex Hasha Jaime Fitzgerald,
Data Scientist @ Founder @
Bundle Corp Fitzgerald Analytics
Leading development of data products
Transforming data into value for clients
Responsible Designing statistical methods / algorithm
For… that transform data into insights for
Creating meaningful careers for employees
consumers
Helps consumers with financial mgt by Helps clients convert Data to Dollars™
At a
providing tools & spending behavior data Brings a strategic perspective to improve
Company
that are available nowhere else ROI on investments in technology, data,
That
Is growing and hiring! people, and processes
Learning about Hadoop, Hive, Etc.
Also
Writing a book about the role of “thought-
Working
Buying a home and renovating a bathroom style” in the information-era
On
Knowledge Management for Analytics 2

3. About Bundle Corporation and Bundle.Com
A joint venture of Citi, Microsoft Money, & Morningstar, Bundle is:
Bridging the gap between personal financial management and the good
living that healthy finances makes possible.
Helping people save well & spend well, combining free personal
financial mgt tools with data tools and recommendation engines built
from the anonymous spending behavior of over 20 million households.
Cross-referencing this data with other public and private information,
we developed tools available nowhere else:
"Everybody's Money”: learn how your peers spend and save
Restaurant Recommender: based on card spending data
Merchant Recommender: our newest recommendation tool
Knowledge Management for Analytics 3

4. The restaurant recommender: predictive analytics meets dining!
Knowledge Management for Analytics 4

5. Executive Summary
1. Complex analysis is high stakes, risky, and hard to
manage informally
2. Success requires knowledge management
standards and tools, even within small teams
3. “No Silver Bullet”: to empower analysts,
knowledge management methods must be tailored
to a team’s workflow
Knowledge Management for Analytics 5

6. Table of Contents
1. Challenges of Analytic Thought-Work
2. The Role of Knowledge Management
3. Implications for all Thought-Workers
Knowledge Management for Analytics 6

7. 1. Challenges
Challenges Analysts Face
In this section we will discuss…
1. A core set of universal challenges faced by analytic pros
2. Specific examples of these challenges at:
1. Bundle Corporation
2. Other analytic teams that are working hard as we speak
Knowledge Management for Analytics 7

8. 1. Challenges
It’s a Different World
While analytic teams are indispensible in today’s information economy, the nature of their work
makes teamwork, management, and coordination challenging.
source: www.xkcd.com
This creates a set of pitfalls…
Knowledge Management for Analytics 8

9. 1. Challenges
Problems Analysts Face
There are several pitfalls into which analysts fall.
Analysis is
The Embarrassing Facts
often:
1. Hard to
A top complaint regarding analysis is that it is confusing and unclear
understand
2. Impossible to
verify, audit, Few executives report they fully trust analysis they receive
or replicate
90%+ of spreadsheets used in the field are estimated to have
material errors
3. Flawed
In 201- Aetna cancelled a 19% rate increase due to flawed analysis
Average analysts spend less than 10% of their time actually
4. Inefficient performing core analysis (with most time going to data gathering,
troubleshooting, etc)
Knowledge Management for Analytics 9

10. 1. Challenges
Analyst Pitfalls: Real Life Examples
Based on personal experience, but modified to protect the guilty…
Analysis is often: Examples (not @ Bundle) Impact
An analysis involving both simple and
weighted averages of borrower credit
1. Hard to Misinterpretation
scores
understand Wrong inputs
In various parts of the analysis, both
metrics were called “credit score”
Unaware of risk
2. Impossible to Documentation of key input variables was
Hard to tell
verify, audit, or not consolidated, hard to find, and
whether correct
replicate therefore not widely known
input was used
A chain of analysis mixed up simple vs.
weighted averages. Consequences can
3. Flawed
Error persisted 5 years before it was be significant
discovered.
2 PhDs took a week to solve a problem Expensive waste of
4. Inefficient
that should never have happened. skilled time.
Knowledge Management for Analytics 10

11. 1. Challenges
Data Science at Bundle: Lots of Prototyping
It is crucial for us to
1. Goal: Solve a document, share, and re-use
New Problem lessons learned from each
cycle of effort
4. User Feedback 2. Design Solution
3. Testing +
Validation
Data Product
on Website
Knowledge Management for Analytics 11

12. Table of Contents
1. Challenges of Analytic Thought-Work
2. The Role of Knowledge Management
3. Implications for all Thought-Workers
Knowledge Management for Analytics 12

13. 2. The Role of Knowledge
Challenges Analysts Face Management
In this section we will discuss…
1. How KM helps analysts in general
2. Case Examples from Bundle
Knowledge Management for Analytics 13

14. 2. The Role of Knowledge
Knowledge Management to Avoid Analyst Pitfalls… Management
Knowledge Management makes a big difference in outcomes for analysts
Pitfall How KM Helps
Lack of detailed specifications often leads to Consistent coding and analysis standards
useless results, yet writing detailed prevent make specifications easier to create
specifications for another teammate can take and execute
longer than doing it yourself.
Learning curve remains, but progress is faster.
Slow on boarding due to “knowledge Senior teammates spend less time training new
diffusion” and lack of access teammates.
Reinventing the wheel due to lack of Teammates post documentation of solutions to
awareness of previous work-products common problems, and are encouraged to
search this documentation as a first step.
Misinterpreting data definitions, or
misunderstanding provenance of data leads to Enforcement of standardized, descriptive, field
incorrect analyses. names and centrally available data dictionaries
make these mistakes harder to miss.
Knowledge Management for Analytics 14

15. 2. The Role of Knowledge
Workflow at Bundle Management
Inputs Process Outputs
Credit Card
Transaction Data Insights &
Natural Language Customer/
Processing/ Merchant Recommendations for
Categorization Spending Survey Consumers
Merchant
Listing Data Everybody’s Money™
Restaurant / Merchant
Anonymous Customer Recommender
Demographic Data
• Loyalty/Popularity
Scores
Census Data & Statistical Consumer
• “Web of Offline
Government Consumer Sample Demographic
Merchants”
Spending Data Rescaling Profiles
Consumer Segment
Analysis
Geographic Data
Knowledge Management for Analytics 15

16. 2. The Role of Knowledge
KM at Bundle Management
Wiki-Based Metric Definition
Algorithms
Dialog
Knowledge Meta-data
Work Flow
In-System “Persistent” Code & Scripts
Knowledge (vs. ad hoc data processes)
Knowledge Management for Analytics 16

17. Table of Contents
1. Challenges of Analytic Thought-Work
2. The Role of Knowledge Management
3. Implications for all Thought-Workers
Knowledge Management for Analytics 17

18. 3. Implications
Key Implications for all Through-Workers
Concept Basis
Technical work draws upon more
1. The more technical the work, the
dimensions of knowledge than teams can
more you need KM!
manage informally
Since the goal of KM is to improve Quality
2. Good KM supports YOUR
and Efficiency of outcomes, it is essential
workflow and thought-work
to customize KM to worker processes
Because workflow for technical workers is
3. One size does not fit all so variable, they require flexible KM
solutions
In contrast to less technical knowledge,
4. At times technical KM must span
there are times when technical knowledge
multiple platforms
is better managed in multiple places
Knowledge Management for Analytics 18

19. 3. Implications
Our Thought-Work Creates Value
Like a good chef, we need to have the ingredients in place when we need them…
“The Chef’s Shelf”
Individual Work Collaboration Communication
Knowledge Management for Analytics 19

20. Let’s Stay in Touch
We look forward to learning from each other…contact us anytime.
Alex Hasha Jaime Fitzgerald
Twitter: www.twitter.com/alexhasha Twitter: www.twitter.com/jaimefitzgerald
LinkedIn: www.linkedin.com/pub/alexander- LinkedIn: www.linkedin.com/in/jaimefitzgerald
hasha/8/26a/30a
Email: alex@bundle.com Email: jfitzgerald@fitzgerald-analytics.com
Knowledge Management for Analytics 20