Presentation at the North Central BI Special Interest Group (BISIG) going over a case study of converting an Excel Spreadmart solution to a SSAS data mart solution

1. Spreadmart to Data Mart Conversion
Joe Beeck – GfK Custom Research
Dan English
Principal Consultant Principal Developer/Team Lead
dane@magenic.com Joe.beeck@gfk.com

2. Who are we? – Dan and Joe
Dan English Joe Beeck
http://denglishbi.spaces.live.com/
• •
Developing with Microsoft technologies for over Principal Developer/Team Lead at GfK Custom
10 years Research North America.
• •
Over 5 years experience with Data Warehousing Has been working with Microsoft technologies for
and Business Intelligence over 10 years
• •
Experienced in ETL and Analysis Services Current role primarily focuses on working with
development, requirements gathering and data business users to identify requirements and
modeling managing the project team
• •
Microsoft Certified IT Professional (MCITP) and Microsoft Certified Solution Developer (MCSD)
Microsoft Certified Technology Specialist (MCTS)

3. Who is Magenic?
 Founded in 1995, Magenic is a technical consulting firm
focused exclusively on Microsoft technologies and has
designed and delivered more than 500 Microsoft-based
applications
 Headquartered in Minneapolis, with offices in Chicago,
Boston, Atlanta and San Francisco
 2005 Microsoft Partner of the Year, Custom
Development Solutions – Technical Innovation
 2007 Microsoft Partner of the Year Finalist, Data
Management
 Microsoft Gold Certified Partner and National Systems
Integrator
 40 Enterprise Data Services (EDS) consultants

4. Who is GfK?
Founded in1934 and headquartered in Nuremberg, Germany
Size
• $1.43B + in annual revenue
• 9,300+ full-time employees (USA – 700+)
• 2nd largest custom research company in North America
• 2nd largest custom research company worldwide
Full Service
• Knowledge and resources to meet any client need
• Global databases and custom research expertise
• Qualitative and quantitative practices
Global Coverage
• 130 offices located in more than 70 countries

5. Today‟s Agenda
• Market Research Overview
• The Original Spreadmart Solution
• What is the BI Maturity Model?
• Spreadmarts vs. Data Marts
• Case Study and Demo
• Lessons Learned
• Questions?

6. Market Research Overview

7. Why Do Market Research?
To reduce the risk of decision making:
• What hidden opportunities exist in the current
market?
• To whom should we target our advertising?
• What product should we market next?
• Should we change the formula of an existing
product?

8. Case Study – Reversing Category Decline
Industry:
• Dairy Industry
Business Problem:
• How to stop and reverse declining dairy sales
Background:
• Dairy sales slipping
• Negative publicity about dietary fats from dairy
• Fewer servings per day recommended
• The client, Dairy Trade Association, needed to understand
consumer attitudes toward dairy products to direct strategy

9. Case Study – Reversing Category Decline
Approach
• Attitudinal segmentation
• Identify how narrowly or broadly people view dairy
• Understand/quantify the consumer perception that dairy
is unhealthy
• Measure consumers attitudes on:
• Dairy category overall
• Individual products
• Health and lifestyle issues
• Cross this attitudinal information with consumption
patterns, lifestyle habits, and demographics
• Combine and model results to create in-depth profiles of
the respondent

10. Case Study – Reversing Category Decline
Results
• Major recommendation: It‟s about
milk! Milk should be at the core of
the communication message.
• Results: Very successful campaign
to reverse the trend and make milk
cool again.
• Milk sales rose
• Public perception changed

11. Ways to Collect Data
Type Situation
• A moderate number of questions
Telephone • A lot of people
• No visual or sensory stimuli needed
• A few questions – simple
• A lot of people
Mail • Few security concerns
• Visual and/or sensory stimuli
• Fewer questions
• Simple to complex
Online • A lot of people for relatively little money
• Visual stimuli
• More questions
In-person • More complex
• Visual and/or sensory stimuli

12. Types of Questions
Closed End Open End
Provides choice for the respondent. Respondent answers in own words; no
responses for respondent to choose from.
Good for “What do you do, Example: What, if anything, do
where is it done, who uses it” you like about the product?
type questions Please clarify.
Should generally be used when
Example: Why do you say that
all (or most) of the possible
you [respondent‟s answer to
responses can be determined
question 3]?
beforehand.

13. Market Research Process
Define Survey and Measures
Conduct Survey
Collect Data
Process and Clean Data
Report Results

14. The Original Spreadmart Solution

15. Business Requirements
300 000 survey responses per year – 25 000 per month
12 report templates
1500 reports generated per month
Ability to generate historical reports
24-hour turnaround after receipt of data
Perfect data

16. Speadmart Solution
Run PERL
Run VBA
Pre- Use Adobe Manually
script to
Load and script to
“stitch”
aggregate Distiller to post files to
validate data generate
data and convert SharePoint
together
using 1,500
individual
split into 17 everything to according to
PostScript
tabulation
separate PDF a predefined
reports
software files (250 at
Excel tabs documents file structure
according to
a time)
the hierarchy
16 Hours 16 Hours 30 Hours 1 Hour 1 Hour 1 Hour

18. Spreadmart Issues
Data had become decentralized over the course of 3+ years
Excel became unusable due to increasing data volume and memory errors
Unable to run historical reports without returning to saved versions of Excel
documents
Prone to error because of so many manual updates, lack of versioning control,
and lack of integrity-checking software
Custom updates increased reliance on individual developers. Transfer of
knowledge became very difficult
System/process became so slow that even small issues would cause delays in
delivery to the client
Solution had become fragmented and new report requests were no longer cost
efficient
Errors and delays were beginning to put contract in jeopardy

19. There must be a better solution…

20. BI Maturity Model – where are you at?
STRUCTURE: Mgmt Reports Spreadsheets Data Marts Data Warehouses Enterprise DW BI Services
System Individual Department Division Enterprise Inter-Enterprise
SCOPE:
By Wayne Eckerson, Director of Research, TDWI

21. Spreadmart BI – Infant (2nd) Stage
Are the users What happens when
Did they extract all How long does it
extracting and the person responsible
of the necessary take to extract
reporting on the for the report goes on
data to allow the data and how
right data? vacation or is sick or
management to ask clean is it once it
leaves the company?
further questions? is extracted?
MS Access MS Excel MS PowerPoint Business Users
Do they have enough
What logic is
Source Data data collected to
being applied and
perform yearly
Is all of the data
is this common
comparisons or
available in the
logic within the
trends over time?
source system?
organization?

22. Data Mart BI – Child (3rd) Stage
OLAP Engine
Data Mart
Source Data Business Users

23. Spreadmart vs. Data Mart BI
Spreadmart Data Mart
• High end-user control
• Shared/consistent view of data
• Easy to create and use
• Centralized logic
• Can be pieced
Pros • Highly interactive (slice-and-
together
Pros dice)
• Highly customizable for
• Secured
the intended audience
• Very Flexible
• Low cost solution
• Extremely Fast response time
• Inconsistent view of the data
• •
No centralized logic Takes time to generate
Cons • •
Typically no security applied Less end-user control
Cons
• •
Silos of data throughout Costs more to develop
organization • Could potentially introduce
new tools (training)

24. Spreadmart to Data Mart Case Study
Spreadmart
• Excel file report system
• Lots of embedded business logic and conditional formatting
• Generated over 1500+ files (most contained multiple reports) with macro
• Process took approximately 30 hours to run
• Initial Excel file was created and tested over a 6 month time period
• If there were any data issues or report creation errors process had to be re-run
• Not easy to implement additional change requests
Data Mart
• Star schema database engine designed
• Analysis Service database created with centralized logic
• Reporting Service reports created and data driven subscription setup
• Generated same reports in approximately 30 minutes
• Entire database along with reports was created and tested in 2 month time frame
• Database and reporting structure extremely flexible to change requests

25. Data Mart Case Study

26. Reporting Services with SSAS data
SSAS Designer within SSRS
• Keep measures in the columns
• Flattened hierarchy information
• Very nice drag-n-drop feel and parameter setup
MDX Query within SSAS data source
• No drag-n-drop designer
• Custom MDX scripting capability
SSIS data source
• OLE DB Source or DataReader (ADO.Net)
• Ability to customize output
• Join multiple datasets
SQL Server Stored Procedure
• Similar capabilities like SSIS
• Custom formatting and data merging logic within stored procedures
• OPENQUERY commands with linked server (SSAS)

27. Data Mart Conversion Steps
1. Received the business requirements for the deliverables
2. Reviewed the reporting deliverables, data files, and calculations required for
the reports
3. Created the star schema database model
4. Created the ETL process to import the data file and load the star schema
5. Created the Analysis Service database
1. Setup the necessary dimensions, attributes, hierarchies
2. Produced the cube with necessary measures, measure groups, and
calculations
6. Setup the linked server within SQL Server to access the SSAS database
7. Created the stored procedures to be used by Reporting Services
8. Created the Reporting Service reports
9. QA reports and all data associated with them
10.Setup data driven subscription to generate all of the reports to be delivered
to the client

28. SSAS data to SSRS Demo
DEMO

29. Lessons Learned
The client needs to understand how their hierarchical data is applied
( re-casted each month or applied using type 2 dimension )
The benefits of the future BI solution need to be emphasized throughout the project
Automate, Automate, Automate
Stick to your process
Business users are „key‟ – keep them involved throughout the process and use them for Q&A and
validation
Data is never as clean as you would expect – „trust but verify‟
Nothing is ever as „easy‟ as you think – even rounding can cause issues
Document and comment on all processes with reasons why

30. Resources
Microsoft BI Site
http://www.microsoft.com/bi/
SharePoint BI Features Introduction
http://office.microsoft.com/en-us/sharepointserver/HA100872181033.aspx
PerformancePoint Home Site
http://www.microsoft.com/business/performancepoint/default.aspx
PerformancePoint Developer Portal
http://msdn.microsoft.com/en-us/office/bb660518.aspx
Channel9 MSDN BI Screencasts
http://channel9.msdn.com/Showforum.aspx?forumid=38&tagid=277
SQL Server 2008 Home Site
http://www.microsoft.com/sqlserver/2008/en/us/default.aspx
Microsoft Virtual Labs (TechNet and MSDN)
http://www.microsoft.com/events/vlabs/default.mspx
Magenic Blogs
http://blog.magenic.com/blogs

31. Source Information
BI Maturity Model – http://www.dmreview.com/issues/20041101/1012391-1.html or
http://www.tdwi.org/publications/display.aspx?ID=7199
Dan‟s Blog postings – Using Reporting Services (SSRS) with SSAS data and SSAS MDX
Round = Banker‟s Rounding
DateTool - http://www.sqlbi.eu/datetool.aspx and
http://sqlblog.com/blogs/marco_russo/archive/2007/09/02/datetool-dimension-an-alternative-
time-intelligence-implementation.aspx

32. Contact Information – Thank You!
Dan English - dane@magenic.com
Dan‟s BI Blog - http://denglishbi.spaces.live.com
Dan‟s Videos - http://www.youtube.com/user/denglishbi or
http://video.msn.com/video.aspx?mkt=en-us&user=-
3657354010876223112
Magenic - info@magenic.com
Joe Beeck - Joe.beeck@gfk.com