Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

DAS Slides: Data Quality Best Practices

635 visualizaciones

Publicado el

Tackling Data Quality problems requires more than a series of tactical, one-off improvement projects. By their nature, many Data Quality problems extend across and often beyond an organization. Addressing these issues requires a holistic architectural approach combining people, process, and technology. Join Nigel Turner and Donna Burbank as they provide practical ways to control Data Quality issues in your organization.

Publicado en: Datos y análisis
  • DOWNLOAD THI5 BOOKS INTO AVAILABLE FORMAT (Unlimited) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download Full EPUB Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... ACCESS WEBSITE for All Ebooks ......................................................................................................................... Download Full PDF EBOOK here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download EPUB Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download doc Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí

DAS Slides: Data Quality Best Practices

  1. 1. Copyright Global Data Strategy, Ltd. 2019 Data Quality Best Practices Donna Burbank & Nigel Turner Global Data Strategy, Ltd. August 22nd 2019 Twitter Event hashtag: #DAStrategies
  2. 2. Global Data Strategy, Ltd. 2019 Donna Burbank 2 Donna is a recognised industry expert in information management with over 20 years of experience in data strategy, information management, data modeling, metadata management, and enterprise architecture. Her background is multi-faceted across consulting, product development, product management, brand strategy, marketing, and business leadership. She is currently the Managing Director at Global Data Strategy, Ltd., an international information management consulting company that specializes in the alignment of business drivers with data-centric technology. In past roles, she has served in key brand strategy and product management roles at CA Technologies and Embarcadero Technologies for several of the leading data management products in the market. As an active contributor to the data management community, she is a long time DAMA International member, Past President and Advisor to the DAMA Rocky Mountain chapter, a contributor to the DMBOK 2.0, and was recently awarded the Excellence in Data Management Award from DAMA International in 2016. Donna is also an analyst at the Boulder BI Train Trust (BBBT) where she provides advice and gains insight on the latest BI and Analytics software in the market. She was on several review committees for the Object Management Group’s for key information management and process modeling notations. She has worked with dozens of Fortune 500 companies worldwide in the Americas, Europe, Asia, and Africa and speaks regularly at industry conferences. She has co- authored two books: Data Modeling for the Business and Data Modeling Made Simple with ERwin Data Modeler and is a regular contributor to industry publications. She can be reached at donna.burbank@globaldatastrategy.com Donna is based in Boulder, Colorado, USA. Follow on Twitter @donnaburbank Twitter Event hashtag: #DAStrategies
  3. 3. Global Data Strategy, Ltd. 2019 Nigel Turner Nigel Turner has worked in Information Management (IM) and related areas for over 20 years. This experience has embraced Data Governance, Information Strategy, Data Quality, Data Governance, Master Data Management, & Business Intelligence. He spent much of his career in British Telecommunications Group (BT) where he led a series of enterprise wide IM & data governance initiatives. After leaving BT in 2010 Nigel became VP of Information Management Strategy at Harte Hanks Trillium Software, a leading global provider of Data Quality & Data Governance tools and consultancy. Here he engaged with over 150 customer organizations from all parts of the globe. Currently Principal Consultant for EMEA at Global Data Strategy, Ltd, he has been a principal consultant at such firms as FromHereOn and IPL, where he has led Data Governance engagement with customers such as First Great Western. Nigel is a well known thought leader in Information Management and has presented at many international conferences. He also works part time at Cardiff University, where he is setting up a Student Software Enterprise company. In addition he has also been a part time Associate Lecturer at the UK Open University where he taught Systems & Management. Nigel is very active in professional Data Management organizations and is Vice- Chair Of the UK Data Management Association (DAMA). He was the joint winner of DAMA International’s 2015 Community Award for the work he initiated and led in setting up a mentoring scheme in the UK where experienced DAMA professionals coach and support newer data management professionals. Nigel is based in Cardiff, Wales, UK. Follow on Twitter @NigelTurner8 Today’s hashtag: # DAStrategies 3
  4. 4. Global Data Strategy, Ltd. 2019 DATAVERSITY Data Architecture Strategies • January 24 - on demand Emerging Trends in Data Architecture – What’s the Next Big Thing? • February 18 - on demand Building a Data Strategy - Practical Steps for Aligning with Business Goals • March 28 - on demand Data Modeling at the Environment Agency of England - Case Study • April 25 - on demand Data Governance - Combining Data Management with Organizational Change • May 23 - on demand Master Data Management - Aligning Data, Process, and Governance • June 27 - on demand Enterprise Architecture vs. Data Architecture • July 25 - on demand Metadata Management: Technical Architecture & Business Techniques • August 22 Data Quality Best Practices (w/ guest Nigel Turner) • Sept 26 Self Service BI & Analytics: Architecting for Collaboration • October 24 Data Modeling Best Practices: Business and Technical Approaches • December 3 Building a Future-State Data Architecture Plan: Where to Begin? 4 This Year’s Lineup
  5. 5. Global Data Strategy, Ltd. 2019 Today’s Topic Tackling Data Quality problems requires more than a series of tactical, one-off improvement projects. By their nature, many Data Quality problems extend across and often beyond an organization. Addressing these issues requires a holistic architectural approach combining people, process, and technology. 5
  6. 6. Global Data Strategy, Ltd. 2019 Data Quality is Part of a Larger Enterprise Landscape 6 Successful Data Quality Improvement Requires Many Inter-related Disciplines “Top-Down” alignment with business priorities “Bottom-Up” management & inventory of data sources Managing the people, process, policies & culture around data Coordinating & integrating disparate data sources Leveraging & managing data for strategic advantage
  7. 7. Global Data Strategy, Ltd. 2019 Agenda 7 What we’ll cover today What is data quality and why does it matter What can happen when data quality goes wrong The continuing impact of poor data quality Traditional approaches to tackling data quality Holistic approaches to tackling data quality Q & A
  8. 8. Global Data Strategy, Ltd. 2019 DataQuality–aSimpleDefinition Data that is demonstrably fit for purpose 8 Meets defined business needs for: • Accuracy • Completeness • Reliability • Accessibility • Timeliness 8
  9. 9. Global Data Strategy, Ltd. 2019 Poor Data Quality: Overall Impact on Companies & Organizations ECONOMIC: REVENUES, COSTS, PROFITS LAW & REGULATION BRAND, REPUTATION & CUSTOMER LOYALTY 9
  10. 10. Global Data Strategy, Ltd. 2019 Recent Data Quality Bloopers – Online Retail • Lens normally retails at circa $13,000 (around £10,700) • At start of Prime Day, price quoted by Amazon was $94.98 (£78) • Error spotted and price adjusted to $9,498 (£7,816) • Hundreds purchased at $94.98 (£78) • Amazon honored the deal at a potential loss of $7,738 (£6,370) per lens 10 Canon EF 800mm f/5.6L IS Telephoto Lens
  11. 11. Global Data Strategy, Ltd. 2019 Recent Data Quality Bloopers – Healthcare • Man entered hospital for a cystoscopy procedure • Unfortunately his name very similar to another patient awaiting surgery • When he came around, he found that he had been confused with the other patient • Given a circumcision instead of a cystoscopy • One of a string of similar errors at the hospital so major investigation undertaken 11
  12. 12. Global Data Strategy, Ltd. 2019 Data Quality – Real Stakeholder Feedback 2019 12 “We should be checking the data before we enter it on the system but we don’t have the time” “There is a lack of appreciation of what happens to data from the front end to the back end” “There is no accountability for bad quality data” “Our customers say ‘you’re embarrassing yourselves – the maths are wrong!’” “We’ve got lots of data, but it’s hard to connect it” “If we could just know that we can trust what we are looking at” “A lot of time (spent) fixing things & not enough time to be creative about the future” “We need to be Partner- ready, but every partner meeting discusses the challenges with data” “We cannot achieve the growth we need if we cannot sort out the data.” “We only know if we have a data problem if a customer contacts us…… [too much marketing activity is done] in blind faith… I press ‘send’ and hope for the best” “Our systems don’t talk to each other”
  13. 13. Global Data Strategy, Ltd. 2019 The Industry Impact of Poor Data Quality – The Evidence On average, half of all organisations believe at least 26% of their data is inaccurate (Source: BARC 2019) On average, poor data quality costs companies between 15-25% of revenue (Source: MIT Sloan 2017) The US economy loses $3.1 trillion a year because poor data quality (Source: IBM 2016) Poor quality customer data is costing UK companies an average of 6% of their annual revenues (Source: Royal Mail Data Services 2017) 13 13
  14. 14. Global Data Strategy, Ltd. 2019 Why Does Poor Data Quality Persist? • The data world has become more complex and diffuse • The world changes, and data models the world • Not recognising that poor data quality is a business problem, not an IT problem • People will make mistakes with data • Conflict or absence of common data definitions & metadata context • The data Newton’s Cradle • Lack of accountability for improving data 14
  15. 15. Global Data Strategy, Ltd. 2019 The Data World Becoming More Complex…. 15
  16. 16. Global Data Strategy, Ltd. 2019 B2C Data Volatility – UK facts 3.2 million people move house each year 821,600 babies are born each year 568,800 people die each year 126,400 get divorced 517,800 immigrants come from overseas 352,100 leave the UK 16
  17. 17. Global Data Strategy, Ltd. 2019 B2B Data Volatility – UK facts Average decay of UK B2B contact database is 2% per month There are 4.9 million businesses in the UK 489,636 new companies start up every year 30% of people change email addresses each year 17
  18. 18. Global Data Strategy, Ltd. 2019 It’s NOT an IT Problem…It’s a Business Problem • Human error • No data accountability • Poor training • Internal politics • Denial • Data capture & user design failures • Multiple data silos • Interface errors • Poor data architecture & design • Poor business process design • Process failures • Flawed goal setting • No agreed data standards 18
  19. 19. Global Data Strategy, Ltd. 2019 Lack of Governance & Accountability for Data 19 In many organizations, nobody is formally responsible for data and its improvement… … therefore bad data never gets systematically fixed “If we are all supposed to be responsible, no one is responsible and nothing changes” (Quote from senior GDS client – Professional Services Organisation 2019)
  20. 20. Global Data Strategy, Ltd. 2019 Turning the Data Quality tanker around 20
  21. 21. Global Data Strategy, Ltd. 2019 Turning the Data Quality Oil Tanker Around: Key ‘Must Dos’ 21 • Implement a business led, structured Data Governance framework & organization to ensure clear priorities for data quality improvement • Develop reusable data improvement approaches that can be applied across the organization and able to tackle traditional & new data challenges • Automate data quality improvement – invest in a data quality toolkit to support reusable approaches • Link data quality to Data Architecture – design improvements into data design & implementation • Build a business case for every data quality improvement initiative so that the focus is on highest value work • Manage each initiative via a Data Improvement Plan
  22. 22. Global Data Strategy, Ltd. 2019 Applying a Structured Data Governance Framework 22 22 Organization & People Process & Workflows Data Management & Measures Culture & Communication Vision & Strategy Tools & Technology Business Goals & Objectives Data Issues & Challenges
  23. 23. Global Data Strategy, Ltd. 2019 Evolution of Data Quality Since 2000… TIMELINE BATCH REAL TIME / ONLINE REACTIVE PROACTIVE IT DRIVEN BUSINESS DRIVEN PLATFORM SPECIFIC ENTERPRISE WIDE DATA CLEANSE DATA RE-ENGINEERING MANUAL AUTOMATED OPERATIONAL FOCUS REPORTING / ANALYTICS FOCUS IN-PLATFORM DATA QUALITY DATA QUALITY AS A SERVICE SILOED FOCUS PART OF HOLISTIC DM CHANGE 23
  24. 24. Global Data Strategy, Ltd. 2019 Traditional Approaches to Data Quality • Inspect data sources and highlight data deficiencies, omissions and duplication • Develop data standards and common data definitions • Build business rules to enforce and police data standards • Automate data cleanse and enhancement projects, using the business rules defined • Embed the standards & rules into both batch and real-time environments to keep the data clean • Produce Data Quality KPIs and measures to monitor ongoing quality and track trends 24 Will continue to have great value
  25. 25. Global Data Strategy, Ltd. 2019 The New Age of Data Quality • Increased focus on real time data validation and improvement at the point of data creation, ingestion or use • Automated digitized processes and self-service will fail if the data is not fit for purpose • Data validation and improvement must be done in real time on large data volumes & varieties • ‘After the fact’ data cleanse and improvement is now too late • Opportunity to exploit IoT and AI to develop self-checking data quality capabilities • Business users need more control over the creation & management of business rules • They need the ability to create business rules dynamically when preparing data • Different data users will potentially require the application of varying business rules • The paradigm where business rules are created and held centrally by IT is obsolete • End user self-service data quality functionality is essential • Data preparation & formatting • Data parsing & cleansing • Data enhancement & enrichment • Toolsets must support a wider variety of platforms & data types • Legacy and Big Data environments (e.g. Data Warehouses and Data Lakes) • Real-time and batch • Structured / semi-structured / unstructured data types – via Data Profiling, Data Preparation & Metadata tagging 25 New approaches for data quality in digital organisations
  26. 26. Global Data Strategy, Ltd. 2019 Finding the Right Balance 26 Human Automated Resolve at Source Resolve via Post-Processing • Business Process Change • Policies & Procedures • Governance Steering Committees • End User Training • Industry Advisory Councils • Data definition & glossary • Data Quality Working Groups • Application-Driven Data Entry & Workflow • Application-level data validation • Database-level data validation & integrity (data models) • Data Quality tool validation at source • Data Cleansing Tools and/or SQL • ETL (Data Warehouse) • Data Stewardship • “Conscious Disregard” Proactive Business Management Reactive Business Management Proactive Technical Management Reactive Technical Management • Data Audit & Dashboards • External Data Sources
  27. 27. Global Data Strategy, Ltd. 2019 Repeatable Approaches: need for an integrated toolset Problem Ticketing Workflow Data Glossaries / Metadata Repositories Data Modelling Data Analysis / Profiling Data Preparation Data Quality Re-engineering Data Analytics & Data Mining Data Reporting & Dashboarding 27
  28. 28. Global Data Strategy, Ltd. 2019 Data Architecture Data Quality Data Governance Data Quality, Data Governance & Data Architecture – the Virtuous Circle Provides the structure to deliver Drives the need for Scopes & helps prioritize DATA ARCHITECTURE DATA GOVERNANCE DATA QUALITY Model entity & attribute relationships Overarching strategic framework Data profiling baselines current state Scope & prioritize key data Help identify data stakeholders Raise awareness of data quality issues in source data Develop business led business rules Assign owners & stewards to lead data quality efforts Data cleanse, enrichment & maintenance Models provide communication tools Ensure data quality alignment with changing business needs Automate business rule enforcement through DQ tools First step in defining data standards & KPIs Creates vehicle for cross- organization collaboration Metrics to provide factual foundation for action Links business rules > definitions > physical data designs Leads business case development & realization Helps build the business case for action 28
  29. 29. Global Data Strategy, Ltd. 2019 What is a Data Improvement Plan? A Data Improvement Plan (DIP) is a formal plan to specify and manage improvements to a specified data domain and / or data problem 29 The benefits of a Data Improvement Plan are: • sets out goals and expectations for data improvement • acts as a focal point for all data improvement activities • prioritises improvement activities • can be used to track improvements and communicate successes • can evolve to align with the changing needs of the business • data domain DIPs can be rolled up to form the core of a company wide Data Improvement Programme
  30. 30. Global Data Strategy, Ltd. 2019 Creating a Data Improvement Plan 30 INVESTIGATE ORGANISE PRIORITISE IMPROVE STEP 1 STEP 2 STEP 3 STEP 4 INDICATES ITERATION • Define data domain and its uses (Processes and Functions) • Identify data stakeholders (Creators, Modifiers, Consumers) • Engage with stakeholders (e.g. interviews, workshops, documents) • Identify key data fields (i.e. what data really matters) • Baseline data (Systems & Quality) & Data Governance maturity • Identify data problems & impact • Set up Data Domain Working Group (Business, IT and key stakeholders) • Create a log of data problems, opportunities and business impact • Initially identify and define potential improvement initiatives (People / Process / IT) • Prioritise data problems • Define improvement projects • Create improvement team(s) (from Working Group & others) • Produce Motivation Model & business case(s) for action • Finalise initial Data Improvement Plan for Steering Group endorsement • Launch improvement initiatives • Set KPIs and success measures • Perform root cause analysis and propose & evaluate changes • Design and implement improvements (People / Process / IT) • Produce improvement plans, monitor progress and measure data improvements • Log benefits, publicise successes, and identify lessons learnt
  31. 31. Global Data Strategy, Ltd. 2019 DATA DOMAIN BENEFIT TYPE DESCRIPTION EXPECTED REVENUE INCREASE / COST SAVING Year 1 Year 2 Year 3 CUSTOMER COST REDUCTION BONUS ABUSE REDUCTION 125,000 £125,000 £125,000 COST REDUCTION EMAIL M/K COST REDN £10,000 £10,000 £10,000 COST REDUCTION REDUCTION IN 3RD PARTY ADDRESS CLEANSE £50,000 £50,000 £50,000 SALES RISK AVOIDANCE AUTOMATED REGULATORY REPORTS £20,000 £20,000 £20,000 REVENUE INCREASE CROSS-SELLING OPPS 50,000 50,000 50,000 TOTAL £255,000 £255,000 £255,000 Business Case: Example of Benefit Analysis 31 31
  32. 32. Global Data Strategy, Ltd. 2019 Over $800M from Data Quality Improvement 32 • BT Group plc - a British multinational telecommunications holding company which had the following challenges: • Business Agility to deliver new products and services • Regulatory Compliance to meet industry demands in a heavily-regulated industry • Customer Satisfaction through positive customer interaction. • Unfortunately, these efforts were being hampered by poor data quality: • Poor Supplier & Customer data hampered self-service interactions • Inaccurate inventory led to increased capital expenditure • Billing errors caused customer dissatisfaction • To improve Data Quality, BT embarked on an enterprise-wide data improvement journey which included both technology and culture change. The Challenge • As a result, BT achieved in aggregate more than $800 million in quantified benefits including: • Capital Cost Avoidance: Through improving accuracy of inventory data, BT optimized equipment inventory and reduced inventory costs. • Improved Revenue Assurance: By improving Billing Data accuracy, revenue loss decreased from more than 15% to less than 1%. • Productivity Gains in B2B Processes: By resolving data quality issues, BT was able to automate customer and supplier interactions, and reduce cycle times and manual effort. The Result A Focus on Data Quality Yields Big Benefits for BT 1 1 Gartner, Publication Number G00138085, 24 March 2006
  33. 33. Global Data Strategy, Ltd. 2019 Summary • Addressing data quality issues requires a holistic architectural approach combining people, process, and technology. • Data quality is complex because businesses and organizations are complex • Data governance helps orchestrate the people, processes and organizational structure required to improve data quality • Data architecture provides business and technical alignment to implement data quality business rules into core systems • Build quantifiable data improvement plans to show demonstrable ROI 33
  34. 34. Global Data Strategy, Ltd. 2019 DATAVERSITY Data Architecture Strategies • January 24 Emerging Trends in Data Architecture – What’s the Next Big Thing? • February 18 Building a Data Strategy - Practical Steps for Aligning with Business Goals • March 28 Data Modeling at the Environment Agency of England - Case Study (w/ guest Becky Russell from the EA) • April 25 Data Governance - Combining Data Management with Organizational Change (w/ guest Nigel Turner) • May 23 Master Data Management - Aligning Data, Process, and Governance • June 27 Enterprise Architecture vs. Data Architecture • July 25 Metadata Management: Technical Architecture & Business Techniques • August 22 Data Quality Best Practices (w/ guest Nigel Turner) • Sept 26 Self Service BI & Analytics: Architecting for Collaboration • October 24 Data Modeling Best Practices: Business and Technical Approaches • December 3 Building a Future-State Data Architecture Plan: Where to Begin? 34 Join Us Next Month
  35. 35. Global Data Strategy, Ltd. 2019 About Global Data Strategy, Ltd • Global Data Strategy is an international information management consulting company that specializes in the alignment of business drivers with data-centric technology. • Our passion is data, and helping organizations enrich their business opportunities through data and information. • Our core values center around providing solutions that are: • Business-Driven: We put the needs of your business first, before we look at any technology solution. • Clear & Relevant: We provide clear explanations using real-world examples. • Customized & Right-Sized: Our implementations are based on the unique needs of your organization’s size, corporate culture, and geography. • High Quality & Technically Precise: We pride ourselves in excellence of execution, with years of technical expertise in the industry. 35 Data-Driven Business Transformation Business Strategy Aligned With Data Strategy Visit www.globaldatastrategy.com for more information
  36. 36. Global Data Strategy, Ltd. 2019 Questions? 36 • Thoughts? Ideas?

×