Every day, consumers, businesses and not for profit organisations generate increasing volumes of data. Initiatives such as Smart Meters in the utilities sector, along with user generated 'Web 2.0' data sources and High Energy Physics are causing an exponential growth in available data. Many business seek to take advantage of this data to analyse business performance or understand trends in customer or prospect behaviour. This analytical data often requires looking at very high volume, complex data sources. To bring this together in a format that is easy for analysts to understand and query is often very challenging - particularly for businesses when business requirements for this data change and a rapid response can mean the difference between profit and loss. This is just one of many areas that Data Integration tools and technologies are being applied - providing the 'plumbing' from a source system to a target system. DI tools are designed to offer an order of magnitude increase in developer productivity compared to using languages such as SQL, Java and .NET. This productivity allows developers to deliver more quickly, respond to changes faster or deliver more with fewer resources. According to Gartner, the market for such tools is estimated to grow to $2..7 billion by 2013, and is currently dominated by a handful of enterprise class vendors. However, a new crop of Data Integration tools is emerging, with a mix of open source and commercial offerings each that seek to challenge the dominance of the established players. This talk will discuss the history of this area of technology to help understand the conditions we see today, offer a view of the future of the market and describe how these tools can help drive value within today's business and academic communities. At the end of the talk, attendees will have an opportunity to use one of the commercial tools and make their own minds up about the value of such technology. Phil Watt is Principal Consultant at one of the world’s largest Systems Integrators, and has been working with high volume enterprise data for more than 17 years, building and designing data warehouses for customers in telco, media, utilities and financial services sectors. During the last 10 years, Phil has worked with a number of Data Integration technologies and advised many businesses about choosing a DI tool and applying best practices in their deployment.

1. Unlocking value from data with Data Integration Tools Phil Watt, Principal Integration Architect, HP Business Intelligence Solutions, EMEA 29/04/2010 1

2. Outline Introduction Business drivers – why use a DI tool? the challenge private sector public sector Background and history DI tools timeline Emerging features – and value Governance and Best Practice Selecting a tool for your situation Demonstration: Summary – followed by hands on session 29/04/2010 2

3. About me 29/04/2010 3 19 years big data 10 years Data Integration tools High volume Complex business rules Governance and metadata management Clients include BSkyB BT Barclays/Barclaycard Centrica Experian John Lewis Partnership Microsoft A major UK political party Strong focus on pragmatic delivery Best practices Design patterns Tool evaluation, selection and implementation

4. Scope 29/04/2010 4

5. Glossary 29/04/2010 5

6. The challenge 29/04/2010 6

7. Data warehouse example sizes 29/04/2010 7

8. Public and academic examples 29/04/2010 8 Birmingham City Council http://www.experian.co.uk/www/pages/about_us/our_clients/ http://www.qas.co.uk/company/press/new-experian-software-helps-public-sector-to-enhance-single-citizen-view-projects-503.htm University of Toulouse – academic medical research http://www.talend.com/open-source-provider/casestudy/CaseStudy_Academic_Medical_Research_EN.php

9. Benefits of DI tools 29/04/2010 9

10. Extract, Transform and Load 29/04/2010 10 e.g. CRM or ERP system Hub and spoke Shared DW and ETL server

11. Extract, Load and Transform 29/04/2010 11 e.g. CRM or ERP system Shared DW and ETL server

12. ETL versus ELT 29/04/2010 12

13. Multiple sources and targets 29/04/2010 13

14. DI Tools Features Timeline1995 – 2005 29/04/2010 14

15. DI Tools Features Timeline from 2006 29/04/2010 15

16. Market features 29/04/2010 16

17. Gartner Magic Quadrant Taken from research document, ‘Magic Quadrant for Data Integration Tools’ Authors: Ted Friedman, Mark A. Beyer, Eric Thoo Full report available by registering at www.talend.com 29/04/2010 17 Image removed for web publication as agreed with Gartner

18. Magic Quadrant Disclaimer The Magic Quadrant is copyrighted November 25, 2009 by Gartner, Inc. and is reused with permission. The Magic Quadrant is a graphical representation of a marketplace at and for a specific time period. It depicts Gartner's analysis of how certain vendors measure against criteria for that marketplace, as defined by Gartner. Gartner does not endorse any vendor, product or service depicted in the Magic Quadrant, and does not advise technology users to select only those vendors placed in the "Leaders" quadrant. The Magic Quadrant is intended solely as a research tool, and is not meant to be a specific guide to action. Gartner disclaims all warranties, express or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose. 29/04/2010 18

19. Best practices 29/04/2010 19

20. Worst Practices 29/04/2010 20

22. Allocate 20 - 30% to mapping and transformation rules

23. Avoid custom-coding or desktop tools

24. Increase business user involvement to improve successBest Practices Mitigate Data Migration Risks and Challenges – May 2009

25. Governance and the data integration lifecycle 29/04/2010 22

27. Spend 50% of project time doing discovery, analysis, design

28. Get business users involved early and often

29. Use tools to accelerate and compress timescales

30. Pay attention to governance and metadata

31. So you can:

32. De-risk the project

33. Reduce overall cost and timescales

35. Qualification matrix (PW ) 29/04/2010 25

36. Demonstration 29/04/2010 26

37. 29/04/2010 27

38. 29/04/2010 28

39. 29/04/2010 29

40. 29/04/2010 30

41. 29/04/2010 31

42. 29/04/2010 32

43. Demo metrics 29/04/2010 33 Performance Hardware – dual core 2.0Ghz Intel Centrino, 2.5Gb Ram Environment – WinXP, Oracle Express (DB) +DI tool (Expressor 2.0) 3 data sources Customers 155 MB 1000K records Today’s orders 112 MB 100K records Yesterday's orders 0.3 MB 3K records Total data volume 267 MB 1.1M records Execution time 72 seconds Throughput 3.7 MB/sec 41k/sec

44. Demo features 29/04/2010 34 Developer Productivity Graphical development Semantic Rationalisation and Re-usable Business Rules Demo represents a generic business scenario XML, message queues (MSMQ) , database inputs/outputs, joins, aggregations and referential integrity management Similar features to the ATG/Integrated Basket challenges?

45. Summary 29/04/2010 35 Business drivers – why use a DI tool? the challenge private sector public sector Background and history DI tools timeline Emerging features – and value Governance and Best Practice Selecting a tool for your situation Demonstration:

46. Questions 29/04/2010 36

47. References 29/04/2010 37 Curt Monashhttp://www.dbms2.com/2009/04/30/ebays-two-enormous-data-warehouses/ Wired: http://www.wired.com/wired/archive/12.04/grid.html Zdnet: http://blogs.zdnet.com/storage/?p=213 Professor Chris Bishop: http://conferences.theiet.org/lectures/turing/ Gartner http://www.gartner.com LHC data (2007): http://www-conf.slac.stanford.edu/xldb07/xldb_lhc.pdf