This is a copy of a presentation I gave at the Mastering Business Objects conference in Sydney, June 2011. It explains the move Star Track Express is making towards Active Data Warehousing to support both Analytical and Operational needs from a single platform.

1. Active Data Warehousing Can you have your cake and eat it too ? Jeff Monico General Manager Information Systems

3. THE ‘REAL TIME ENTERPRISE’

4. THE TRIGGER FOR CHANGE

5. ACTIVE DATA WAREHOUSING

6. THE RESULTS

7. OPERATIONAL INTELLIGENCE

9. Founded in 1974 by Greg Poche. Greg’s influence and core guiding principles are very much part of the business ethos today.

10. December 2003 saw Australia Post and Qantas form a joint venture to acquire Star Track Express from its founder.

11. On the 18th May 2011 the retail division of Australian Air Express was merged into Star Track Express to create Australia’s largest door-to-door express service provider.

12. 4,500+ Employees, 3,750+ Vehicles (Vans, Trucks, Prime Movers, Trailers, Forklifts).

14. How we collect and use data Business activity has a very short life cycle, with data being collected and used at many points during that lifecycle…… 18:30 18:40 19:00 16:45 DATA COLLECTION (SCANS) TIME EVENT DESCRIPTION 16:45 Freight picked up 18:30 Freight unloaded at Sortation Depot 18:40 Freight sorted at Sortation Depot into Lane 21 19:00 Freight loaded onto interstate trailer M200 05:00 Freight unloaded at Sortation Depot 05:35 Freight sorted at Sortation Depot onto Centre Wishbone 07:00 Freight loaded onto delivery truck ABC123 09:30 Freight successfully delivered 09:30 Consignment Note signed by Bob. 09:30 DATA USAGE 05:00 05:35 07:00

15. Problems we face Large Volumes of Data Producing millions or records per day which are needed for operational and analytical queries. Fast Response Times Users demand sub second response times on operational queries. Near Real Time Loads Real time business needs real time data from a variety of sources. Latency of more then a minute or two is an issue.

16. Large Data Sets Extending the data being collected from financial, to analytical, to transactional created exponentially larger data sets……..

17. Low Latency Data comes from multiple sources, from inside and outside of local networks and needs to be available with very low latency……… OLTP SQL Server Wireless IP Delivery Event ? ? File Transfer Mainframe (RMS) Sortation Event EDW

18. Varied Query Response Needs <1s <1s <10s <1m

20. Customer is curious and a bit frustrated as to why 2 trucks are needed…

21. 1 connote, 5 cartons and 1 pallet arrive at destination depot, pallet goes to bulk warehouse

22. Cartons and bulk are loaded ontoonetruck for delivery

23. Customer is pleased to receive the consolidated deliveryTDWI The Business Case for Real-Time BI Based on concept developed by Richard Hackathorn, Bolder Technology Accelerating Decisions

24. The trigger for change By 2010 we had identified two needs in the business……. New Operational Data Store to act as central data repository for SOA based architecture. New Enterprise Data Warehouse to meet data growth as we extended the subject areas captured. ODS EDW ADW Single platform to provide both an Enterprise Data Warehouse and an Operational Data Store – an Active Data Warehouse

25. What is an Active Data Warehouse ? Activating the Data Warehouse means more then just a very powerful fast database server……

26. MPP Data Warehouse Server Large volumes, active load and active access needs a different type of database technology – Massive Parallel Processing…… EXAMPLE: 100 random cards, return all 7 of Hearts. 1 ‘CPU’ 100 cards. 100 ‘CPU’ Cycles 100 Seconds 4 ‘CPU’s’ 25 cards each Each ‘CPU’ holds a suit. 25 ‘CPU’ Cycles from 1 CPU (3 ‘CPU’s’ Idle) 25 Seconds 4 x Improvement in Time 4 x Improvement in CPU 2 ‘CPU’s’ 50 cards each. 100 ‘CPU’ Cycles (50 cycles each) 50 Seconds 2x Improvement in Time NO Improvement in CPU

27. Active Workload Management Active management of workload is needed to ensure information is delivered at the ‘right time’ and at the lowest total cost of ownership……

28. Streaming Data Traditional batch processing will not deliver the near real-time data loads an Active Data Warehouse demands…… ENTERPRISE SERVICE BUS RMS OLTP Change Data Capture OLTP EDW

29. Information Delivery Traditional Business Intelligence tools (reporting) will not deliver information in a manner that enables fast decision making…… Well designed Dashboards deliver information which can be consumed very rapidly using good visual design; Visual based ad-hoc analysis tools (Explorer) provide users capability to rapidly get to the information they need from vast data sets; Mobile delivery provides information to people when and where they need it.

30. Architecture ADW Change Data Capture ENTERPRISE SERVICE BUS Data Services WebI Dashboards

31. The results August 2010 – ‘Lift and Drop’ of existing SQL Data Warehouse onto Teradata ADW. Approximately 500GB migrated, and approximately 1,000 complex BI queries per day are run against this. November 2010 –ODS and new Web Site goes Live. Supporting hundreds of thousands of tactical queries per day on the ADW platform. Now ADW platform is supporting this mix of BI and ODS loads and queries, and has maintained 100% up-time and 100% of queries meeting SLA’s. 1 DBA + 1 Platform = Low Total Cost of Ownership. EDW ODS

32. Operational Intelligence ACTIVATINGMAKE it happen! OPERATIONALIZING WHAT IShappening? PREDICTING WHAT WILL happen? Event-based triggering takes hold ANALYZING WHY did it happen? Value Continuous update and time-sensitive queries become important REPORTING WHAT happened? Batch Ad Hoc Analytics Analytical modeling grows Increase in ad hoc analysis Continuous Update/Short Queries Event-Based Triggering Primarily batch and some ad hoc reports Complexity

33. Questions, Comments and Observations