2. Topics Introduction & Current Perceptions What is Data Mining? How is Data Mining used? Why is Data Mining important? Questions
3. A little story involving … New G/L system Curious Audit Manager Questionable accounting entries
4. Introduction IT Audit Manager for Hillsborough County Certified as a CISA and CIA Spend 50% doing Data Mining Audit Risk Assessment Testing control effectiveness Compliance Fraud Detection
5. Introduction Who are you? Accountants Auditors Consultants Other industries
7. Heard of CAATs? Computer Assisted Audit Techniques Formerly a specialized skill for IT Auditors Common in every audit Term is practically obsolete
8. What is Data Mining? Automate the detection of relevant patterns Look at current & historical data Predict future trends Efficient method for analyzing large amounts of data Enhance key item sampling Means for continuous auditing
9. How is Data Mining used? Proactively review business processes Identify anomalies Risk Assessment Reactively assist law enforcement in investigations
10. How is Data Mining used? Outside of Audit, DM is used to generate revenue Automating the detection of relevant patterns Look at current & historical data Predict future trends – “Predictive Analysis” aka Business Intelligence / Data Warehouse
12. Charity Fundraising Hospital in San Diego, CA Patients get their treatment and their assets scanned Donor research on your salary history, LinkedIn connections, satellite images of your pool
14. Charity Fundraising ASPCA – using DM and Predictive Analysis to determine donors 4x donations over 5 years - $80M
15. Charity Fundraising Charity’s Junk Mail Strategy Mailings Double Dip – another request Virus effect “Dear Friend” – personal recognition
16. How is Data Mining used? Audit Process Risk Assessment Control Assessment Observations Fraud Detection and Prevention
17. How is Data Mining used? Risk Assessment Data analysis for high risk areas High Dollar amounts Potential for fraud Potential for non-compliance
18. How is Data Mining used? Risk Assessment What can be detected? Potential fraud or control weaknesses Duplicate vendors Duplicate invoices Duplicate amounts Benford’s Law – identify suspicious transactions Focus audit on high risk areas
19. How is Data Mining used? Control Assessment Traditional audit used sampling approach Auditors placed disclaimers regarding the accuracy of their statistical sampling Not affordable or available anymore Total assurance & clear indication of errors DM uses 100% of transactions Increases credibility & value of audit
20. Why is Data Mining important? Examples of Data Mining Proactive - Purchasing and Procurement Reactive - Health Plan Auditing
21. Purchasing and Procurement Common area for fraud Abuse of financial authority Technical manipulation of specifications Internal collusion to circumvent controls External collusion with suppliers Manipulation of bid review Overbilling Bogus invoices
22. Purchasing and Procurement 2003 review - five local government procurement agencies in London Purchasing managers Specified the bid criteria Suggested pre-approved companies Suspected in collusion with suppliers DM used to identify possible trends
23. Purchasing and Procurement Analyzed “win/lose” statistics of the vendors Number of bids won Number of bids lost due to cost Number of bids where vendor failed approval Number of bids lost for “other” reasons
24. Purchasing and Procurement Determined two key elements: Vendors that consistently lost their bids – “shadow bidders” Vendors that won over 95% of bids Team of forensic experts were used for contract review and work performed
25. Purchasing and Procurement Collusion to circumvent controls in place Required to have five bids Used shadow bidders Bid review group Ensured that a selected (and corrupt) supplier was chosen every time Allowed substitution of inferior materials
26. Health Plan Auditing 2010 case study – Conducted by St. Joseph’s Univ. & Healthcare Data Mgmt Two large companies’ health insurance claims data over two year period Company A – 108,000 claims, $25.3M paid Company B – 464,000 claims, $118.4M paid
27. Health Plan Auditing Compared the results of 100% auditing vs. random-sampling claims (300-400 samples) 100% auditing produced very distinct results Company A - $3.12M exception claims Company B - $5.47M exception claims
32. Health Plan Auditing Random-sampling claims - used best “analysis” to simulate the audits Using exceptions from the 100% auditing approach 100 random samples of 300 exceptions 100 random samples of 400 exceptions Statistically close to their “population of exceptions” parameters from 100% auditing
33.
34. Health Plan Auditing Random-sampling missed a significant amount of exception claim amounts Increasing the sample size from 300 to 400 Did not significantly identify more errors Over 90% of claim errors still missed Significant amounts of money are wasted
35. Health Plan Auditing Random-sampling does not identify root cause of errors Trend analysis only possible through data mining
37. Contact Information ed.tobias@hillsclerk.com LinkedIn - http://www.linkedin.com/in/ed3200
38. References ACFE. 2008 Report to the Nation on Occupational Fraud & Abuse. 2008. Retrieved 6/1/10 fromhttp://www.acfe.com/documents/2008-rttn.pdf Barrier, M. One right path: Cynthia Cooper. 2003. Retrieved 6/2/10 from http://findarticles.com/p/articles/mi_m4153/is_6_60/ai_111737943/ Bourke, J. Computer Assisted Audit Techniques or CAATS. 2010. Retrieved 5/25/10 from https://www.cpa2biz.org/Content/media/PRODUCER_CONTENT/Newsletters/Articles_2010/CPA/Jan/CAATS.jsp Deeson, M. Audit says Office Depot overcharged county one million dollars. 2010. Retrieved 6/3/10 from http://www.wtsp.com/news/local/story.aspx?storyid=130518 Denker, B. Data Mining and the Auditor’s Responsibility. 2003. Retrieved 6/1/10 from http://www.isaca.org/Content/ContentGroups/InfoBytes/20032/Data_Mining_and_the_Auditors_Responsibility.htm Kadet, A. Are Charity Fundraisers Spying on You? 2010. Retrieved 6/1/10 from http://www.smartmoney.com/personal-finance/estate-planning/are-charity-fundraisers-spying-on-you
39. References Kadet, A. Your Charity’s Junk-Mail Strategy. 2010. Retrieved 6/1/10 from http://www.smartmoney.com/personal-finance/estate-planning/are-charity-fundraisers-spying-on-you Kusnierz, R. A Case for Data Mining. 2003. Retrieved 6/1/10 from http://www2.northumberland.gov.uk/fraud/Documents/HM%20Treasury%20Reports/fraud_anti_fraud_adv_02-03.pdf Sayana, A. Using CAATs to Support IS Audit. 2003. Retrieved 6/1/10 from http://www.isaca.org/Journal/Past-Issues/2003/Volume-1/Pages/Using-CAATS-to-Support-IS-Audit.aspx Sillup, G. and Klimberg, R. Health Plan Auditing: 100-Percent-of-Claims vs. Random-Sample Audits. 2010. Retrieved 6/1/10 from http://www.sawgrassbc.com/captives/Health%20Plan%20Auditing_%20100%20Percent%20verses%20Random%20Sample%2001.2010%20.pdf Silltow, J. Data Mining 101: Tools and Techniques. 2006. Retrieved 5/25/10 from http://www.theiia.org/intAuditor/itaudit/archives/2006/august/data-mining-101-tools-and-techniques/ Wolfe, J. Effective Data Mining for Financial Services Companies. 2008. Retrieved 6/1/10 from http://www.theiia.org/intAuditor/in-the-industry/2008/november/effective-data-mining-for-financial-services-companies/index.cfm?print&search=jonathan%20wolfe&Y=1899 (IIA members only URL) Zink, J. Office Depot billing disputed. 2010. Retrieved 6/3/10 from http://www.tampabay.com/news/localgovernment/article1090581.ece