Enviar búsqueda
Cargar
Elder
•
1 recomendación
•
463 vistas
George Ang
Seguir
Tecnología
Denunciar
Compartir
Denunciar
Compartir
1 de 25
Descargar ahora
Descargar para leer sin conexión
Recomendados
Learning Local Lessons in Software Engineering
Learning Local Lessons in Software Engineering
CS, NcState
Root cause analysis
Root cause analysis
Simmy Sharma
SafeguardAI and Surprise Based Learning -- Protect your AI solutions from Uni...
SafeguardAI and Surprise Based Learning -- Protect your AI solutions from Uni...
NAVER Engineering
L6 handout maher mc carthy are you ready for this
L6 handout maher mc carthy are you ready for this
Helen Bevan
IaaS Cloud Benchmarking: Approaches, Challenges, and Experience
IaaS Cloud Benchmarking: Approaches, Challenges, and Experience
Alexandru Iosup
S.P.A.C.E. Exploration for Software Engineering
S.P.A.C.E. Exploration for Software Engineering
CS, NcState
wicked problems 20-may-14_rev-rkg
wicked problems 20-may-14_rev-rkg
Bob Garrett
Unit Testing Case Study for COJUG - 05.11.2010
Unit Testing Case Study for COJUG - 05.11.2010
Nick Watts
Recomendados
Learning Local Lessons in Software Engineering
Learning Local Lessons in Software Engineering
CS, NcState
Root cause analysis
Root cause analysis
Simmy Sharma
SafeguardAI and Surprise Based Learning -- Protect your AI solutions from Uni...
SafeguardAI and Surprise Based Learning -- Protect your AI solutions from Uni...
NAVER Engineering
L6 handout maher mc carthy are you ready for this
L6 handout maher mc carthy are you ready for this
Helen Bevan
IaaS Cloud Benchmarking: Approaches, Challenges, and Experience
IaaS Cloud Benchmarking: Approaches, Challenges, and Experience
Alexandru Iosup
S.P.A.C.E. Exploration for Software Engineering
S.P.A.C.E. Exploration for Software Engineering
CS, NcState
wicked problems 20-may-14_rev-rkg
wicked problems 20-may-14_rev-rkg
Bob Garrett
Unit Testing Case Study for COJUG - 05.11.2010
Unit Testing Case Study for COJUG - 05.11.2010
Nick Watts
Drivers of Housing Demand: Preparing for the Impending Elder Boom
Drivers of Housing Demand: Preparing for the Impending Elder Boom
GNOCDC
Children under the age of three in formal care in Eastern Europe and Central ...
Children under the age of three in formal care in Eastern Europe and Central ...
UNICEF Europe & Central Asia
Many Children Left Behind
Many Children Left Behind
Melissa Fulton
Angola child soldiers
Angola child soldiers
guestba9d13
GAEBrochureFinal2010
GAEBrochureFinal2010
Simon Roberts
State of Nigerian children
State of Nigerian children
Young Bill
Thia Presentation April 2 2008 B
Thia Presentation April 2 2008 B
Scott Nystrom, Ph.D.
Universal health care
Universal health care
Dr.Hemant Kumar
Debugging machine-learning
Debugging machine-learning
Michał Łopuszyński
Outliers and Inconsistency
Outliers and Inconsistency
Neil Rubens
Statistics in the age of data science, issues you can not ignore
Statistics in the age of data science, issues you can not ignore
Turi, Inc.
NYC Open Data Meetup-- Thoughtworks chief data scientist talk
NYC Open Data Meetup-- Thoughtworks chief data scientist talk
Vivian S. Zhang
920 plenary elder
920 plenary elder
Rising Media, Inc.
910 plenary Elder
910 plenary Elder
Rising Media, Inc.
2014 aus-agta
2014 aus-agta
c.titus.brown
L15. Machine Learning - Black Art
L15. Machine Learning - Black Art
Machine Learning Valencia
Data Science Folk Knowledge
Data Science Folk Knowledge
Krishna Sankar
840 plenary elder_using his laptop
840 plenary elder_using his laptop
Rising Media, Inc.
Artificial Intelligence for Medicine
Artificial Intelligence for Medicine
Tassilo Klein
Input modeling
Input modeling
De La Salle University-Manila
Dm sei-tutorial-v7
Dm sei-tutorial-v7
CS, NcState
Machine Learning for Incident Detection: Getting Started
Machine Learning for Incident Detection: Getting Started
Sqrrl
Más contenido relacionado
Destacado
Drivers of Housing Demand: Preparing for the Impending Elder Boom
Drivers of Housing Demand: Preparing for the Impending Elder Boom
GNOCDC
Children under the age of three in formal care in Eastern Europe and Central ...
Children under the age of three in formal care in Eastern Europe and Central ...
UNICEF Europe & Central Asia
Many Children Left Behind
Many Children Left Behind
Melissa Fulton
Angola child soldiers
Angola child soldiers
guestba9d13
GAEBrochureFinal2010
GAEBrochureFinal2010
Simon Roberts
State of Nigerian children
State of Nigerian children
Young Bill
Thia Presentation April 2 2008 B
Thia Presentation April 2 2008 B
Scott Nystrom, Ph.D.
Universal health care
Universal health care
Dr.Hemant Kumar
Destacado
(8)
Drivers of Housing Demand: Preparing for the Impending Elder Boom
Drivers of Housing Demand: Preparing for the Impending Elder Boom
Children under the age of three in formal care in Eastern Europe and Central ...
Children under the age of three in formal care in Eastern Europe and Central ...
Many Children Left Behind
Many Children Left Behind
Angola child soldiers
Angola child soldiers
GAEBrochureFinal2010
GAEBrochureFinal2010
State of Nigerian children
State of Nigerian children
Thia Presentation April 2 2008 B
Thia Presentation April 2 2008 B
Universal health care
Universal health care
Similar a Elder
Debugging machine-learning
Debugging machine-learning
Michał Łopuszyński
Outliers and Inconsistency
Outliers and Inconsistency
Neil Rubens
Statistics in the age of data science, issues you can not ignore
Statistics in the age of data science, issues you can not ignore
Turi, Inc.
NYC Open Data Meetup-- Thoughtworks chief data scientist talk
NYC Open Data Meetup-- Thoughtworks chief data scientist talk
Vivian S. Zhang
920 plenary elder
920 plenary elder
Rising Media, Inc.
910 plenary Elder
910 plenary Elder
Rising Media, Inc.
2014 aus-agta
2014 aus-agta
c.titus.brown
L15. Machine Learning - Black Art
L15. Machine Learning - Black Art
Machine Learning Valencia
Data Science Folk Knowledge
Data Science Folk Knowledge
Krishna Sankar
840 plenary elder_using his laptop
840 plenary elder_using his laptop
Rising Media, Inc.
Artificial Intelligence for Medicine
Artificial Intelligence for Medicine
Tassilo Klein
Input modeling
Input modeling
De La Salle University-Manila
Dm sei-tutorial-v7
Dm sei-tutorial-v7
CS, NcState
Machine Learning for Incident Detection: Getting Started
Machine Learning for Incident Detection: Getting Started
Sqrrl
Ml pluss ejan2013
Ml pluss ejan2013
CS, NcState
Jsm big-data
Jsm big-data
Sean Taylor
Machine Learning for Data Extraction
Machine Learning for Data Extraction
Dasha Herrmannova
Icse15 Tech-briefing Data Science
Icse15 Tech-briefing Data Science
CS, NcState
Decision Modelling for n00bs
Decision Modelling for n00bs
Suranga Nath Kasthurirathne
Biological Foundations for Deep Learning: Towards Decision Networks
Biological Foundations for Deep Learning: Towards Decision Networks
diannepatricia
Similar a Elder
(20)
Debugging machine-learning
Debugging machine-learning
Outliers and Inconsistency
Outliers and Inconsistency
Statistics in the age of data science, issues you can not ignore
Statistics in the age of data science, issues you can not ignore
NYC Open Data Meetup-- Thoughtworks chief data scientist talk
NYC Open Data Meetup-- Thoughtworks chief data scientist talk
920 plenary elder
920 plenary elder
910 plenary Elder
910 plenary Elder
2014 aus-agta
2014 aus-agta
L15. Machine Learning - Black Art
L15. Machine Learning - Black Art
Data Science Folk Knowledge
Data Science Folk Knowledge
840 plenary elder_using his laptop
840 plenary elder_using his laptop
Artificial Intelligence for Medicine
Artificial Intelligence for Medicine
Input modeling
Input modeling
Dm sei-tutorial-v7
Dm sei-tutorial-v7
Machine Learning for Incident Detection: Getting Started
Machine Learning for Incident Detection: Getting Started
Ml pluss ejan2013
Ml pluss ejan2013
Jsm big-data
Jsm big-data
Machine Learning for Data Extraction
Machine Learning for Data Extraction
Icse15 Tech-briefing Data Science
Icse15 Tech-briefing Data Science
Decision Modelling for n00bs
Decision Modelling for n00bs
Biological Foundations for Deep Learning: Towards Decision Networks
Biological Foundations for Deep Learning: Towards Decision Networks
Más de George Ang
Wrapper induction construct wrappers automatically to extract information f...
Wrapper induction construct wrappers automatically to extract information f...
George Ang
Opinion mining and summarization
Opinion mining and summarization
George Ang
Huffman coding
Huffman coding
George Ang
Do not crawl in the dust different ur ls similar text
Do not crawl in the dust different ur ls similar text
George Ang
大规模数据处理的那些事儿
大规模数据处理的那些事儿
George Ang
腾讯大讲堂02 休闲游戏发展的文化趋势
腾讯大讲堂02 休闲游戏发展的文化趋势
George Ang
腾讯大讲堂03 qq邮箱成长历程
腾讯大讲堂03 qq邮箱成长历程
George Ang
腾讯大讲堂04 im qq
腾讯大讲堂04 im qq
George Ang
腾讯大讲堂05 面向对象应对之道
腾讯大讲堂05 面向对象应对之道
George Ang
腾讯大讲堂06 qq邮箱性能优化
腾讯大讲堂06 qq邮箱性能优化
George Ang
腾讯大讲堂07 qq空间
腾讯大讲堂07 qq空间
George Ang
腾讯大讲堂08 可扩展web架构探讨
腾讯大讲堂08 可扩展web架构探讨
George Ang
腾讯大讲堂09 如何建设高性能网站
腾讯大讲堂09 如何建设高性能网站
George Ang
腾讯大讲堂01 移动qq产品发展历程
腾讯大讲堂01 移动qq产品发展历程
George Ang
腾讯大讲堂10 customer engagement
腾讯大讲堂10 customer engagement
George Ang
腾讯大讲堂11 拍拍ce工作经验分享
腾讯大讲堂11 拍拍ce工作经验分享
George Ang
腾讯大讲堂14 qq直播(qq live) 介绍
腾讯大讲堂14 qq直播(qq live) 介绍
George Ang
腾讯大讲堂15 市场研究及数据分析理念及方法概要介绍
腾讯大讲堂15 市场研究及数据分析理念及方法概要介绍
George Ang
腾讯大讲堂15 市场研究及数据分析理念及方法概要介绍
腾讯大讲堂15 市场研究及数据分析理念及方法概要介绍
George Ang
腾讯大讲堂16 产品经理工作心得分享
腾讯大讲堂16 产品经理工作心得分享
George Ang
Más de George Ang
(20)
Wrapper induction construct wrappers automatically to extract information f...
Wrapper induction construct wrappers automatically to extract information f...
Opinion mining and summarization
Opinion mining and summarization
Huffman coding
Huffman coding
Do not crawl in the dust different ur ls similar text
Do not crawl in the dust different ur ls similar text
大规模数据处理的那些事儿
大规模数据处理的那些事儿
腾讯大讲堂02 休闲游戏发展的文化趋势
腾讯大讲堂02 休闲游戏发展的文化趋势
腾讯大讲堂03 qq邮箱成长历程
腾讯大讲堂03 qq邮箱成长历程
腾讯大讲堂04 im qq
腾讯大讲堂04 im qq
腾讯大讲堂05 面向对象应对之道
腾讯大讲堂05 面向对象应对之道
腾讯大讲堂06 qq邮箱性能优化
腾讯大讲堂06 qq邮箱性能优化
腾讯大讲堂07 qq空间
腾讯大讲堂07 qq空间
腾讯大讲堂08 可扩展web架构探讨
腾讯大讲堂08 可扩展web架构探讨
腾讯大讲堂09 如何建设高性能网站
腾讯大讲堂09 如何建设高性能网站
腾讯大讲堂01 移动qq产品发展历程
腾讯大讲堂01 移动qq产品发展历程
腾讯大讲堂10 customer engagement
腾讯大讲堂10 customer engagement
腾讯大讲堂11 拍拍ce工作经验分享
腾讯大讲堂11 拍拍ce工作经验分享
腾讯大讲堂14 qq直播(qq live) 介绍
腾讯大讲堂14 qq直播(qq live) 介绍
腾讯大讲堂15 市场研究及数据分析理念及方法概要介绍
腾讯大讲堂15 市场研究及数据分析理念及方法概要介绍
腾讯大讲堂15 市场研究及数据分析理念及方法概要介绍
腾讯大讲堂15 市场研究及数据分析理念及方法概要介绍
腾讯大讲堂16 产品经理工作心得分享
腾讯大讲堂16 产品经理工作心得分享
Último
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
Anna Loughnan Colquhoun
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
The Digital Insurer
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
Rafal Los
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
gurkirankumar98700
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
debabhi2
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Igalia
🐬 The future of MySQL is Postgres 🐘
🐬 The future of MySQL is Postgres 🐘
RTylerCroy
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
giselly40
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
Results
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
hans926745
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Katpro Technologies
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
wesley chun
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
Principled Technologies
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
ThousandEyes
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
Allon Mureinik
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
Gabriella Davis
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Neo4j
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
Puma Security, LLC
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Drew Madelung
Último
(20)
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
🐬 The future of MySQL is Postgres 🐘
🐬 The future of MySQL is Postgres 🐘
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Elder
1.
Top 10 Data
Mining Mistakes -- and how to avoid them Salford Systems Data Mining Conference New York, New York March 29, 2005 Elder Research, Inc. 635 Berkmar Circle John F. Elder IV, Ph.D. Charlottesville, Virginia 22901 elder@datamininglab.com 434-973-7673 www.datamininglab.com © 2005 Elder Research, Inc. 0
2.
© 2005 Elder
Research, Inc. 1 © Hildebrandts
3.
You've made a
mistake if you… 0. Lack Data 1. Focus on Training 6. Discount Pesky Cases 2. Rely on One Technique 7. Extrapolate 3. Ask the Wrong Question 8. Answer Every Inquiry 4. Listen (only) to the Data 9. Sample Casually 5. Accept Leaks from the Future 10. Believe the Best Model © 2005 Elder Research, Inc. 2
4.
0. Lack Data •
Need labeled cases for best gains (to classify or estimate; clustering is much less effective). Interesting known cases may be exceedingly rare. Some projects probably should not proceed until enough critical data is gathered to make it worthwhile. • Ex: Fraud Detection (Government contracting): Millions of transactions, a handful of known fraud cases; likely that large proportion of fraud cases are, by default, mislabeled clean. Only modest results (initially) after strenuous effort. • Ex: Fraud Detection (Taxes; collusion): Surprisingly many known cases -> stronger, immediate results. • Ex: Credit Scoring: Company (randomly) gave credit to thousands of applicants who were risky by conventional scoring method, and monitored them for two years. Then, estimated risk using what was known at start. This large investment in creating relevant data paid off. © 2005 Elder Research, Inc. 3
5.
1. Focus on
Training • Only out-of-sample results matter. (Otherwise, use a lookup table!) • Cancer detection Ex: MD Anderson doctors and researchers (1993), using neural networks, surprised to find that longer training (week vs. day) led to only slightly improved training results, and much worse evaluation results. • (ML/CS often sought models with exact results on known data -> overfit.) • Re-sampling (bootstrap, cross-validation, jackknife, leave-one-out...) is an essential tool. (Traditional significance tests are a weak defense when structure is part of search, though stronger penalty-based metrics are useful.) However, note that resampling no longer tests a single model, but a model class, or a modeling process (Whatever is held constant throughout the sampling passes.) © 2005 Elder Research, Inc. 4
6.
2. Rely on
One Technique • "To a little boy with a hammer, all the world's a nail." For best work, need a whole toolkit. • At very least, compare your method to a conventional one (linear regression say, or linear discriminant analysis). • Study: In refereed Neural Network journals, over 3 year period, only 1/6 articles did both ~1 & ~2; that is, test on unseen data and compare to a widely-used technique. • Not checking other methods leads to blaming the algorithm for the results. But, it’s somewhat unusual for the particular modeling technique to make a big difference, and when it will is hard to predict. • Best: use a handful of good tools. (Each adds only 5-10% effort.) © 2005 Elder Research, Inc. 5
7.
Data Mining Products
Model 1 © 2005 Elder Research, Inc. 6
8.
Decision Tree
Nearest Neighbor Delaunay Triangles Kernel Neural Network (or © 2005 Elder Research, Inc. 7 Polynomial Network)
9.
Relative Performance Examples:
5 Algorithms on 6 Datasets (John Elder, Elder Research & Stephen Lee, U. Idaho, 1997) Error Relative to Peer Techniques (lower is better) © 2005 Elder Research, Inc. 8
10.
Error Relative to
Peer Techniques (lower is better) Essentially every Bundling method improves performance © 2005 Elder Research, Inc. 9
11.
3. Ask the
Wrong Question a) Project Goal: Aim at the right target • Fraud Detection (Positive example!) (Shannon Labs work on Int'l calls): Didn't attempt to classify fraud/nonfraud for general call, but characterized normal behavior for each account, then flagged outliers. -> A brilliant success. b) Model Goal: Get the computer to "feel" like you do. [e.g., employee stock grants vs. options] • Most researchers are drawn into the realm of squared error by its convenience (mathematical beauty). But ask the computer to do what's most helpful for the system, not what's easiest for it. [Stock price ex.] © 2005 Elder Research, Inc. 10
12.
4. Listen (only)
to the Data 4a. Opportunistic data: • [School funding ex.] Self-selection. Nothing inside the data protects analyst from significant, but wrong result. 4b. Designed experiment: • [Tanks vs. Background with Neural networks]: Great results on out- of-sample portion of database. But found to depend on random pixels sunny (Tanks photographed on ______ day, Background only on _______).cloudy • [Tanks & Networks 2]: Tanks and Trucks on rotating platforms, to train to discriminate at different angles. Used radar, Fourier transforms, principle components, polynomial networks. But, source of the key signal = platform corner And, discriminated between the _____________. bushes two classes primarily using _______. © 2005 Elder Research, Inc. 11
13.
5. Accept Leaks
from the Future • Forecasting ex.: Interest rate at Chicago Bank. output was a candidate input N.net 95% accurate, but _________________________. • today Financial ex. 2: moving average of 3 days, but centered on ______. • Many passes may be needed to expel anachronistic leakers - D. Pyle. • Look for variables which work (too) well. Insurance Ex: code associated with 25% of purchasers turned out to describe type of cancellation. • Date-stamp records when storing in Data Warehouse, or Don't overwrite old value unless archived. • Survivor Bias [financial ex.] © 2005 Elder Research, Inc. 12
14.
6. Discount Pesky
Cases • Outliers may be killing results (ex: decimal point error on price), or be the whole answer (ex: Ozone hole), so examine carefully. • The most exciting phrase in research isn't "Aha!", but "That's odd…" • Internal inconsistencies in the data may be clues to problems with the process flow of information within the company; a larger business problem may be revealed. • Direct Mail example: persisting in hunting down oddities found errors by Merge/Purge house, and was a major contributor to doubling sales per catalog. • Visualization can cover a multitude of assumptions. © 2005 Elder Research, Inc. 13
15.
7. Extrapolate •
Tend to learn too much from first few experiences. • Hard to "erase" factoids after an upstream error is discovered. • Curse of Dimensionality: low-dimensional intuition is useless in high-d. • Philosophical: Evolutionary Paradigm: Believe we can start with pond scum (pre-biotic soup of raw materials) + zap + time + chance + differential reinforcement -> a critter. (e.g., daily stock prices + MARS -> purchase actions, or pixel values + neural network -> image classification) Better paradigm is selective breeding: mutts + time + directed reinforcement -> greyhound Higher-order features of data + domain expertise essential © 2005 Elder Research, Inc. 14
16.
“Of course machines
can think. After all, humans are just machines made of meat.” - MIT CS professor Human and computer strengths are more complementary than alike. © 2005 Elder Research, Inc. 15
17.
8. Answer Every
Inquiry • "Don't Know" is a useful model output state. • Could estimate the uncertainty for each output (a function of the number and spread of samples near X). Few algorithms provide a conditional σ with their conditional µ. Global Rd Optimization when Probes are Expensive (GROPE) © 2005 Elder Research, Inc. 16
18.
9. Sample without
Care • 9a Down-sample Ex: MD Direct Mailing firm had too many non-responders (NR) for model (about 99% of >1M cases). So took all responders, and every 10th NR to create a more balanced database of 100K cases. Model predicted that everyone in Ketchikan, Wrangell, and Ward Cove Alaska Sorted zip code would respond. (______ data, by ________, and 100Kth case drawn before bottom of file (999**) reached.) • "Shake before baking". Also, add case number (and other random variables) to candidate list; use as "canaries in the mine" to signal trouble when chosen. • 9b Up-sampling Ex: Credit Scoring - paucity of interesting (Default) cases led to quintupling them. Cross-validation employed with many techniques and modeling cycles. Results tended to improve with the complexity of the models. Oddly, this didn't reverse. Noticed that Default (rare) cases were better estimated by complex models but others were worse. (Had duplicated up-sampling _________ Defaults in each set by ___________ before splitting.) -> Split first. • It's hard to beat a stratified sample; that is, proportional sample from each group. [even with Data Squashing - W. DeMouchel] © 2005 Elder Research, Inc. 17
19.
10. Believe the
Best Model • Interpretability not always necessary. Model can be useful without being "correct" or explanatory. • Often, particular variables used by "best" model (which barely won out over hundreds of others of the millions (to billions) tried, using a score function only approximating one's goals, and on finite data) have too much attention paid to them. (Un-interpretability could be a virtue!). • Usually, many very similar variables are available, and the particular structure of the best model can vary chaotically. [Polynomial Network Ex.] But, structural similarity is different from functional similarity. (Competing models often look different, but act the same.) • Best estimator is likely to be bundle of models. [Direct Marketing trees] [5x6 table] [Credit Scoring ex.] © 2005 Elder Research, Inc. 18
20.
Lift Chart: %purchasers
vs. %prospects • Ex: Last quintile of customers are 4 times more expensive to obtain than first quintile (10% vs. 40% to gain 20%) • Decision Tree provides relatively few decision points. © 2005 Elder Research, Inc. 19
21.
Bundling 5 Trees
improves accuracy and smoothness © 2005 Elder Research, Inc.
22.
Credit Scoring Model
Performance #Defaulters Missed (fewer is better) Bundled Trees SNT NT Stepwise Regression NS ST Polynomial Network MT PS SPT PNT SMT SPNT Neural Network PT NP MPT SPN SMPT MARS MNT MS SMN SMP SMNT MN SMPNT SMPN MP MPNT MPN #Models Combined (averaging output rank) 21 © 2005 Elder Research, Inc.
23.
Median (and Mean)
Error Reduced with each Stage of Combination 75 M 70 i s 65 s e d 60 55 1 2 3 4 5 No. Models in combination © 2005 Elder Research, Inc. 22
24.
Fancier tools and
harder problems –> more ways to mess up. How then can we succeed? Success <- Learning <- Experience <- Mistakes (so go out and make some good ones!) PATH to success: • Persistence - Attack repeatedly, from different angles. Automate essential steps. Externally check work. • Attitude - Optimistic, can-do. • Teamwork - Business and statistical experts must cooperate. Does everyone want the project to succeed? • Humility - Learning from others requires vulnerability. Don’t expect too much of technology. © 2005 Elder Research, Inc. 23
25.
John F. Elder
IV Chief Scientist, ERI Dr. John Elder heads a data mining consulting team with offices in Charlottesville, Virginia and Washington DC, and close affiliates in Boston, New York, San Diego, and San Francisco (www.datamininglab.com). Founded in 1995, Elder Research, Inc. focuses on investment and commercial applications of pattern discovery and optimization, including stock selection, image recognition, process optimization, cross-selling, biometrics, drug efficacy, credit scoring, market timing, and fraud detection. John obtained a BS and MEE in Electrical Engineering from Rice University, and a PhD in Systems Engineering from the University of Virginia, where he’s an adjunct professor, teaching Optimization. Prior to a decade leading ERI, he spent 5 years in aerospace defense consulting, 4 heading research at an investment management firm, and 2 in Rice's Computational & Applied Mathematics department. Dr. Elder has authored innovative data mining tools, is active on Statistics, Engineering, and Finance conferences and boards, is a frequent keynote conference speaker, and was a Program co-chair of the 2004 Knowledge Discovery and Data Mining conference. John’s courses on data analysis techniques -- taught at dozens of universities, companies, and government labs -- are noted for their clarity and effectiveness. Dr. Elder holds a top secret clearance, and since the Fall of 2001, has been honored to serve on a panel appointed by Congress to guide technology for the National Security Agency. John is a follower of Christ and the proud father of 5. © 2005 Elder Research, Inc.
Descargar ahora