An Open-source Similar-name Finder presented by Dallan Quass at RootsTech 2012 An improvement on Soundex

1. An Open-source Similar-name Finder Dallan Quass [email_address]

2. What's the problem?

3. People can't spell unusual names Maybe a piece of mail addressed to Solverg Quast? Solverg Quast 5934 Phoenix Ave. Shoreview, MN 55126 Johnston Bros. 1256 Bristol St. Mapleton, MN 55126 Should be: Solveig Quass

4. People use nicknames John Johnny Jack

5. Transcribers make typos Jhon

6. Most of our ancestors didn't know how to read or write  signature

7. What does it matter?

8. How do you find records? Johnny Snith John Smith

9. How do you match people? John Smith Johnny Smithe

10. Not a new problem

11. Lots of solutions Soundex Nysiis Double Metaphone Refined Soundex Daitch-Mokotoff Caverphone Levenstein Jaro Winkler Monge Elkan Needleman Wunch Smith Waterman

12. No Bullseye

13. Why is this so hard?

14. How similar are two names? We’re neighbors John Jonny Joe I don’t know those guys

16. First approach: Coders Jim John Jane Johan Johannes

18. Second approach: Distance functions Jim John Jane Johan Johannes Better results, but Doesn't scale well

19. Can we do better?

20. Warning: Machine learning ahead!

22. A closer look at Levenstein Jon John Bohn -1 -1

25. Wait, i sn't this just another distance function? Distance functions don't scale, right?

26. Right

27. Back to the basics x f(x) -5 -1 -3 4.5 0 7 2 3.5 4 2

28. Long tail

29. Long tail 200,000 Surnames 70,000 Given names ≤ 1/5,000,000 names

30. Long tail Use distance function with table here Use coder here

31. Result: Table initialized by a function Dallan: Dalana Daleen Dalen Dalin … Talan Tallon Ryan: Aaran Aran Arrin … Rian Riana ...

32. A nice thing about tables... Dallan: Dalana Daleen Dalen Dalin … Talan Tallon Ryan: Aaran Aran Arrin … Rian Riana ...

33. Add to the table Nicknames BehindTheName.com The New American Dictionary of Baby Names by Leslie Dunking and William Gosling A Dictionary of Surnames by Patrick Hanks and Flavia Hodges WeRelate community

34. Thank you BehindTheName.com! Fascinating Family Trees for given names

35. Result Soundex Our approach Precision Recall 28% decrease in false negatives Given names Soundex Our approach Precision Recall 28% decrease in false negatives Surnames 97 65 97 74 89 68 89 77

36. Who is using it?

37. WeRelate.org

38. Continuous improvement

39. Continuous improvement

40. Community oversight

43. Future work