Going through a PhD may be seen as a requirement for an academic career or a different kind of job, simply as “the next step” in education, as something to do “because why not?”, or even just as a hobby you have on the side. What it really is though, is a life-changing experience, something that can be terribly painful and amazingly rewarding at the same time. In that journey I learned a few lessons in the hard way, lessons that I wish someone had told me about at the time. In this talk I’ll try to do just that and not talk about the content and process of a PhD, but rather about you, the person, during your PhD.

1. Your PhD and You
@julian_urbano
7 Nov 2018 · UNED, Spain

2. • Julián Urbano
• Assistant Professor @ TU Delft, Netherlands
• (Music) Information Retrieval (Evaluation)
• And a few other things over the years
– Information Extraction
– Crowdsourcing
– Software Engineering… ¯_(ツ)_/¯
• Currently
– Reliability of evaluation experiments
– Statistical methods for the construction of datasets
– Stochastic simulation for evaluation
2

3. EDUCATION
3

4. • College is where we go to become professionals
• We acquire the knowledge that is necessary to
perform as such
• Teachers impart that knowledge
4

5. Undergrad
School
Professional
5

6. “You will not be an engineer
by the time you graduate.
You will just have the necessary vocabulary
to understand what real engineers talk about.”
—Some Professor from Córdoba
6

7. • But then I went to college…
• The world is much bigger, more complex, and yet
more beautiful than I ever imagined
• Becoming a professional is a life-long process
• Teachers give fundamental knowledge and the
tools to use it and become self-sufficient
7

8. Undergrad
School
Job
Self-learner
Professional
8

9. • Keep you motivated
• Enjoy learning as opposed to dislike studying
• Focus on why, not on how
• Think vs. memorize
9

10. Undergrad
School
Job
Life
Self-learner
Professional
10

11. • …and then I went to Grad School (twice)
• Produce research, new knowledge
• Rigor
• Evidence-based reasoning
• Scientific Method
11

12. Undergrad
School
Grad
School
Job
Life
ResearcherSelf-learner
Professional
12

13. • But a lot of what we learn is useful in real life
• Capability to avoid and detect
– Cognitive biases
– Logical fallacies
– Paradoxes
• How terrible we humans are at reasoning
13

14. Undergrad
School
Grad
School
Job
Life
Critical
Thinker
Self-learner
Professional
14

15. Sad Times
• We live in the times of the alternative truths,
where
– we have loads of information around us
– people don’t care about the truth, only about what
support their beliefs
– the media doesn’t give information but opinions
– everyone knows about everything
– clicking the ads is more important than veracity
– people don’t want to learn, but to be entertained
– you’re not allowed to disagree on certain topics
– scientific facts can be disputed with mere opinions
15

16. Undergrad
School
Grad
School
Job
Life
Critical
Thinker
Self-learner
Professional
16

17. Your PhD
• It’s not about the book
• It’s not about the title
• It’s not about choosing “Dr.” when you book an
airplane ticket
• It’s not really about your research
It’s about you
17

18. You
“Learned Doctor,
You now have the right to use the title of doctor.
Your doctorate means that society can rely on your
judgment, that you will act transparently and communicate
independently about your results and the societal relevance
of your work. In other words, your doctorate implies that
you will uphold scientific integrity.
I wish you a great deal of wisdom and prosperity with your
new status. On behalf of the Board for Doctorates of Delft
University of Technology, I congratulate you and your family
on earning your doctoral degree.”
18

19. • This talk is not be about my research
• It’ll be about you, the PhD student, the person
• Based on own experience and someone else’s
19

20. LESSONS LEARNED
21

21. Many things to say for the 1st time
22

22. Many things to say for the 1st time
22

23. Many things to say for the 1st time
22

24. 23

25. Outline
• Tools
• The fundamentals
• Your message
• The others
• You
• Life
24

26. TOOLS
25

27. Master your toolbox
• Develop your own data processing workflow ASAP
– It’ll take some iterations, so don’t paralyze
– Learn best practices
• Use Version Control (eg. Git and Github)
– Not just for code, but also data, papers, notes, reviews…
– Might take some time, but saves time in the long run
– Will also save you many times
• Automate, learn scripting languages
• Ditch MsWord. Today. Just do it. Learn LaTeX
• It’s not about how many tools you know, but rather
about how well you know them
26

28. Be Tidy
• Develop the habit of organizing everything
• Write code as if you wrote it for others
– Do future-yourself that favor
– Learn best practices
• Release code and data
– Don’t be afraid, we’re not professional developers
– Will force you to meet some standards
– There’s never too much documentation, even for you
– Ask someone to try and reproduce you
• When you submit a paper, organize and clean
everything as if it were already accepted
27

29. Backup
• Everything
• Everyday
• To paranoid levels
• Use synchronization tools
• Pair with version control for mental health
28

30. THE FUNDAMENTALS
29

31. Research Methods
• “Everyone is entitled to their own opinions,
but not their own facts”
• Learn the process to reach to the facts
• Learn research methods from day 1
– Not only to carry out your own research, but to assess
someone else’s
• Read from different disciplines
• There’s never too much of it
30

32. Validity and Reliability
• Validity: are we measuring what we want to?
– Conclusion: are conclusions supported by the data?
– Internal: are observed effects due to hidden factors?
– Construct: do variables reflect the concepts I want?
– External: are my results generalizable?
• Reliability: how much confidence do I have in my
results? How likely is it to obtain the same results
if I ran the study again?
31

33. 18
Not Valid
Reliable
Valid
Not Reliable
Not Valid
Not Reliable
Valid
Reliable

34. Error
• All your experiments are subject to error. Accept it.
• Systematic error or bias (validity)
– Introduced by measurement instruments, bad
experimental procedure or environment
– If identified, we can often eliminate it
• Random error or noise (reliability)
– Inherent and unpredictable fluctuations
– We can neither control nor explain it, but we can reduce it
Dep.Vars. = f(Ind.Vars.) + errorsystematic + errorrandom
33

35. Validity & Reliability in CS
• Lab experiments, typically dataset-oriented
– High internal validity
– Low construct and external validity
– More and more reliability
• Field studies, typically user-oriented
– Hard to achieve high interval validity
– High construct validity
– Not much reliability
34

36. It’s hard
• You’ll hardly ever be completely sure about the
correct way to carry out a study
• But there are many different ways of doing it
wrong
• Learn statistics, data analysis. Really.
• Get familiar with typical mistakes
• At least get familiar with terminology
• Realize that if something is published, it doesn’t
mean it’s correct
35

37. Same data, different conclusions
36https://fivethirtyeight.com/features/science-isnt-broken/

38. Intake of Vitamin D
37https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4210929/

39. Intake of Vitamin D
38https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4210929/

40. Simpson’s Paradox
• Eg. success of kidney stone treatments, 1986
39
Treatment A
(invasive)
Treatment B
(non-invasive)
78% (273/350) 83% (289/350)

41. Simpson’s Paradox
• Eg. success of kidney stone treatments, 1986
39
Treatment A Treatment B
Small Stones 93% (81/87) 87% (234/270)
Large Stones 73% (192/263) 69% (55/80)
Treatment A
(invasive)
Treatment B
(non-invasive)
78% (273/350) 83% (289/350)
When the less effective treatment (B) is applied more frequently to less severe
cases, it can appear to be a more effective treatment.

42. Inspection Paradox
• Eg. the train arrives every 10m on average
• How long are you expected to wait on average?
40

43. Inspection Paradox
• Eg. the train arrives every 10m on average
• How long are you expected to wait on average?
40
Your arrival at the platform is also random, and you’re more likely to arrive at a
longer interval because, well, it’s longer

44. Remember
• It’s not about pointing people to the flaws in their
work
• It’s about detecting and preventing them, mainly
in your own work
• It’s the kind of knowledge that those around you
will look you up for
• Use it to help
– Learn from mistakes
– Question everything
– Understand results
41

45. YOUR MESSAGE
42

46. Communication
• Your ideas are not as important as how well you
tell the world about them
• If you fail to communicate them, they don’t exist
• Presentations and face-to-face chats can make
the difference
• Often it’s not about telling results, but stories
• Communication is a two-way street
• Know who you’re speaking with/to
43

47. English
• Learn it. Period.
• It’s not fair, you’re right, but suck it up
– Most of them know it’s not fair
– Try to understand their position as well
• People will simply forget you if you can’t speak
well with them
• Some will not even try, and will ignore you
• Technical English ≠ everyday English
44

48. Presentations
• Learn the do’s and don’ts
– J. Asher, “Even a geek can speak”
• Give talks whenever you can
• Volunteer
• Practice. Practice. Practice
45

49. Writing
• Learn how to write technical documents
– W. Zinsser, “On Writing Well: The Classic Guide to
Writing Nonfiction”
• Let it rest for some days and come back to it
– Fresh new eyes
• Find out what works best for you
– Iterative or sequential
– From the abstract or the conclusion
– etc
46

50. 3 Capital Sins
• The assumption that everyone is as familiar with
the topic as you, that they know all the context
• The need to include every single detail to show
how much effort you put in your work
• The need to overcomplicate everything to show
that your work is of high quality
47

51. Read and Write
• Read. Then read some more
• Take notes of everything you read
• Write summaries for yourself
– Helps remembering
– Helps organizing your ideas
– Makes it easier to explain later
• Log everything you do and think
– Make sure you can search in it
– Keep a notebook always at hand and write nowhere
else, except the whiteboard, of course
48

52. THE OTHERS
49

53. 50http://matt.might.net/articles/phd-school-in-pictures/

54. 50http://matt.might.net/articles/phd-school-in-pictures/

55. 50http://matt.might.net/articles/phd-school-in-pictures/

56. 50http://matt.might.net/articles/phd-school-in-pictures/

57. 50http://matt.might.net/articles/phd-school-in-pictures/

58. 50http://matt.might.net/articles/phd-school-in-pictures/

59. 50http://matt.might.net/articles/phd-school-in-pictures/

60. 50http://matt.might.net/articles/phd-school-in-pictures/

61. 50http://matt.might.net/articles/phd-school-in-pictures/

62. 50http://matt.might.net/articles/phd-school-in-pictures/

63. 50http://matt.might.net/articles/phd-school-in-pictures/

64. 50http://matt.might.net/articles/phd-school-in-pictures/

65. 51

66. Them
• Research doesn’t happen in a vacuum
• Know the community, the people
• Know where to read from and where to publish
• Go to conferences, make friends
• Create a Twitter account (for work)
– You’ll know the people before going to conferences
– Be active and positive, don’t just complain
• Review papers
– Ask your advisor
52

67. Them
• Don’t lose sight of the big picture
• Don’t burst the bubble and make up problems
that don’t exist
• Show interest in the work of others
• Explore other communities and other fields
– Realize that STEM > nothing
• Reach out to industry
53

68. Them
• Person ≠ idea
• Burn no bridges
• Learn to smell the bullshit, the charlatans
• Join debates, but avoid wars. Learn from them
• Surround by people who complement you
• Learn by working with others
– That you get along well is maybe the most important
– Everyone works differently, so work around it
• We all like to be acknowledged, so acknowledge
• Talk to your fellow students
54

69. Your Advisor
• You’re on your own
• Be as self-sufficient as possible
• Listen and ping your advisor
• Always ask for feedback
– Take silence as positive feedback
– Ask for hypercritical feedback
• Be proactive in meetings
– Set agenda
– Be clear, direct
– Explain what you’ve done, where you’re going, and how
you want to get there
55

70. YOU
56

71. Procrastination
• Try to learn why you do it
– Fear of failure, fear of success
– Fear of attachment, fear of separation
• Make it work for your own advantage
– Accept that you’ll do it last minute anyway, so do those
tedious things in between
• Planning fallacy
• It’s all about when you feel the pressure
– Work in 45m time slots
– Schedule time for yourself
– Use a time tracker, the pressure of yourself
– Set deadlines with your collaborators
57

72. The Valley of Shit
• You’ve always been a good student, but suddenly
it feels like you can’t manage
• You lose perspective, confidence and belief in
yourself, and start second-guessing everything
• Everyone thinks you’ll make it, but how do they
really know?
• You’re not the best person to judge the value of
your own work, let alone while in the valley
58https://thesiswhisperer.com/2012/05/08/the-valley-of-shit/

73. The Valley of Shit
• The place is full of brown stuff, and nobody will
be with you because it stinks
• You’re alone in it, but there’s no need to be lonely
• May happen once or multiple times, for long or
short periods, but it will happen, rest assured
• If can feel endless, but it does have and end
• Just keep walking
59

74. The Pit of Despair
• Nothing seems to go right, it’s all slow, difficult
and stinks more and ore
• Where is it all going to anyway?
• The smell turns into constant depression and
anxiety for no reason
• You stop walking through the valley and start
digging. The more you dig, the deeper you are
• You don’t care for anything anymore, and are
incapable of enjoying what used to make you
happy
60

75. The Pit of Despair
• It’s time to take time off, ask for help, and get well
• Be kind to yourself, allow you to stand up
• You’ll start caring again, even if just to hate
everything, but that’s the first step to get out
• It’s ok to quit, but only because you don’t like it,
not because you think you’re not capable
• Persistence is more important that intelligence
61

76. Impostor Syndrome
• You doubt of your accomplishments and fear to
be exposed as a fraud
• If you succeed you were just lucky and don’t
deserve it
– Always put extra effort because otherwise you’ll fail
• They think you’re good because they’re fooled
– You ignore absolutely all forms of positive feedback
• You’re not the best one to judge yourself
• Maybe you’re just normal :-)
62

77. Dunning-Kruger Effect
• The incompetent believe they know it all
• The wise doubt they know anything
63
knowledge
confidence
Peak of
Mr. Stupid
Valley of
despair
“It’s complicated”

78. Dunning-Kruger Effect
• Keep learning and practicing
– The more you know about a topic, the more you’ll
recognize how much there is still to learn
• Ask others how you’re doing
– Constructive criticism
• Question what you know and what you do
– Don’t just have eyes to what confirms what you
already know
64

79. Stupidity
• So far you’ve been evaluated based on your
answers to questions, but in research nobody
knows the answers
• So why should you?
• The amount of things you don’t know is, for all
purposes, infinite
• Rather than discouraging, it should be liberating
• If you don’t feel stupid, you’re doing it wrong
• You’ll fail over and over again, be confortable with
it and take it as an opportunity to learn
65http://jcs.biologists.org/content/121/11/1771

80. I think, therefore I’m biased
• Cognitive biases are shortcuts that help us
overcome certain problems
– Too much information. Filter what is useful
– Not enough meaning. Connect and fill in the gaps
– Need to act fast. Decide and predict
– What to remember. Simplify and generalize
• They will mostly fail us and make us be wrong
• They are everywhere. Once you know about
them, you’ll see them all around
66

81. 67https://betterhumans.coach.me/cognitive-bias-cheat-sheet-55a472476b18

82. Survivorship bias
• The tendency to concentrate on the people or
things that made it past some selection process
and overlooking those that did not, typically
because of their lack of visibility
• You don’t see other people’s failures
• Quantity ≠ quality
• Success has many faces
68

83. Confirmation Bias
• The tendency to search for, interpret, focus on
and remember information in a way that confirms
one's preconceptions
69

84. Anchoring Bias
• The tendency to rely too heavily on one piece of
information, usually the first one acquired, when
making decisions
70

85. Authority Bias
• The tendency to attribute greater accuracy to the
statement of an authority figure, regardless of the
content
71https://www.youtube.com/watch?v=vGc4mg5pul4&t=1h22m

86. Hindsight Bias
• The tendency to see past events as being
predictable at the time those events happened
• Everything is obvious once you know the answer
• Should you publish it?
72

87. Self-Serving Bias
• The tendency to give yourself credit for successes
but lay the blame for failures on outside causes
73

88. Self-Handicapping
• The tendency to avoid efforts in the hope of
keeping potential failures from hurting self-
esteem, and amplify the merit of the successes
• Win-win situation
74

89. Backfire Effect
• In the face of contradictory evidence, established
beliefs do not change but actually get stronger
• If you get a paper rejected, don’t read the
reviews immediately
• Wait until the next day, and listen
75http://theoatmeal.com/comics/believe

90. You’re biased too, and that’s great
• You may easily fall for the blind spot bias
– Tendency to see oneself as less biased than others
• Don’t kid yourself. We are all biased. Accept it
• Acknowledge that there’s room for improvement
• Confirmation bias will actually help you identify
your own biases, everywhere
• Ultimately helps you understand yourself, how
you think, and how to become better at it
76

91. LIFE
77

92. It’s always there
• “That thing that happens while doing your PhD”
• Don’t take family and friends for granted
• They’ll still be there for you, but it won’t be the
same and you’ll have to catch up
• You’ll miss many things, and you’ll regret it
• Learn when you really need to keep working
– Most of the times you don’t
– Go out, have a beer
– Disconnect
78

93. It’s always there
• Don’t be afraid to talk about your work
– Be enthusiastic about it, you’re supposed to like it
– Teach whenever you can, spread knowledge
• Don’t keep a list of taboo topics
– Use them to improve your skills and help others
• Exercise
– It’s good for your health
– It’s good for your mind
– You feel accomplished
– More ≠ better. When is the key
79

94. TAKE HOME MESSAGES
80

95. • A PhD is a wonderful thing
• Remember why you’re doing it
• There will be bad phases
• It happens to everyone
• There’s no point in suffering
• Enjoy it, and grow as a human being
81

96. It’s all about You
82