How I Fail S01E23: Daniël Lakens (PhD’10, Psychology)

Daniel Lakens works at the Human-Technology Interaction group at Eindhoven University of Technology. He publishes on how to design and interpret studies, applied (meta)-statistics, and reward structures in science, in addition to his empirical research lines on conceptual thought and meaning. He teaches a Massive Open Online Course on Coursera on how to improve your statistical inferences, blogs at the 20% Statistician, and comments on science related topics on Twitter @Lakens.  

1. Hi Daniël, thanks for joining How I Fail! I wasn’t sure whether you were joking when in response to my invite you said you never failed, so I’m excited you decided to join after all.. Can you tell us a little bit about yourself?

I think whether you fail or not depends a lot on the type of goals you set – I’m also not sure if I was joking, but see my answer on question 6 for my thoughts on failing!

I got my PhD in experimental psychology, mainly studying how we think about abstract concepts (valence, morality, or power). Now I also work a lot on statistics and research methods. I live in Rotterdam (I’ve never lived anywhere else in my life) with my wife and our dog. When I’m not working, I’ll be listening to podcasts while walking the dog, going for a coffee or beer with friends, playing bass guitar, or exploring the cultural life in Rotterdam with my wife.

2. On Twitter you shared the story of searching for a PhD position for two years – could you tell us more about that experience? Who or what helped you to not give up?

After getting my Masters degree in Leiden, I had this idea of doing a ‘social service’ year. As opposed to military service, sometimes people suggest that youngsters should spend a year working somewhere in our society that would expose them to part of society they might not see otherwise. So I decided to work in elderly care for a year – mainly just cleaning and talking over a coffee. After a year, I started to apply for PhD positions. Back then, I was strangely naive about the fact that working in elderly care for a year was not seen as a positive career choice.

What worked in my favor is that my Master thesis supervisor, Dancker Daamen from Leiden University, had convinced me to present my master thesis (on the classic anchoring effect) at the Dutch Association for Social Psychology. His support and my interest in continuing to work on research with Dancker got me invited to many job talks – but people still wouldn’t hire me. I took on all sorts of jobs to pay the rent, and at the point that I started to apply with the same research groups because after 1.5 years had passed, they had a new PhD vacancy, I thought it might be time to try for other jobs. I was invited to a job talk for a PhD position at the VU university, and wasn’t hired – again.

However, Wilco van Dijk, who was in the hiring committee, had some money to hire a student assistant to program experiments. I had some programming experience, and he offered me a job for three days a week. I worked there for a year, and when I told people at the department my contract as a student assistent was ending and I was going to look for a PhD position, they apparently gotten used to having me around, and they offered me a PhD position.

This story has made me very aware that getting to where you are is largely a matter of luck. And when hiring people, I’ve realized that a messy CV happens. For example, I’ve hired two PhD students who quit their previous PhD position, which some people might consider risky – but both are doing great. Careers are often not a straight line – and that’s fine.

3. Are there any other failures you’d like to share that were particularly memorable?

Around the end of 2012, about 2 years after I completed my PhD, a researcher contacted two colleagues and myself about a paper we had written in 2009. His analyses showed that it was statistically improbable that we would have observed the results as we reported them. He was correct – we had indeed performed multiple comparisons without statistically correcting for them, and we didn’t report a study we had performed because it had not revealed a significant effect (which we immediately shared online). When we received this criticism, my statistics knowledge was not good enough to understand it, but more importantly,

I realized I did not consider myself knowledgeable enough to perform good research. I was quite disappointed in myself, and basically stopped doing empirical research for two years and tried to learn more about how to do better research. I enjoyed figuring this out so much, and I realized so many of my colleagues had similar gaps in their knowledge that I could now help them with, that I ended up switching my main research area. I still do empirical research, but most of my time I now work on how we can do better and more reliable research.

4. What do you think about sharing failures online?

Hearing from failures from people like me is obviously survivorship bias. My life would have been completely different if I hadn’t been hired as a lab assistant by Wilco van Dijk. What is now a nice story of struggle and success, would have been been a story of how I wasted two years trying to get into science, before finding a real job.

What I see online is mainly two failure stories: 1) I struggled, but I managed (like my story). Or 2) I struggled, and decided I didn’t want to work in academia.

But there is a third type of story we rarely hear: 3) I struggled, I didn’t get tenure, and now I’m not really sure where to work because academia was probably the best fit for me.

I have two Chinese statues on my desk – on of a dragon, the other of a Buddha – that were given to me a decade ago by a colleague who was an excellent, meticulous, dedicated and cooperative scholar, but had to leave academia because he didn’t manage to get a tenured job. I keep these statues on my desk as a reminder that people who I really would like to see in science, and who really want to work in science themselves, don’t end up in science. I’m not always sure the selection filter of who ends up in science and who doesn’t is set to the right parameters, to be honest. And I don’t think we hear their stories enough.

5. Have you experienced it that people decide to downplay their successes, so as to not hurt others?

I got a VIDI grant in 2017, which is quite large (800k euro) and has all sorts of positive consequences for your career. I’m on the FWO grant committee in Belgium for psychology and pedagogy, and evaluating grants there has made me fully aware that people try really hard to rank extremely different grant proposals in a fair manner. But in the end, when the Great Excel Spreadsheet gets sorted, a good deal of random luck goes into making the final cut, or not.

I think almost all my colleagues work hard to do the best research they can do. When a single person then gets a large amount of money (and a lot of non-monetary benefits that come with this particular grant) it upsets the dynamics within a group. Both these things – the decent amount of luck you have when you get a grant, and the fact that at the end of the day, we are all doing our best – made me feel quite uncomfortable after getting this VIDI grant. I’m not sure my colleagues felt hurt by my success (we have an extremely collaborative department, and I think the group benefits when any individual has success). But I did try to make not too big a deal out of it, mainly because the amount of luck involved was substantial.

6. Next to the “traditional” failures such as rejections, are there also other types of things that we can fail at?

So this gets back to my first answer why I have a peculiar view on failure: I don’t consider rejections failures. In psychology a distinction is made between performance goals and mastery goals. Performance goals mean trying to achieve things like a number of publications, an amount of research grants, are recognition from your peers, and often involves trying to perform better than others. I have always tried to ignore this type of goal. When I started my tenure track at my current department, I received a list of goals that I needed to achieve to get tenure. I never opened that letter, but left it unopened on my desk.

Instead, I set mastery goals. I have spend a lot of time training myself to no longer compare my performance to that of others – I compare myself against my past self, or to the goals I set. And the goals I set are typically just to try my best, given the time I have. So I consider something a failure when I do something (e.g., teach a lecture) that I know I had the time for to prepare better. Or I consider it a failure when I write code that I should have checked better because I had the time for it, but I didn’t check it more carefully. Beyond that, trying our best is really all we can do. If you have tried your best, and you keep improving, you have succeeded. What happens after that is largely out of your control, so I try not to assign evaluations like success or failure to the outcomes of things.

This didn’t come naturally to me (I used to compare my performance to what others did, or what others thought was good, especially as a PhD student) but I trained to stop doing this during my post-doc. On the one hand, it has removed pressure to achieve things other people have achieved. On the other hand, it has increased pressure, because sometimes I have quite high standards for myself and I need to work hard to improve myself.

7. Is there anything that you feel like you are currently failing at yourself?

I’m failing at time management. Things are currently moving quickly in my career, I’m not doing a good job of prioritizing what I want to do and what I say no to. As a consequence, I end up spending too much time on work. I love my work, but I want to spend more time reflecting and thinking, and less time reacting and responding in my work. I had blocked 3 weeks earlier this year, one week each month, just to think about stuff – but each week ended up being filled by activities with strict deadlines. I think this is a common problem for people in my phase of their careers – but I still would like to do better.

8. When hiring somebody or reviewing a grant application, how would finding somebody’s CV of failures online influence your evaluation?

I like real things. We currently have a science where we often pretend things are better than they are. This happens in our journals, where we tell stories that do not accurately reflect the research we actually did, which creates unrealistic expectations in young scholars of what doing research looks like. A CV of ‘failures’ makes things more realistic, which I value positively.

9. If you were in a position to change how evaluation of researchers would be done [at your university or country level], what would you address?

I think we greatly undervalue collaboration. Science is a collaborative enterprise – but we do not hire team players. We hire people who have their own research lines, but we don’t hire people who help out their colleagues all the time in incredibly important ways. As a consequence, we end up with a science where everyone runs their own shop – and their is not enough collaboration. If I could change one thing, I would try to much more strongly value scientists who do not pursue their own little thing, but dedicate their time to larger collaborations.

10. Have you made any decisions that helped you to be evaluated as more successful, but that you didn’t yourself fully agree with? What would your advice to younger researchers in such situations be?

I feel quite fortunate that I made the decision to large ignore external criteria for failure and success, and focus on what I believe in instead. So I don’t have any regrets – I’m just grateful these choices worked out. My advice for younger researchers would be to find a good working environment. I’ve worked in places where I would not have had the freedom to ignore external criteria for success. At my current job, I didn’t apply for any external grants for over five years, because I didn’t feel I had any ideas that were good enough to receive funding. And, crazy enough, the head of my department was ok with this was a very long time. It was only after 5 years he told me that he believed I should now write a grant, because the stuff I was working on had potential, and my I would have a chance when applying. I initially disagreed, but he convinced me, and he was right. I feel incredibly grateful to work in an environment where this was possible. So find a place where you can work the way that you want to work, but where you are also supported and pushed at the right moment to do something. This is an incredibly tricky balance – but try to find it. It will make your work-life a lot more enjoyable.

11. How do you approach success/failure within your lab? Is it something you discuss with researchers you mentor?

I think it is really important to make students feel as comfortable as possible to admit errors. This already starts at the level of a bachelor thesis or master thesis. Things will go wrong, and when they have gone wrong, it’s ok to feel bad, but we have to transparently report them.

12. How do you think adopting open science practices affects the way failure is perceived in academia?

People have a tendency to fool themselves, and to pretend things look better than they really are. Open science will make the research we do much more transparent. Now, we show photoshopped pictures when we publish research – it’s often not close to reality. We hide studies where our predictions failed. We hide imperfections in the data of the method. Open science will show a version of our work that is more realistic. This will also reveal much more mistakes we make, in data analysis, coding, the materials we used, etc. It is impossible that everything is done at 100% perfection. We currently don’t see the errors, even when they might be important. This is something we will need to learn to accept – especially when the errors are discovered in our own work.

13. What is the best piece of advice you could give to your past self?

Don’t expect that other people know what they are doing. When I was in elementary school, I thought people in high school knew what they were doing. In high school, I thought people studying at the university knew what they were doing. While studying at the university, I thought staff members knew what they were doing. But now I’m a staff member myself, I realize we are all just stumbling along, trying to do our best, but there is so much we don’t know, it’s almost overwhelming. I’m really trying my best, but very often I have no clue what I’m doing while I am doing it.

14. What do you imagine your future self’s advice would be to the present day you?

People must almost by definition be horribly bad at this question! But let me try be slightly extrapolating from more recent thoughts I’ve had.

Pay more attention to the goals and motivations of your fellow scientists. There can be many reasons to want to work in science, and not everyone shares your viewpoints. A better understanding where other people come from makes collaboration easier and better.