How I Fail S02E02: Natalia Bielczyk (PhD’20, Neuroscience)

Dr. Natalia Bielczyk is an entrepreneur, researcher, author, and philanthropist. She graduated from the College of Inter-Faculty Individual Studies in Mathematics and Natural Sciences at the University of Warsaw, Poland, with a triple MS title in Physics, Mathematics, and Psychology. Thereafter, she obtained a PhD in Computational Neuroscience at the Donders Institute for Brain, Cognition, and Behavior in Nijmegen, the Netherlands. In 2018, she launched a public foundation, Stichting Solaris Onderzoek en Ontwikkeling, aiming to help early career researchers find new careers in industry. She also owns Welcome Solutions, a company developing new tools and practices to help professionals in navigating on the job market, and in finding/creating their dream jobs. Even though she chose to work in the open market, she is still a researcher in her free time and has a strong belief in the compatibility of science and entrepreneurship. She recently released the second edition of her book entitled “What Is out There For me? The Landscape of Post-PhD Career Tracks”.You can find out more about her on her website or on Twitter.

Hi Natalia, thanks for joining How I Fail! Next to your official bio, could you say a bit more about yourself?  

Hi Veronika, thank you so much for inviting me! 

Well, I guess the best way to start is to say that, ever since I was a kid, I wanted to be a scientist – a physicist at first, and then a neuroscientist for the last ten years. In recent years, my enthusiasm towards doing science for a living started fading away though, and for multiple reasons. I recently reviewed these reasons here. Funny enough, this text probably got more attention than all my research papers combined ever did!

To briefly summarize this process of disillusion, two important things have happened. Firstly, I understood that the rules of the game in science is not what personally suits me. To make it clear, academia is neither better nor worse than any other place; in every working environment, you have some – written or unwritten – rules of the game and you have to accept these rules or you need to go. I felt that entrepreneurship suits me better for many reasons. For instance, I like the fact that entrepreneurship promotes hard work and the sky’s the limit: the more you work, the more functional products you create, the more problems you solve and for more people, the more you will eventually earn and the more colorful people you will know. On the contrary, in academia, and many other environments it’s often the case that while you are working hard and producing ten papers as a PhD candidate and someone else doesn’t publish anything but rather, sips coffee with the boss at lunch every day, they get the postdoc contract and not you. I also highly enjoy the fact that I have the opportunity to meet the beneficiaries of my work, shake hands, and observe their progress in real-time. This is very rewarding to me – especially given that I used to do fundamental research in neuroscience and I never had that opportunity.

Secondly, the fierce love for neuroscience that I felt in my twenties, slowly faded away. I realized that I’m like an onion – I have layers, and that somewhere deep underneath, I always had yet another interest, namely, interest in people and in their decision making, their choices, and life trajectories. Now, this hidden passion came to the surface and I pursued it.

You recently published a book on careers after a PhD entitled “What is out there for me? The landscape of post-PhD career tracks”. Are there some lessons about failure that you have learned from the people you interviewed? 

Yes, indeed! While collecting materials for the book I interviewed many researchers who moved to industry (as well as a few researchers who made a journey in the opposite direction and migrated to academia to industry) – their testimonials are included in the book as well. What I learned is that the vast majority of these people don’t perceive their previous career choices – including a large part of their life spent in academia – as failures. I actually asked this specific question to every interviewed person. It turned out that only one among over twenty participants had an opinion that the decision to do a PhD was a mistake. This was a very optimistic result! Indeed, it’s better to treat our previous choices and their consequences as lessons rather than failures.

Can you tell us more about your book?

Sure. I wrote this book because over the last three years – ever since my PhD contract expired – I learned a lot about the job market for PhDs. I have a very broad academic background, as I was trained in Physics, Mathematics, Psychology, Economy, and Neuroscience, which is in itself a combination of multiple disciplines of science. I also have some personal interest in sociology, mentoring, self-development, business, and IT. Therefore, I was trying a ton of things and testing myself in multiple roles. I was coordinating a huge online mentoring program for the Organization for Human Brain Mapping, I was trading cryptocurrencies and other speculative assets (and I got completely bust but that’s a material for yet another blog post :)), I was writing white papers for blockchain projects. 

At some point, I decided to find a “normal job.” So, I started applying for jobs to corporations, smaller companies, public institutions. At some point, I realized that my knowledge and experience are not properly valued in the job market. At the same time, wherever I go to the job interview, I never really have the feeling that I’m in the right place – I feel that mentally, I’m very different from people interviewing me. I started analyzing this problem. Since no official textbook about “how the job market looks like” seemed to exist, I went “to the street” and I did a lot of field research by myself. I was talking to people in large organizations, corporations, startups, startup accelerators, consultancy companies. And I started noticing patterns as these people were reporting very common problems. For instance, pretty much every corporate person I talked to, complained that they have too many meetings which often prevents them from completing the actual tasks. After over a year of working full time on this know-how, I decided to wrap up what I knew and give this text a shape of a book. 

The main message of this book is that, 80% of the effort while looking for a job, is to get to know yourself very well – with your values, habits, strengths, and weaknesses – and find a group of people on the job market who think alike. Then, finding a fulfilling job will be only the remaining 20% of the work – learning how to draft a CV and a cover letter, and how to prepare for the interviews, is almost algorithmic. Therefore, in the main part of the book, I highlighted and characterized 8 tribes where PhD graduates typically go to, together which the perks and downsides of each one of them. I also included a lot of self-discovery exercises that will help to better discover where you mentally fit. The second edition of the book with 30 pages of extra information, has just come out on Amazon!

I also eventually found my tribe, and it turned out to be the tribe of entrepreneurs. When I’m among other company owners, I feel supported and understood. And I laugh a lot. Since I wanted to solve the problem associated with academics looking for their first jobs industry, I decided to create a company dedicated to this particular problem.

Are there any lessons or failures you can share about publishing a book in general? 

Ha, that’s a very good question! Even though I wrote one unofficial book as a kid (which was about adventures of my plush toys and it was a bit of a Sin City-style parody of Winnie the Pooh), this time I released my first official book. And this was a major learning point indeed!

Firstly, I decided from the very beginning that I would self-publish through Amazon. This was because I chose for the entrepreneurial way of living for the sake of personal freedom it offers, and after years and years in research, I couldn’t imagine handing my work to the army of editors and reviewers all over again [laugh]. I also knew that given the audience, i.e., researchers scattered across the world, this was the easiest and the most straightforward way to make the book quickly available to those who need it. Thus, I informally asked many people close to me to critically read the material before publication, and I pressed the “Publish manuscript” button on Amazon! 

So, one surprise that I got out of this, was that the whole process went relatively smooth. Since I had all the concepts I was going to cover in the book, planned out and on paper, converting this list into a full length book took me a few weeks of full time work, and I loved it! Publishing on Amazon is also very convenient and user-friendly. I was always wondering how it feels to be a book author, and now I know – it feels just great! It feels like doing something really useful; much more useful than publishing my theoretical research papers has ever been. It’s also good for a very practical reason: I often get repetitive questions from PhDs looking for their way on the job market, and now, instead of repeating the story every time, I can just point to my book where the topic is well explained. I think if you have that itchy thought in the back of your head for many years telling you that you’d like to try something, you should just try and see how that pans out.

But one thing also learned, was that it’s true what they say about sales – namely that it’s a very important part of entrepreneurship if not the most important. Without mastering this skill, you won’t get far in any area of the market. The same concerns writing books: even if you have good quality content, no one will notice your book without the proper online promotion, recommendations, and the exposition effect. And, it’s very good to promote a book without having a big publisher standing behind you, planting your book in stationary stores to expose it to customers and organizing meetings with the readers. I’m still working on mastering the skill of sales and there is still a long way for me to go! The good side of it, is that it’s actually a nice game to play: instead of staring at the charts and observing how some stocks jump up and down, you look at the charts showing the distribution of a real product that represents a real value. 

What’s a memorable failure for you? 

Where to even start! I will let myself skip the obvious, clear, binary failures such as rejections of all kinds, from paper rejections, through a ceased PhD project (yes, I had to reboot my PhD in another lab and start all over!), broken engagements, to rejections from dozens of jobs. I was thrown out from many places; I was even thrown out from MENSA Association for skipping the annual membership fee! I could write the whole book about all these formal failures. 

But I think what is more important, are these little, plain bad or a bit suboptimal, everyday choices that are not obvious failures at the first glance – but in the long run, they add up and can end up in a disaster. Life is an integrative process where not only strategic decisions when life takes a turn, but also everyday little decisions matter. It’s a position game, a bit like chess. That’s why most careers and relationships fall apart – it’s not an outcome of just one mistake but rather, a joint impact of multiple little missteps and misunderstandings on the way.

So, related to this point, I guess throughout my twenties, my biggest sin was always wishful thinking. For instance, I used to choose many subjects during my undergraduate studies solely based on the fact that my high school teachers praised me for my school test results (which I used to interpret as a “talent”), and based on the fact that I had a belief that this particular knowledge would lead to better jobs in the long run. I mean subjects such as programming, theoretical physics, or the most abstract branches of algebra. Whereas in fact, I didn’t really enjoy the process of learning these things… I would rather say it was very draining and frustrating to me. But I was telling myself, “come on, it’s going to be fine one day!” No, it won’t… If from the very beginning I had oriented myself at doing what I really enjoyed – such as writing, teaching, talking to people, building projects, researching people’s motivation – I would have been in a much better position right now, professionally. Not that it’s bad right now! It’s just that I feel that my development is now much faster than it used to be in the past, namely in the times when I was torturing myself in the name of what I thought other people expected of me. Going in the right direction for one year will bring you much further than going in the wrong direction for fifteen years.

Is there anything that you considered a success in the past, but in retrospect is a failure? Or the other way round?

Ha, another really good question! We often get stuck in the local maximum of our landscape of potential before we reach the global maximum. What I mean is, if you are really good at something at school, you might go in that direction just because no one ever told you that you are even better at something else that happens not to be a school subject. So, I was always good at maths at school (and in many other subjects, but I was guessing at that point that maths would give me the most transferable skills), so I went with that and studied maths instead of going for economics or straight for business as I probably should have. No one ever told me at school that I might be a successful company owner! In a sense, I could interpret this lost time as a sort of failure as most probably, I will never use the vast majority of the knowledge gained during my undergraduate and graduate studies in my future projects. And time is everything; time is money, time is life. 

Is there anything you regret not trying, even if you had to add it to your failure CV?

Hehe, I regret that I didn’t attend the FYRE festival 😉 I have a really weird sense of humor, and instead of being angry that my money just got bust, I would have probably had a blast watching all the chaos around me. I’m generally interested in crowd psychology (a.k.a. sociology) and I think that watching thousands of panicked millennials running around in mayhem on a deserted island would be just worth the money. 

But now seriously, I think that I didn’t spend enough time on music in my life. In fact, music is my respirator and it always has been. It pulled me out from the deepest ends, and it always gives me energy. So, what I regret not trying is that I didn’t dance more – especially when I was a teenager. I was raised in Poland which is still very judgmental towards females and female bodies, and this highly affected me when I was very young. I was so shy that I could even imagine getting onto the dance floor. Only after I went for studies, I discovered hip hop, street dance, salsa and other dance styles, and I discovered that I’m actually good at it, especially if I have autonomy and some space for improvisation on the floor. So, I regret that I lost the battle with my complexes. If not that, I might be a really good-class dancer now. Who knows, maybe I would have been a professional dancer and not a scientist today! 

I also keep on promising to myself that one day, I will go for a course of DJ-ing and learn how to make my own music but so far these plans always lose a competition with more urgent everyday matters such as running the company or releasing research papers. But my strong resolution is that in the future, as soon as the situation is stable and I can afford this time-wise, I will dance much more and go for my DJ-ing ambitions!

Can you share a success that traditionally would not be on a (regular) CV?

Hmm, I think what I succeeded at so far, is keeping good faith no matter what. Of course, there were dark times on the way, especially at the end of the undergraduate studies and at the end of graduate studies, when for a long time I wasn’t sure how to proceed further and I had to face a lot of insecurity. I had periods of depression and multiple neurotic phases when I saw the world in black colors. But, even in the periods when I felt very unwell and I looked miserable on the outside, I never lost energy to stand up in the morning and proceed with my plans and projects. Deep inside, I always had faith that I have a huge potential and a lot to offer to society, and that in the long run, I will get far and thrive in one way or another. I didn’t really need to hear this from anybody else to know that.

Is there anything that you are still failing at yourself?

Sure, many things! I definitely fail at telling people whom I value what I value them for, specifically. I somehow always assume that the fact that I choose to talk with them on a regular basis automatically means that I value them – but they just don’t know that 🙂

I also fail at making reasonable plans and I always put way too much on my plate. I made a plan for this year in January and I’m not even close to half of the pipeline while it’s already August. I’m just never good enough for my own standards… – I always have this feeling that I might have just worked harder to meet my internal deadlines! I need to preselect ideas for further execution better, most probably.

What do you think about sharing failures online? Should everybody do it, or are there caveats?

Well, this is a tricky question because this depends on who you are and in what field you work. In academia, we have a culture in which openly sharing everyday struggles, including mental health issues, is welcome and meets with a lot of peer support. But this is not true about some of the other working cultures. For instance, mind that successful entrepreneurs such as Bill Gates, or Mark Zuckerberg, never talk about their failures in present tense but rather, they always talk about failures in a form of feel-good anecdotes many years after the fact. Humblebragging, to simply put. This is because as a company owner or a CEO, you need to be a strong leader! You can’t have public doubts or mental health issues as this will sink the image of the company, your employees will stop listening to you, and all your investors and clients will run away. You know, when I set up a company, I promised myself that I would do 100 crunches every time I fail at something. And then, I grew a six-pack after three months! But if you only know me from social media, you won’t have any idea of what mayhem was really happening behind the curtains.

Also, if you work for a private company, sharing your mental health status to the public might be taken as implicit criticism of the employer and result in removing you from the company (usually under some other excuse). For private companies, image is everything and if you share to thousands of your followers that you have yet another frustrating day at the office, it won’t be taken as a positive sign at your workplace, that for sure. You just need to be much more diplomatic when you work in industry.

What about sharing successes? Do we do it too much, or not enough? 

I personally like people sharing their successes and being proud of themselves. I think we would all be better off if there was more pride and fewer complexes in this world. Of course, there is always the question “How large does a success need to be to make it valuable enough for sharing?” Like, if you cook a good soup, should you let the whole world know about it? I think it’s perfectly fine if everyone develops their own personal criteria as people value things very differently. One person’s trash is another person’s treasure 🙂 

Are there any sources of inspiration [people, books, …] who have helped you deal with failure along the way? 

Of course. I would put people over books here and say that at least 90% of my success with dealing with failure is due to the wise and strong people whom I met on my way. I’m not sure if I should start listing names here as it took me 14 pages in my PhD thesis to even list people who influenced me during the PhD not mentioning about all the time before 🙂

About the books, sometimes the simple truths that you can find in classic books such as “Awaken the Giant Within” by Tony Robbins, just work. At the end of the day, everything boils down to whether you do what you really like in life, and whether your intuition tells you that you are going in the right direction. I think some popular books are just popular for a reason and there is no reason to frown upon them just because they became a part of pop culture.

If you’ve been a reader of How I Fail last season, what are your favorite lessons from the series?

I must admit that I didn’t read every single article in the series just yet 🙂 I like the fact that you ask a different set of questions every time! I wouldn’t even imagine that one can generate so many different questions on the same topic! Maybe you should do more journalism.

I see some interesting intrasubject differences here. Apparently, everyone perceives “failure” in their own way. For instance, when talking about failures, I focused on my own bad choices while most of the interviewed guests focused on specific events (such as job, grant, or paper rejections) that made them feel like being treated unfairly. That makes me think about myself once again. I think there were times when I didn’t have this internal sense of control, but I have developed it in recent years. Now I feel that in the long run, everything depends only on me. And all the rest, like rejections from external parties, are just unimportant hiccups whose digesting is not worth my mental capacity to the smallest extent. External evaluation is always a lottery to some extent, so I treat it as such. So, when I get a rejection, I do nothing – I just take a deep breath, keep on working, and put on some Tiesto on the headphones to work faster.

Is there any way we could reimagine academia or perhaps education in general, that would have been a better fit for you, and I think many other researchers?

When I think about this now, I can’t imagine academia or any other system to reshape in a way that it would perfectly fit me as a person. I think I was destined to have my own company and the signs of that were always present in my life ever since I was a child; I just didn’t see these signs or I didn’t want to see them. What I have now is “IT” for me and I couldn’t imagine it to be any better. 

But, academia could definitely reshape to make the lives of researchers who are currently working there, better. It might be less hierarchical in a sense that early career researchers might have more autonomy to propose their own research projects without the necessity to get an approval of their direct boss but rather, of some committee representing the whole institute. Also, to release a bit of the peer pressure and the employment bottleneck in academia, reducing the number of open PhD candidate positions would probably be necessary. 20 or 30 years ago academia was a much healthier place as the disproportion between the numbers of faculty members and PhD candidates was an order of magnitude lower. Now, it’s a jungle where people use elbows a lot just because they feel they need to do that to survive. Other than that, I genuinely don’t know how to improve academia. It’s an archaic, individualistic system that has no right to function in the XXI century when society is becoming more of a cloud and when only well functioning teams survive. I think it will stay more or less as it is, and it will live on only because the tax players are forced to pay for this malfunctioning machinery.

What would a five/ten year younger you think of you now? What advice would you give to that person?

Buy some Bitcoin right now! Also, your life mission of making neuroscience great again is not as important as you think, and you don’t really need to spend 60-80 hours per week working like a maniac. Neuroscience will be doing as well without you. 

Also, pay more attention to the process and don’t fixate on the ultimate goals any more than necessary. At the end of the day, life happens now rather than starting for real once you crawl up to the level of the professorship.

***

Thanks again Natalia for joining this season of How I Fail!

How I Fail S0201: Pedro Leão (PhD’18, Microbiology)

Pedro Leão has a Ph.D. in Microbiology (2018) from UFRJ (Federal University of Rio de Janeiro). His thesis was named “Diversity of Magnetotactic organisms: New frontiers for magnetotaxis evolution.” At the moment, Pedro is an FSE fellow at the University of Groningen, working in the Groningen Biomolecular Sciences and Biotechnology Institute (GBB). His main research interest is in linking genomics data with ultrastructure characterization. You can find Pedro on Twitter (@Leao_pel), where he mostly advocates about empathy and emotional education in academia.

Hi Pedro, thanks for joining How I Fail! Next to your official bio, could you say a bit more about yourself?

Thank you for having me, Veronika! I’m a Brazilian microbiologist passionate about two things: Science and Sports. I join a research group in my second year as a Microbiology and Immunology bachelor (2010). I end up staying in the same group for the coming 9 years, where I receive my bachelor’s (2013), master (2015) and PhD (2018) titles under the supervision of professor Ulysses Lins. In my second year as a PhD candidate, I had the chance to stay for one summer at professor Arash Komeili’s lab at the University of California, Berkeley.

All my research formation was done on the same topic: magnetotactic bacteria (MTB). More precisely, I analyze the diversity (phylogenetic, ultrastructural, morphological) of this group of microorganisms. I consider myself really lucky to have been “raised” in this scientific community, which is amazingly inclusive and, in general, super friendly and open to collaborations, but especially lucky to have these two mentors along with me through this journey in academia so far.

In the middle of my stay at UC Berkeley, I received a call about my advisor’s death in Brazil. I stayed there for one more month and went back to Brazil and decided to finish my PhD 1 year earlier. Here I am two years after, as a FSE fellow doing some lecture and research work as a postdoc at the University of Groningen in the Netherlands! I’m currently under the supervision of professor Dirk-Jan Scheffers doing research in Bacillus subtilis’ cell wall and membrane structure.

What’s a memorable failure for you?

My most memorable failure is a recent one! Right after I finish my PhD, a position at the institute I had been working during my whole career was open. Not any position, a position to fill the chair left by the premature death of my supervisor – a guy I had worked with for the past 9 years! At first, it did not even cross to my mind to apply for it. Our group was in great hands, led by a young professor who had been mentored by Ulysses also.

Two weeks after the announcement of the position, some friends and professors approached me, telling me that I should apply for it. “You have all the requirements?”, “What can go wrong? It is worth the shot”.

Long story short, I applied for the position with 16 other people, and I did not even pass the first round of selection (which in Brazil is a writing exam). I failed to get the position, but that is OK. For me, the big failure was to not fulfill the expectations of my peers and from myself that I would be at least competitive in this scenario. I was crushed for a while. That impostor syndrome that we all know about in academia hit me hard. In the end, I had a PhD, was unemployed, and my self-esteem was on the ground.

Lucky, a 2-years visiting professor position in the same institute was open, and I was able to be selected for this one. After 18 months in this position, I had to start to search for a new job, but because of significant cuts in science funding in Brazil (as always), no one was hiring. At this point, I gained my confidence back and started to apply for some postdocs abroad.

Do you keep track of failures? Why/why not?

Sort of. I keep track of all applications I made, so I can check some old ones when preparing new ones. Doing this helps me in two ways: (i) I can keep track of improvements that need to be done; (ii) It allows me to travel back to the time I submitted that application. By remembering the will to get these things, the high hopes and plans that I had while applying keeps me ground it. I know that I fail to achieve those goals, but I still here! Things did not go as I wish, and that is OK, I was able to set the route and find alternatives to keep seeking my pathways towards my goals.

I still have a folder with all the 29 applications I made for postdocs while in Brazil. From those, I received 9 (31%) replies and was invited for 4 (13%) interviews. Now I see this as a huge success, but back then, it feels like a complete disaster.

Often people use the words rejection and failure interchangeably, what do you think about that?

I don’t think they are interchangeable. I believe rejection is when we don’t fulfill the standards of someone else. It could be a company, a person, an agency. Failure is something more deep and personal, as we don’t cope with our own expectations. We were unable to reach even what we set as a standard for ourselves. And as always, we are our worst critics, so a failure, in my opinion, always hit harder than rejection.

From my perspective, some rejections can be failures for you – like in my case, not getting the position, and performing way below my expectations.

Is there anything you regret not trying, even if you had to add it to your failure CV?

No. If I really want something I have to try it. I can’t live with the uncertainty of the “what if?”. To have this posture, you have to deal well with failure, because the truth is the chance to fail is enormously higher than the one for you to succeed.

It took me some time to learn this, but maybe it was one of the most valuable lessons I had: failing is fine, as long as you don’t fail to the point of no return. In other words: Crash! Just make sure you don’t crash and burn! How you build this to deal well with failure? Failing and mostly importantly, normalizing failure by not being too harsh on yourself and surrounding yourself with people that support you!

Can you share a success, that traditionally would not be on a (regular) CV?

I think a CV of failure itself. I’m really proud of all the rejected applications I had. It represents how many failures I could handle and still get to the place I’m today. Be proud of your failure is a tough call, but I think all of us should at least not be ashamed of them. They tell more about you than your successes.

Is there anything that you feel you are failing at yourself?

To be honest, yes. I’m in limbo at the moment. The thing is, I had a safe place in my advisor. He was a friend and a mentor who knew me thought all my academic career. I always thought I could do anything if I set a good plan because I would approach him to look for guidance at any point in this plain. Now that he is not here, I sometimes feel a little lost, thinking I’m aiming too high, sometimes too low. I miss the feeling that someone that knows me will keep me accountable in my ambition.

I built a level of tolerance to failure, knowing how much I could handle. This tolerance was taking into account my support system, and he was a big part of it. I need to relearn what are my limits, or build self-esteem and realize that I can do it with the people I have by my side now.

I feel I’m failing at this. I have some plans that are on hold because I don’t feel confident enough to make a move. I will learn my new limits, one fail at a time, but now I’m afraid to fail, as I hadn’t been for a while ad this is scary and frustrating. Fortunately, I still have great friends and colleagues that don’t know me as well as him but are valuable advisors to help me through this.

Are there any people who have been important to the way you deal with failure?

Yes! I have amazing friends on the field, some better established and renewed scientists, and others growing in the field side by side with me. The exchange of experiences with these people and the openness to talk about failure were really important in understanding that failure is a significant player in our career. Failure is inevitable in science, what you are going to do with it is what decides how healthy your work/life balance will be.

Everyone should have a support group! Dealing with failure on a daily basis is not something trivial. Surrounding myself with people having the same experiences that I was having helped me see that I was not alone, sometimes things go entirely unexpected, not only with me.

Are there any books, podcasts etc you would recommend on this topic?

I cannot recommend enough the book “Failure: Why Science Is So Successful” by Stuart Firestein. I think all scientists should read and reflect on it! It shows failure as an inevitable part of scientific progress, and do it in a fresh why without glorifying it. I love this book.

For me, mental toughness and emotional intelligence are key factors to deal properly with failure. A book that opens my mind to the importance of being educated on this subject was “The Mindful Athlete: Secrets to Pure Performance” by George Mumford. You can virtually interchange the sport’s scenarios for academic ones in this book.

Do you think there are any factors (field, country) etc, that influence how failure is seen?

Yes! In Brazil, it is common to see people saying they are sorry after committing a mistake. Even if what went wrong was completely out of his/her control. I think we perceive failure as always our fault. You can imagine how energy-draining a mindset like this can be for a scientist.

From one side it is good because if the problem is with you, it is easy to fix it, you can work on it. On the other hand, the problem is not always on you, and if you don’t handle this carefully, you can be trap in a self-depreciation loop. That is why it is so important to have a good PI as an advisor and a nice support group.

In the Netherlands, I think it doesn’t happen that often. People are more secure of their skills, and if they fail at something, it is how it is, they try to fine-tune some details, but if it keeps going to a dead end, they just move on to the next one. I’m about to finish my first year here, so this can be just a superficial first impression.

In the US, I believe the extremely competitive and fast-paced atmosphere in academia doesn’t give people time to properly handle failure. Once you fail, your first reaction must keep moving! Put that plan B, C, D in practice to achieve your results. In a scenario where productivity is the goal, this works perfectly, but if you want to form a good scientist, this is a disaster! Ignoring failures deprives you of learning how to deal with them, and once a big failure is inevitable, you have no experience to deal even with small ones. That is a recipe for that “crash and burn” scenario.

What about sharing failures online? Does the perception of people who do this differ? Should more people try to do it?

I understand why some people don’t do it. But my philosophy is simple on this subject. You can choose if you are going to be proud of your failures or not, but you never should be ashamed of them. They made the professional that you are as much as your successes.

Is there anything else we could all do to improve the conversations about failure?

We “baby doctors” (as some of my senior scientist friends used to call me) have a huge responsibility to normalize the conversation about failure!

The battle I picked to fight in academia is the spread of more empathy in our community. All of us are failing in something daily. More than that, all PIs were stressful postdocs with deadlines, all postdocs were insecure grad students, that once were fresh out of college curious bachelor students.

As new PhDs, we are still in contact with memories of the struggles grad students have and are starting to experience some of the responsibilities a PI has to deal with as well. Fight the speech of “Us against them (doctors/staff vs. students)” is essential for us to talk about one of the most important things that put us together: Failing!

If you’ve been a reader of How I Fail last season, what are your favorite lessons from the series?

I really enjoyed Mike Yassa’s episode (S01E18). At one point, he said, “I worry more about the ones who don’t get a sufficient number of rejections while they’re in grad school”. This is the same feeling I have.

He talks about mentoring his students to handle criticism and failure (his bulletproof and crocodile skin concepts). This resonates with my idea of knowing yourself through small failures, so you feel more comfortable each time with the feeling of failing. This comfort would allow you to aim for more challenging things because you can handle the probable failure.

What would a five year younger you think of you now? What advice would you give to that person?

My younger self, 5 years ago, would be entering the PhD program. He would never believe me if I told him everything that happens in these 5 years, and where he would be now! My plans had changed drastically, but not my goals!

I would tell him that it is ok to be afraid. It is ok not to feel prepared for some situations. The worst thing that can happen is to fail, which is not bad, and he can trust it will be people to help him pick up the lessons from these challenges, and eventually, he will feel less scared and more prepared to move forward.

Also to enjoy as much as possible the beach and the sun! They won’t have such things in Groningen! 🙂

Is there anything else you’d like to add?

I talked a little about empathy here. If you can take away just one thing from this conversation, that it be to be kind to yourself and others. Failure is part of life and an even more significant part of science. We can only normalize the conversation about failure if we understand we all have them on our past and will keep having them in our future, no matter how successful we became.

*****

Thanks Pedro for joining the series and kick-starting our second season! Enjoy the Dutch sun for now 🙂

Guest post: “Avengers for Better Science” has made me a Better Human

Last year together with Aidan Budd, Natalia Bielczyk, Stephan Heunis and Malvika Sharan we organized the Avengers for Better Science workshop. This guest post has been written by one of the participants, Cassandra van Gould-Praag, reflecting on this workshop.

Cass (@cassgvp) is a postdoctoral researcher at the University of Oxford Department of Psychiatry. She provides support for (f)MRI experimental design and analysis in the investigation of treatments for mood disorders. In this role, she has to stay up to speed with the leading edge of analytic tools, and is constantly on the lookout for tips, tricks, and techniques to make this research quicker, slicker, and more effective. This goes hand-in-hand with making the research more transparent and reproducible, and freely sharing the outputs of our labour. She is a contributor to The Turing Way and works with the Wellcome Centre for Integrative Neuroimaging Open Community Team. She is a passionate believer in accessibility and the equitable dissemination of knowledge, and spends a lot of time showing people that programming isn’t scary.

“Avengers for Better Science” has made me a Better Human

Avengers for Better Science” was unlike any academic event I have been to in my 10 year academic career. It will henceforth be my benchmark for collaborative, interdisciplinary, in-person, professional interactions, and a working demonstration of the level of compassion, empathy, understanding and genuine desire to “be better” which is necessary to create the type of research environment I want to be a part of.

I firmly believe that the best tool available to researchers for improving our understanding of the world is to increase the reproducibility of our research. Reproducibility goes hand-in-hand with increasing the diversity of the people who can attempt to reproduce our research; if the only people who can reproduce my work are people who are similar to me, then my work is not reproducible. The added bonus of improving the diversity of contributors is a larger potential reviewing pool. The more eyes which look, the more diverse the viewpoints which can be drawn on to solve problems, and the more likely they are to pick up errors or suggest improvements. This way of thinking underlies the “selfish reasons” to be mindful of inclusivity in research.

I try in my daily life to be aware of issues of inclusivity, but this is not for selfish reasons. This is because life is hard, and for some people life is extra hard, and I’m not about adding to the discomfort. You might call me a “Social Justice Warrior”, and I’d be fine with that. Our society deserves justice and I’m prepared to go into battle. 

The skillfully crafted program of talks and events at Avengers allowed me to demonstrate the value of understanding my own privilege as a white cisgender heterosexual non-disabled person. It also compounded the understanding that my own experience of the world may be very different to someone else’s. This position is supported by my empirical research on perception and conscious experience (for example exploring the experience of synaesthsia) which supports the idea that there is no reality except that which we perceive, and everyone’s perception is personal. 

Despite my pre-existing understanding, I had ample opportunity to learn at Avengers. I was challenged on my assumptions, reminded of the ethical imperative to be kind to myself if I want to do my best work, taught how to support others (and myself) at times of crisis, given some excellent productivity tips, and convinced for the first that there is a research environment which exists outside of academia that I could thrive in. I was also made aware of some ethical concerns in how we practice research, for example in the use of biased artificial intelligence to inform criminal sentencing, and ideas of situatedness when we consider who is leading the agenda on transparent and reproducible research. 

All of these lessons wildly exceed anything I learnt in institutional “Professional Development” courses. This was in no small part due to the excellent leadership demonstrated by the organisers as they all enacted the core values of community and inclusivity which they were aiming to foster within attendees. They worked tirelessly to build a safe space to explore our strengths and weaknesses, and made it abundantly clear that it was “OK” to be vulnerable and less than perfect. This is a lesson which is sorely missing in academia. They helped us to remember that we are all human, and that is an excellent thing. 

The funded travel and accommodation for the workshop meant I didn’t have to work too hard to justify attendance to my department. If I had, I may have struggled to define how “learning to be a better human” would help me do better research. I now understand that acknowledging my humanity makes it easier to accept my mistakes and those of others. This makes me far more open to constructive criticism, which in turn makes it a lot easier to ask questions and comfortably share my code and data. It also helps me to hold my beliefs lightly, which may reduce the bias I bring to analysis. 

An improved understanding around issues of inclusivity allows me to interact more effectively with our volunteer participants, design more ethical research and have a greater awareness of the ethical impact of our work and that of others. It also makes me a better colleague and teacher. I work harder to listen to my colleagues and students, and place more value in their truth. This makes the process of collaborative research (which all research is) much more efficient, effective and enjoyable. I’m also trying to to lead the culture change which is necessary for a healthy academia by taking care of myself, managing my own expectations and that of others, while openly and directly challenging behaviours which violate the rights of others. I am more productive now I understand my own limiting beliefs and am able to communicate my requirements with confidence.

Success in academic research is in part governed by “who you know”. I am therefore sincerely thankful to the organisers and attendees of Avengers for the community that we built at the event and networks which we continue to strengthen. Through open and inclusive research projects I know that it is possible to work as part of a team with shared values, and this usually makes for a pretty fun and productive project. I know that the connections I made through attending Avengers will stay with me throughout my career, and I am excited about the opportunities for collaboration this brings. This passion and curiosity is an excellent motivator for me. I look forward to the next opportunity to learn from my kind and diverse colleagues.

How I Fail S01E26: Casper Albers (PhD’03, Statistics)

How I Fail - Casper Albers | veronikach.com

Casper Albers is chair of Applied Statistics and Data Visualisation at the University of Groningen. With a background and degree in mathematical statistics, he now works at the department of Psychology. You can find out more about him on his website, or by reading his tweets.

1. Hi Casper, thanks for joining How I Fail! Next to your official bio, could you say something more to introduce yourself?

Hi Veronika, thanks for having me on your series! My main line of research involves the analysis of so-called momentary assessment data and other types of longitudinal data in psychology. With the rise of new technologies, such data have become more abundant in psychological research, yet still too little is know about the best ways to build relevant statistical models. Furthermore, I’m interested in statistical communication: how can we make sure that the main message of an advanced statistical model is conveyed correctly to non-technical end-users of the model. Furthermore, I’m a member of the University Council and thus involved in university politics.

I’ve done my undergraduate and PhD in mathematical statistics in Groningen, then did my first postdoc in bioinformatics. After spending four years as postdoc at the Open University in England, I returned to Groningen in 2009 and haven’t left since.

2. You recently wrote a column about being rejected for a prestigious grant (thanks for the shout-out!). Can you share a bit more thoughts about this?

This was an application for the so-called NWO Open Competition scheme. The goal was to obtain money for a PhD-student and postdoc on a project combining my two lines of research: momentary assessment data and communication. It was a ‘pre-proposal’, with a 500 word limit for the main text, and about half the candidates would’ve been invited to write a full proposal. I was part of the other half.

Initially, I felt frustration, of course. One reviewer mentioned a paper (published a week before the deadline) I didn’t cite, and asked why I requested a specific international visit. I did read this paper (but could not cite it due to the limit to the number of allowed citations), and the visit would be to the authors of this paper. This reviewer clearly did not read the proposal clearly. Having put in hours of work myself, that feels very frustrating.

Furthermore, all five reviewers indicated that I needed to elaborate in more detail, which to me seems impossible given the limited word count. Initially, you feel frustrated and wrongly treated. Once the frustration wears off, you realise that the reviewers reviewed multiple proposals. If all five of them claim my proposal lacked detail, then I guess others did succeed in providing more detail and did rightfully ‘win’ at the expense of me. It’s still frustrating of course, but I’ve come to terms with it. I’m now looking for alternative funding possibilities for (part of) the proposal.

3. Are there any other failures you’d like to share that were particularly memorable?

I was a postdoc at the Open University between 2005 and 2009. Near the end of my contract, we applied for a grant with me as named postdoc at a British funder, under the category ‘mathematics for business’. After the deadline, we received a letter stating that out of all proposals in this category, we were ranked first. Shortly thereafter, a second letter followed: due to a large amount of proposals in the medical categories, none of the proposals in our category would be funded. Thus, we had the best proposal, yet still received nothing.

Having participated in a funding scheme that turned out to have no money is a nice story for birthday parties, so I did get something out of it…

4. Next to grant and job rejections, are there any other things that fall under the word failure for you?

I would be hesitant to even call grant and job rejections failures. The word failure indicates that I did something wrong. Grant schemes often have a success rate of about 15%. There’s nothing wrong with belonging to the 85% – especially if you also occasionally belong to the 15%. If the system is such that the system expects you to be part of the lucky 15% every time, then the system is broken. That was also my point in my column, and I think is also yours in this series: failing is normal.

A couple of years ago, after working for this university for nearly 15 years, I got a permanent contract. Last August, I was promoted to professor. So, I’m currently in the more senior part of my career where grants are very helpful, but not essential anymore. That helps tremendously in being able to distance myself from this. When I still had job insecurity, this was much harder, of course.

Instead of failures, I’d like to call grant and job rejections major disappointments. Many other things in academia are as well: paper rejections (especially when the review’s poor or when it involves the first paper of your PhD-student), not meeting deadlines, etcetera. These disappointments are on a continuous scale. I guess how disappointing something bad happening is mainly depends on how unexpected it is and how much of an impact it has on me.

5. Are there are any caveats with sharing failures online?

With just sharing news like “I got rejected for this grant”, I don’t see any caveats. On the contrary, I think it is very good if ECRs get informed that established names also had to struggle through many failures. Having your grant proposal rejected or not being offered a job does not make you unsuitable for academia. Although there’s clear survivor bias in stories like mine. It would be interesting to also hear from people who decided to leave academia because of (too many) grant/job rejections.

But, of course, you can share too much information online. I can totally understand if someone doesn’t want to share the details of her/his rejected proposal online, out of fear of someone scooping your ideas.

6. Now that the grant rejection gave you some time to read all the How I Fail interviews :), what are your favorite take-aways from the series?

I like what Daniel Lakens wrote about mastery and performance goals a lot. I also very much liked various persons, e.g. Melanie Stefan, talking about privilege. For a straight white, male senior-researcher it is easy for me not to realise how privileged I am. It is good to read about the additional struggles that people from other backgrounds have, also in academia. This has certainly shaped the way I look at many academic processes.

7. Who should I ask as the next person I am interviewing for How I Fail?

It would be really interesting to read interviews with people who decided to leave academia because of failures, so to read about the other side of the survivor bias coin. As these people left academia, they also kind of left my ‘bubble’, so I don’t have any names for you.

8. What should I ask the next person I am interviewing for How I Fail?

Whether they have any experiences from the past that they at the time considered a success but now, in hindsight, consider a failure. As an example: massaging your data such that p < .05, making the paper publishable in some ‘good’ journal used to be regarded as a success; and we now know that p-hacking is naughty.

9. Is there anything that you feel like you are currently failing at yourself?

Time management. I always plan to do more in a week than what I can do. This is not a one-off failure as a grant/job rejection, but a source of constant frustration.

10. What do you think your past-self of N-10, N-20, N-30 etc years would think of you now?

My N-30 me would observe that I earn my money by solving puzzles on a computer all day, and playing with LEGO in the evening. I think that would make my N-30 me quite happy. My N-15 me would also be quite happy: I wanted a job in academia, and I got a job in academia.

11. What is the best piece of advice you could give to these past selves?

I’d give my past self a copy of Grays Sports Almanac 1950-2000 and could then fund myself. This works, I’ve seen it in the movies.

More seriously: I’d advise myself to spend less energy jumping through the hoops the system had set up for me, and spend more energy on just doing what I like to do and what I’m good at (fortunately, these two correlate quite well for me). This is fine.

12. What do you think about the current way success/failure are influencing academic research in the Netherlands?

It’s not healthy. Apart from my four years in the UK, I’ve been working at this university since 1996, so that’s about eighteen years. And I’m still in a system where I have to prove myself: if I don’t obtain a grant within seven years (at that time I’ll be fifty and will have literally worked half my life for this university), the university could strip me of my professor title. This puts stress on employees where it is not necessary. If the university can’t decide whether I’m worthy of the title “prof. dr. Albers” after eighteen years, then that says more about the university than about me.

My university is not special in this regard, all Dutch universities are roughly the same in this. Especially when we know that so many things go wrong with grant funding, such as the Matthew effect which has recently been proven to be very prominent at NWO, this is not a very evidence-based/scientific way to run academia. (Note that the university obviously not only selects and promotes based on grant success, and also includes a lot of sensible measures).

13. Do you have any suggestions on how we could change this? What is a concrete step that somebody reading this post could take in this direction tomorrow?

Focus on quality rather than quantity. Success in the Netherlands is mainly measured based on how many papers you have, how many citations you have, your H-index, etc. You can have one excellent paper, or a paper with a lot of societal impact yet few citations. You can also salami slice a semi-decent study into a series of boring papers. If I had to pick which of these versions of you would get a promotion, I would definitely go for the first version.

Furthermore, we should step away from all kinds of quantitative ways of measuring quality and including these ways in assessments. Goodhart’s law – When a measure becomes a target, it ceases to be a good measure – is over forty years old, but still spot on.

Criticising things is, of course, easy. I don’t have an easy solution either. Stepping away from emphasising the importance of all these quantitative, semi-objective measures and just using common sense should be a big step forward.

14. Is this something you discuss with the researchers that you mentor? Is there anything they can do too, to change things?

Yes, we discuss these things. With one of my PhD-students, for instance, we decided to combine two studies we did in one single, hopefully high impact, paper; rather than trying to get both studies published separately, which was the original idea.

But I think that early career researchers cannot do too much: they are at the bottom of the academic food chain, and for them it’s much harder to be heard, especially individually. When they join forces, such as in the Young Academy in the Netherlands, they do reach an audience.

But the main change should come from senior researchers, like me, who are in the position to demand change. With great power comes great responsibility.

How I Fail S01E25: Caroline Rowland (PhD’00, Psychology)

Caroline Rowland runs the Language Development Department at the Max Planck Institute for Psycholinguistics in Nijmegen, and is one of five directors at the ESRC LuCiD Centre. She studies how children learn language, or, more concretely: what are the mechanisms in the human brain that give us the capacity to acquire language?  She’s on Twitter as @CaroRowland.

1. Hi Caro, thanks for joining How I Fail! Can you tell us a little bit about yourself?

I’m a Brit who escaped Brexit by miraculously being offered a job in the Netherlands in 2016. Before that I worked in various universities in the UK, except for one year when I worked as a secretary in various offices around London. I didn’t get a good enough undergraduate degree to obtain funding for a PhD so I did it part-time (self-funded) while working as a Research Assistant. I then moved to the University of Liverpool for 16 years before taking a job at the MPI in Nijmegen.

2. Would you like to share some failures with us?

Oh yes, indeed. There are countless examples; failure at exams (failing S-level history at age 18 still smarts), failure at job interviews (I was rejected from teacher training college), failure with grants and journals. Reviewers have told me that my work is meaningless (“Rowland and her colleagues just count things”), that I work too closely with a small group of people (One of my major reservations is the fact that many of the investigators have already been working together very closely), and that I don’t know what I’m doing (even with significant improvements [this paper] would still not meet typical requirements for peer-reviewed quality papers).

But these kinds of comments are quite unusual. Many of the rejections I have received have been kind and constructive. Even annoying comments like “do X”, when you’ve patently done X on page 12, can be useful – they help you discover what points a hasty reader will miss.

3. What does “failure” mean to you? Are there some misconceptions about it?

Actually, I think ‘fear of failure’ is worse than actual failure. By this, I mean the constant niggling worry in the back of your mind that you’re not quite living up to expectations. For me, it’s not just a fear of failing at research. I live with a background low level worry that I’m not a good enough boss, mentor, colleague and, especially, that I’m not a good enough mother. I find this much harder to manage that actual failure. If anyone has a solution …

4. Do you keep track of failures? Why/why not?

No. Absolutely not. I try to learn from it and move on. If it’s a rejected paper, I revise it to make it better and submit it somewhere else. If it’s something more serious (e.g. that I’ve dealt badly with an issue or colleague at work) I try to apologise, learn from it and move on. I also try not to live with regrets; it’s too exhausting.

5. What do you think about sharing failures online?

Well it’s certainly not objective evidence for a recipe for success, but then it’s not meant to be. It’s narrative, and narrative can be cathartic. Misery loves company. I find it much easier to deal with the immediate aftermath of rejection when I can open a bottle of wine and spend a couple of hours reading stories about how other people have failed too.

But it’s also useful. The recent Twitter discussion we had on grant rejections prompted me to start talking to our researchers and students about how we can better support them in grant writing.

6. Have you experienced differences in attitudes towards failure throughout your career?

Of course. For example, it is much easier for me to accept my own failures now I’m older. In my twenties, I found it really hard to acknowledge being wrong. This is partly due to experience – when you’ve spent 18 years being regularly rejected, you learn to deal with it or you spend your life angry. But in my case, I also had a perfectionist PhD supervisor, so I didn’t have a choice. This was difficult to deal with at the time, particularly when a manuscript came back for the 9th time with the comment “I still don’t like this”. But it was also the best training for dealing with rejection.

7. Do you think different groups of people experience failure differently?

It’s much easier to deal with when you’re older (see 6 above). It’s probably also easier if you’re privileged. This tweet on the #ShareYourRejections thread has got me thinking about that a lot ….

8. Is there anything that you feel like you are currently failing at yourself?

Pretty much everything (see answer to 3 above). But my most spectacular current failure is my failure to learn Dutch. I *have* to learn Dutch – I’ll be living in the Netherlands for many years. And I’m a language acquisition expert, so I know that it’s going to be hard; it’s much more difficult to learn a language at my age than it would have been 20 years ago. But I am still constantly disappointed by my failure to be able to hold a conversation in Dutch. I know quite a lot of words already, but retrieving them at speed, in the right order, during a conversation, is almost beyond me.

9. If you could reach all academics with a single message, what would that message be?

Academia is hard, sometimes brutal. But most jobs are hard. So this isn’t a good reason to quit. If I ever start to feel that the problems of academia outweigh the benefits, I imagine myself in a different job; as a teacher, as a lawyer, or as a businesswoman. I imagine how I would feel every Monday morning, faced with a week of standing in front of a class, or week in an office, or a week in a courtroom. Would I feel more or less content? If the answer is ever “more content”, I will start looking round at my options.

10. If you were in a position to change how funding/hiring/etc decisions were made, what would you do?

This one’s easy. I’d allow the grant or hiring committee ample time to read the applicants’ papers, rather than relying on statistics like H indices and citation counts. This is how the Max Planck Society hires, and it’s been a wonderful experience to be given the freedom, and the time, to read what applicants have published.

11. What can this series do better to improve things for everyone?

I think it’s already doing a great job of raising awareness. It certainly helps me in my “misery loves company” moments. It’s also an opportunity to link to resources that might help people cope with the pressure of rejection (e.g. the Samaritans in the UK do a great job of helping people deal with catastrophic failure. But then I’m a psychologist – counselling is always my default option.

12. What is the best piece of advice you could give to your past self?

I’ve already tweeted it. But it’s not about being a scientist – it’s about being a working parent. I really wish I’d known this when I was younger.

How I Fail S01E24: Greg Wilson (PhD’93, Computer Science)

How I Fail - Greg Wilson | veronikach.com

Greg Wilson has worked for 35 years in both industry and academia, and is the author or editor of several books on computing and two for children. He is best known as the co-founder of Software Carpentry, a non-profit organization that teaches basic computing skills to researchers. You can find out more about him on his website.

1. Hi Greg, thanks for joining How I Fail! Can you tell us a little bit about yourself?

Hi Veronika; thanks for having me. I’m Canadian, I wrote my first program on punch cards in 1980, and my family doesn’t let me use power tools (I’m kind of clumsy).

2. What’s on your list of failures?

Oh, I have a loooong list. Off the top of my head, I have half a dozen unfinished book projects that I only call “unfinished” because I’m not honest enough to call them “abandoned”, all three of the open source software projects I started myself fizzled out years ago, and when I teach software engineering, I use stories from my first startup as examples of what not to do.

3. Was there any particular event that made you decide to join this series?

I organized a workshop earlier this year for programmers who want to learn about community organization because I think it’s pretty clear that decisions about tech are being made by people who either don’t understand it, or don’t care about the collateral damage caused by their greed. About two dozen people showed up, and there was a lot of lively discussion, but there wasn’t much follow-up from participants. I’ve been trying to figure out why not, and I’ve been drawing on my past failures to give myself a vantage point.

4. What does “failure” mean to you?

I think of “weak failure” as “I put in a lot of work and nobody got much out of it”. I think of “strong failure” as “I put in a lot of work and did harm that could have been avoided”. By that standard, my thesis was a weak failure, and things like Twitter and Facebook are strong failures. The fact that many people think they’re successful is, to me, the biggest misconception people have about what failure actually is.

5. Do you keep track of failures?

Absolutely: sometimes, late at night, I’ll put on some Johnny Cash and line my failures up side by side and count ’em one by one. Then I’ll have a look at this cartoon and remind myself that I’m an idiot sometimes and then have some hot chocolate and go to sleep.

6. What do you think about sharing failures online?

Yeah, I used to be one of those guys who said “fail early, fail often, fail loudly”. I also used to tell people that their Stack Overflow profile was their real resume, and that if they weren’t contributing to open source projects they weren’t “real” programmers, so I guess by my own definition, my younger self was a strong failure.

What I’m trying to do now with pieces like this is help people like my younger self understand why these positions are wrong by saying, “This is who I used to be/what I used to believe, and here’s why it was harmful, and here’s a better way to think/speak/act.” I also find that I can only really know what I think about something when I hear myself say it out loud, or when I write it down, so sharing my failures helps me figure out what if anything I learned from them.

7. Have you experienced differences in attitudes towards failure throughout your career?

Yes, but they’ve mostly been self-inflicted: I dwell on my failures much more often than I take pride in my successes. I don’t think that’s healthy, but then, I just ate a completely unnecessary butter tart, so…

8. How do you think the changing landscape in computer science [men/women, number of people in general] affects our relationships towards failure?

There’s a lot more discussion in tech companies and open source communities than there used to be about inclusivity, mental health, and unearned/unrecognized privilege. Clearer and more critical analysis of things like “Stack Overflow is your resume” and “if you’re not working weekends, you’re not really committed to success” is part of that.

9. What can we learn from how failures are dealt with in software engineering?

I think we can learn that people are able to rationalize and explain away almost anything 🙂

10. Is there anything that you feel like you are currently failing at yourself?

Yup. I have so many projects on the go that I’m not making much headway on any of them; I keep telling myself to set some aside, but that’s working about as well as swearing off butter tarts.

11. If you could reach any group of people with a single message, what would that message be?

I would tell everyone – everyone – to get out and vote.

12. I understand you’ve read all the previous interview of the series – can you share (i) something that was a new insight for you and (ii) something you disagreed with?

The thing I remember most clearly is Felienne Hermans’ comment that “Felienne of 2013 would not believe me, I’m sure of that”. I’m equally sure that Greg Wilson of 2010 would probably not believe how much and how little the Greg Wilson of 2018 has accomplished. I honestly can’t recall anything that I strongly disagreed with – I spent most of my time nodding and bookmarking as I read the other interviews.

13. What can this series do better to improve things for everyone?

Be more widely read 🙂

14. What is the best piece of advice you could give to your past self?

Be kind; all else is details.

How I Fail S01E23: Daniël Lakens (PhD’10, Psychology)

How I Fail - Daniel Lakens | veronikach.com

How I Fail - Daniel Lakens | veronikach.com

Daniel Lakens works at the Human-Technology Interaction group at Eindhoven University of Technology. He publishes on how to design and interpret studies, applied (meta)-statistics, and reward structures in science, in addition to his empirical research lines on conceptual thought and meaning. He teaches a Massive Open Online Course on Coursera on how to improve your statistical inferences, blogs at the 20% Statistician, and comments on science related topics on Twitter @Lakens.  

1. Hi Daniël, thanks for joining How I Fail! I wasn’t sure whether you were joking when in response to my invite you said you never failed, so I’m excited you decided to join after all.. Can you tell us a little bit about yourself?

I think whether you fail or not depends a lot on the type of goals you set – I’m also not sure if I was joking, but see my answer on question 6 for my thoughts on failing!

I got my PhD in experimental psychology, mainly studying how we think about abstract concepts (valence, morality, or power). Now I also work a lot on statistics and research methods. I live in Rotterdam (I’ve never lived anywhere else in my life) with my wife and our dog. When I’m not working, I’ll be listening to podcasts while walking the dog, going for a coffee or beer with friends, playing bass guitar, or exploring the cultural life in Rotterdam with my wife.

2. On Twitter you shared the story of searching for a PhD position for two years – could you tell us more about that experience? Who or what helped you to not give up?

After getting my Masters degree in Leiden, I had this idea of doing a ‘social service’ year. As opposed to military service, sometimes people suggest that youngsters should spend a year working somewhere in our society that would expose them to part of society they might not see otherwise. So I decided to work in elderly care for a year – mainly just cleaning and talking over a coffee. After a year, I started to apply for PhD positions. Back then, I was strangely naive about the fact that working in elderly care for a year was not seen as a positive career choice.

What worked in my favor is that my Master thesis supervisor, Dancker Daamen from Leiden University, had convinced me to present my master thesis (on the classic anchoring effect) at the Dutch Association for Social Psychology. His support and my interest in continuing to work on research with Dancker got me invited to many job talks – but people still wouldn’t hire me. I took on all sorts of jobs to pay the rent, and at the point that I started to apply with the same research groups because after 1.5 years had passed, they had a new PhD vacancy, I thought it might be time to try for other jobs. I was invited to a job talk for a PhD position at the VU university, and wasn’t hired – again.

However, Wilco van Dijk, who was in the hiring committee, had some money to hire a student assistant to program experiments. I had some programming experience, and he offered me a job for three days a week. I worked there for a year, and when I told people at the department my contract as a student assistent was ending and I was going to look for a PhD position, they apparently gotten used to having me around, and they offered me a PhD position.

This story has made me very aware that getting to where you are is largely a matter of luck. And when hiring people, I’ve realized that a messy CV happens. For example, I’ve hired two PhD students who quit their previous PhD position, which some people might consider risky – but both are doing great. Careers are often not a straight line – and that’s fine.

3. Are there any other failures you’d like to share that were particularly memorable?

Around the end of 2012, about 2 years after I completed my PhD, a researcher contacted two colleagues and myself about a paper we had written in 2009. His analyses showed that it was statistically improbable that we would have observed the results as we reported them. He was correct – we had indeed performed multiple comparisons without statistically correcting for them, and we didn’t report a study we had performed because it had not revealed a significant effect (which we immediately shared online). When we received this criticism, my statistics knowledge was not good enough to understand it, but more importantly,

I realized I did not consider myself knowledgeable enough to perform good research. I was quite disappointed in myself, and basically stopped doing empirical research for two years and tried to learn more about how to do better research. I enjoyed figuring this out so much, and I realized so many of my colleagues had similar gaps in their knowledge that I could now help them with, that I ended up switching my main research area. I still do empirical research, but most of my time I now work on how we can do better and more reliable research.

4. What do you think about sharing failures online?

Hearing from failures from people like me is obviously survivorship bias. My life would have been completely different if I hadn’t been hired as a lab assistant by Wilco van Dijk. What is now a nice story of struggle and success, would have been been a story of how I wasted two years trying to get into science, before finding a real job.

What I see online is mainly two failure stories: 1) I struggled, but I managed (like my story). Or 2) I struggled, and decided I didn’t want to work in academia.

But there is a third type of story we rarely hear: 3) I struggled, I didn’t get tenure, and now I’m not really sure where to work because academia was probably the best fit for me.

How I Fail - Daniel Lakens | veronikach.com I have two Chinese statues on my desk – on of a dragon, the other of a Buddha – that were given to me a decade ago by a colleague who was an excellent, meticulous, dedicated and cooperative scholar, but had to leave academia because he didn’t manage to get a tenured job. I keep these statues on my desk as a reminder that people who I really would like to see in science, and who really want to work in science themselves, don’t end up in science. I’m not always sure the selection filter of who ends up in science and who doesn’t is set to the right parameters, to be honest. And I don’t think we hear their stories enough.

5. Have you experienced it that people decide to downplay their successes, so as to not hurt others?

I got a VIDI grant in 2017, which is quite large (800k euro) and has all sorts of positive consequences for your career. I’m on the FWO grant committee in Belgium for psychology and pedagogy, and evaluating grants there has made me fully aware that people try really hard to rank extremely different grant proposals in a fair manner. But in the end, when the Great Excel Spreadsheet gets sorted, a good deal of random luck goes into making the final cut, or not.

I think almost all my colleagues work hard to do the best research they can do. When a single person then gets a large amount of money (and a lot of non-monetary benefits that come with this particular grant) it upsets the dynamics within a group. Both these things – the decent amount of luck you have when you get a grant, and the fact that at the end of the day, we are all doing our best – made me feel quite uncomfortable after getting this VIDI grant. I’m not sure my colleagues felt hurt by my success (we have an extremely collaborative department, and I think the group benefits when any individual has success). But I did try to make not too big a deal out of it, mainly because the amount of luck involved was substantial.

6. Next to the “traditional” failures such as rejections, are there also other types of things that we can fail at?

So this gets back to my first answer why I have a peculiar view on failure: I don’t consider rejections failures. In psychology a distinction is made between performance goals and mastery goals. Performance goals mean trying to achieve things like a number of publications, an amount of research grants, are recognition from your peers, and often involves trying to perform better than others. I have always tried to ignore this type of goal. When I started my tenure track at my current department, I received a list of goals that I needed to achieve to get tenure. I never opened that letter, but left it unopened on my desk.

Instead, I set mastery goals. I have spend a lot of time training myself to no longer compare my performance to that of others – I compare myself against my past self, or to the goals I set. And the goals I set are typically just to try my best, given the time I have. So I consider something a failure when I do something (e.g., teach a lecture) that I know I had the time for to prepare better. Or I consider it a failure when I write code that I should have checked better because I had the time for it, but I didn’t check it more carefully. Beyond that, trying our best is really all we can do. If you have tried your best, and you keep improving, you have succeeded. What happens after that is largely out of your control, so I try not to assign evaluations like success or failure to the outcomes of things.

This didn’t come naturally to me (I used to compare my performance to what others did, or what others thought was good, especially as a PhD student) but I trained to stop doing this during my post-doc. On the one hand, it has removed pressure to achieve things other people have achieved. On the other hand, it has increased pressure, because sometimes I have quite high standards for myself and I need to work hard to improve myself.

7. Is there anything that you feel like you are currently failing at yourself?

I’m failing at time management. Things are currently moving quickly in my career, I’m not doing a good job of prioritizing what I want to do and what I say no to. As a consequence, I end up spending too much time on work. I love my work, but I want to spend more time reflecting and thinking, and less time reacting and responding in my work. I had blocked 3 weeks earlier this year, one week each month, just to think about stuff – but each week ended up being filled by activities with strict deadlines. I think this is a common problem for people in my phase of their careers – but I still would like to do better.

8. When hiring somebody or reviewing a grant application, how would finding somebody’s CV of failures online influence your evaluation?

I like real things. We currently have a science where we often pretend things are better than they are. This happens in our journals, where we tell stories that do not accurately reflect the research we actually did, which creates unrealistic expectations in young scholars of what doing research looks like. A CV of ‘failures’ makes things more realistic, which I value positively.

9. If you were in a position to change how evaluation of researchers would be done [at your university or country level], what would you address?

I think we greatly undervalue collaboration. Science is a collaborative enterprise – but we do not hire team players. We hire people who have their own research lines, but we don’t hire people who help out their colleagues all the time in incredibly important ways. As a consequence, we end up with a science where everyone runs their own shop – and their is not enough collaboration. If I could change one thing, I would try to much more strongly value scientists who do not pursue their own little thing, but dedicate their time to larger collaborations.

10. Have you made any decisions that helped you to be evaluated as more successful, but that you didn’t yourself fully agree with? What would your advice to younger researchers in such situations be?

I feel quite fortunate that I made the decision to large ignore external criteria for failure and success, and focus on what I believe in instead. So I don’t have any regrets – I’m just grateful these choices worked out. My advice for younger researchers would be to find a good working environment. I’ve worked in places where I would not have had the freedom to ignore external criteria for success. At my current job, I didn’t apply for any external grants for over five years, because I didn’t feel I had any ideas that were good enough to receive funding. And, crazy enough, the head of my department was ok with this was a very long time. It was only after 5 years he told me that he believed I should now write a grant, because the stuff I was working on had potential, and my I would have a chance when applying. I initially disagreed, but he convinced me, and he was right. I feel incredibly grateful to work in an environment where this was possible. So find a place where you can work the way that you want to work, but where you are also supported and pushed at the right moment to do something. This is an incredibly tricky balance – but try to find it. It will make your work-life a lot more enjoyable.

11. How do you approach success/failure within your lab? Is it something you discuss with researchers you mentor?

I think it is really important to make students feel as comfortable as possible to admit errors. This already starts at the level of a bachelor thesis or master thesis. Things will go wrong, and when they have gone wrong, it’s ok to feel bad, but we have to transparently report them.

12. How do you think adopting open science practices affects the way failure is perceived in academia?

People have a tendency to fool themselves, and to pretend things look better than they really are. Open science will make the research we do much more transparent. Now, we show photoshopped pictures when we publish research – it’s often not close to reality. We hide studies where our predictions failed. We hide imperfections in the data of the method. Open science will show a version of our work that is more realistic. This will also reveal much more mistakes we make, in data analysis, coding, the materials we used, etc. It is impossible that everything is done at 100% perfection. We currently don’t see the errors, even when they might be important. This is something we will need to learn to accept – especially when the errors are discovered in our own work.

13. What is the best piece of advice you could give to your past self?

Don’t expect that other people know what they are doing. When I was in elementary school, I thought people in high school knew what they were doing. In high school, I thought people studying at the university knew what they were doing. While studying at the university, I thought staff members knew what they were doing. But now I’m a staff member myself, I realize we are all just stumbling along, trying to do our best, but there is so much we don’t know, it’s almost overwhelming. I’m really trying my best, but very often I have no clue what I’m doing while I am doing it.

14. What do you imagine your future self’s advice would be to the present day you?

People must almost by definition be horribly bad at this question! But let me try be slightly extrapolating from more recent thoughts I’ve had.

Pay more attention to the goals and motivations of your fellow scientists. There can be many reasons to want to work in science, and not everyone shares your viewpoints. A better understanding where other people come from makes collaboration easier and better.

How I Fail S01E22: Felienne Hermans (PhD’13, Software Engineering)

How I Fail - Felienne Hermans | veronikach.com

How I Fail - Felienne Hermans | veronikach.com
Felienne is assistant professor at Delft University of Technology, where she researches programming for everyone, from spreadsheet users to young kids. Felienne’s biggest passions in life is to share her enthusiasm for programming with others. She teaches programming in a community center in Rotterdam every week, she organizes the Joy of Coding conference, a one day developer conference in Rotterdam celebrating the joy and art of programming, and she is a host at SE radio, one of the most popular software engineering podcasts on the web.

If she is not coding, blogging or teaching, she is probably dancing Lindy Hop, running or playing a (board)game. Felienne blogs at felienne.com

1. Hi Felienne, thanks for joining How I Fail! Can you tell us a little bit about yourself?

I am Felienne, recently tenured assistant professor at TU Delft. For my PhD thesis I worked on helping spreadsheet users design better spreadsheets, with techniques to find errors and to restructure complex spreadsheets. My recent research concerns teaching programming to young children, and their teachers.

2. Was there any particular event that made you decide to join this series?

As a community, academia is very much about celebrating successes, but not that great and celebrating, or even acknowledging failures. Which is weird. Since science is so much about trying and failing and learning from it. Also I love the fact how you put so much efforts into this series so I felt I wanted to give back to you for the amazing work you do on making failures a normal topic.

V: Aww, thanks!

3. What failures have been most memorable to you in your career so far? Were there any differences between your PhD and tenure track?

Well… define failure. There are different types of failure, so I will do a few if you are ok with that.

Immediate failures: An immediate failure is a failure that you know when it happens.

My biggest one or at least the one that hurt most is when I applied for a personal grant in ‘14 that I did not get. One of the reasons, I think, is that I felt like I needed to have that one grant to succeed in academic life, and therefore the writing took lost of energy and the rejection hurt A LOT. What I learned form that is that there is always another grant and there are multiple ways to success (although sadly there seems to be a Matthew effect from this particular grant)

Failed research: When you run an experiment but it does not work out the way you want.

This is an interesting type of failure that we talk too little about I think. I always want to write great blog posts about this, but it never happens. I once ran an experiment where we needed kids with advanced programming knowledge, and a teacher volunteered his class. We drove for 2 hours to get to the school, turns out the kids were not as advanced as the teacher thought. This taught me the value of assessment!

Slow failures: The type of failure that you only know happened in hindsight.

I think if you talk about failures you talk about the immediate ones, but the slow ones are interesting too. Sometimes you are on the wrong path for a long time, but it seems like the right one! For me I think the first year of my tenure track was one big slow failure. I had no research plan other than ‘continue my thesis work’ and in hindsight I was not so interested in that topic anymore. I also had no prioritization or planning skills, so I did lots of tiny things, but since I had not big goal, they did not really add up to anything real. I felt miserable about not achieving anything and about not loving my topic. Of course the immediate failure f not getting the personal grant did not help. I only got out of this failure mode by total coincidence, because I stumbled onto a new topic (programming for kids) not because I did anything concrete to face myself from this failure.

4. Is there anything that you feel like you are currently failing at?

I would love to have more relaxed chat time with students. I do not like it that there is little time to just run into each other and chat, because to keep myself sane I need to plan my day well. I know it is the best for them ultimately, if I take good care of myself then I will be a better supervisor than when I engage on things I cannot keep up like ad hoc meetings, but I still feel bad about it because I would like to be the type of supervisor that can just do that.

Also I fail at being as good as Veronika in blogging!

5. What are your best tips for dealing with rejections, stress etc?

Stress and rejection are not the same, so I deal with them differently too.

Rejection

Rejection remains painful, but I have gotten better at dealing with it. When my personal grant got rejected for the second time (on a different topic) I told myself: ‘it is okay to just feel sad about this, no need to be strong, no need to fix this now’. That helped. What also helped was that the second time around I actually like writing the grant proposal. I had an idea, I was inspired and the background work I did was useful to me in exploring other ideas later on, and some ended up in related work sections of later papers. If you just write a proposal to get funding, getting the grant is the only thing that will make you happy. If you enjoy the process of thinking about a research direction, of finding related papers and of writing the grant, the success of the grant is less important.

Stress

Stress is different of course, just like with failure, there is immediate stress like when a grant is rejected and long lasting stress because of all the work we have to do. I learned a lot from the great post by Philip Guo why academics feel overworked? We have so many sources of work: courses, research, committees etc. I battle stress by doing rigorous time tracking and limiting my working hours to 5 days a week, max 9 hours a day, usually during regular hours. Limiting my time forces me to prioritize heavily and time tracking helps me understand where I spent the most time, and whether I am happy with that. Also I run! 20 km every week. No excuses. Health is more important than work (and also me doing this shows my students that I do not only say it but that I mean it)

6. Do you have any advice on how to approach discussing these issues with your colleagues?

Not sure I have valuable advice here… I think in these types of discussions it is important to know what you want. Take the running for example, when a collaborator suggested that I could also not run to make more time to work on a shared project, I simply said that was an outrageous proposal, as ridiculous as proposing to save time by not eating. But that is how I am. If you simply are not the type of person to do this, this advice is useless. Also things have gotten a lot easier since I got tenure. Both practically (I do not worry about not having a job) and mentally (I can tell myself I am doing fine and I also believe it now :))

7. What type of things do you wish would be given more weight in hiring and promotion decisions?

Of course it is hard to answer this question without bias! You are basically asking what parts of my job I enjoy 🙂 I like giving talks about my research to non-scientists, so that should count a lot more if you ask me.

But I also believe that, in the time we are in, it is important to keep a connection with the ‘general public’. Vaccination grades are going down because people do not believe scientists, funding decreases in many places because people think we aren’t doing valuable work. It is our responsibility to keep people informed and engaged.

A second thing I think should be given more weight is work regarding diversity. I time track so I can tell you I have spent about 50 hours last year on diversity initiatives inside and outside of the university. That is about an hour a week, ranging from giving lectures to young girls to advising colleagues. This does not help my career at all, in fact it could harm it. And the male colleagues that are already more privileged do not have to do any of this. Some do of course (thank you if you are one of those!!) but they are not expected to. Since I am one of the few women in our department, people come to me.

8. How do you think the difference in numbers of men and women in computer science affects our relationships towards failure?

It is hard to say because there are so many aspects at play here! I think that the hardest is that people in our field have an image in their heads of how female success looks like, of that the ‘story’ is for successful women. Does that make sense? Because of that they create new, alternative stories like: the woman in question was lucky, or supported by a smart supervisor or something else. The narrative that some women are just smarter is hard to swallow for some men still even though they might give lip service to diversity.
I tweeted about this a while back:

9. What about the recent rise in student numbers in CS? Are things going to be different for them?

Yes I think so, I spoke to a programmer from Silicon Valley last week and she indicated there is now a surplus of people that can program and it gets harder to find a job unless you possess specific technical skills. With so many 1st year students in our program (my program is said to have about 800 next year!) it will likely be harder in the Netherlands too.

10. What is the best piece of advice you could give to your past self?

Not sure I have something here to be honest! Given advice to your past self is weird, it gets me thinking about alternative timelines. If I give the advice, does that affect my current self? How am I writing this when all my great advice should have resulted in me retiring and living on a beach somewhere and thus not writing this piece? 🙂

And in any case, Felienne of 2013 would not believe me, I am sure of that.

How I Fail - Felienne Hermans | veronikach.com

How I Fail S01E21: Ian Goodfellow (PhD’14, Computer Science)

Ian Goodfellow is a staff research scientist on the Google Brain team, where he leads a team of researchers studying adversarial techniques in AI. He was included in MIT Technology Review’s “35 under 35” as the inventor of generative adversarial networks. He is the lead author of the MIT Press textbook Deep Learning. You can find out more about Ian on his website and on Twitter.

1. Hi Ian – thanks for joining How I Fail! I have to admit, I almost failed by being too scared to invite you, so I’m very excited you agreed. Can you tell us a little bit about yourself?

Thanks for inviting me!

I’m an AI researcher at Google and I lead a team of other researchers. We’re working to understand failures of AI better so that we can establish clear engineering principles for responsible AI development. I spend most of my own time studying how to make AI secure—for example, how to make sure that malicious attackers can’t fool AI-based systems into doing what the attacker wants instead of what the designers intended.

2. On Twitter you posted a list of rejections – can you elaborate on these a bit, for an “unofficial bio” of sorts?

My colleague Moritz Hardt tweeted to remind everyone that it was the “time of the year to keep in mind that the typical start of a successful academic career is getting rejected from a bunch of good grad schools.”

I replied with a list of grad schools and fellowship stipends that I was rejected from.

That one story probably is not much of a bio. Here is more of a bio mentioning some failures along the way:

  • As an undergrad at Stanford, I struggled in biology and chemistry classes while preparing for a career in neuroscience. I got OK grades, but I didn’t think I was doing well enough to become a professor.
  • After I changed my focus to computer science, I applied for several internships as an undergrad. Notably, Google rejected me from an internship.
  • I once applied for a summer internship with a Stanford professor. My transcript was included in my application. He replied “Why do you have an A in my class?” It turned out I wasn’t actually meant to have an A in his class. I thought there had been a generous curve, but there had only been a computer glitch. The result of my internship application was that Stanford downgraded my transcript.
  • Other large tech companies gave me internship offers, but not to work in machine learning or computer vision. On my CV, you don’t see these failures, just the eventual successes (I’m very grateful that Willow Garage and Stanford’s CURIS program gave me the chance to work on vision for robotics during summer internships)
  • In both my masters and my PhD, I spent most of my time without an outside fellowship. This meant I had to work as a teaching assistant or work on specific paying grants rather than focusing primarily on my research interests. I continually applied for fellowship’s like Quebec’s PBEEE. I spent 2009-2013 trying and failing to get open-ended funding until in 2013 I Google gave me the first PhD Fellowship in Deep Learning.
  • Vision conferences like ECCV rejected most of the papers I wrote before my PhD. I did a lot of work on perception for robots that never saw the light of day.

3. What factors do you think helped you overcome these setbacks?

In high school, I spent three years on my school’s debate team, coached by two really great teachers: Kerry Koda and Thomas King.

I’ve been surprised how many different aspects of a career in science my debate experience was able to help me with. In terms of overcoming setbacks, debate is useful because debaters all learn how to deal emotionally with failure. Every debate round has a winner and a loser. No one is so good that they always win. If you stick with debate for very long, you quickly get used to the idea of losing a round and then immediately going to another classroom and doing another round, failing to place at a tournament and then immediately going to another tournament the next weekend. You learn not to ruminate and beat yourself up. Also, your expectations get adjusted a lot. You get used to having a constant stream of both failures and successes.

4. There have been some responses to CV of Failures being a humblebrag or a sign of privilege – what would your answer to that be?

When I tweeted about this before, people didn’t react that way. A lot of people thanked me for sharing my rejections. I can definitely understand why people would see this as a humblebrag, but I think most people also understand that I’m doing this to help other people escape impostor syndrome.

5. On the opposite end, do you think that with failure being common, people might decide not to share their successes?

No, we’re all basically forced to share our successes, either for performance review at work, or in grant applications, etc. When submitting papers to conferences and journals, everyone is very incentivized to show the upsides of their paper and try to sweep the downsides under the rug. I personally fight this incentive as much as possible but I don’t imagine it going away in general anytime soon.

6. So far we are talking about successes and failures as things with discrete decisions, like positions and publications. Are there also other things that we can fail at?

I actually think that most discrete points of failure (acceptance / rejection to a specific grad school, or acceptance / rejection of a paper submitted to a conference, etc.) do not matter all that much.

I tweeted about being rejected from a lot of graduate schools, but that was fine, because I was accepted to a lot of others.

For example, in 2009, the largest obstacle for me was not that I had been rejected from some top schools, like MIT and Carnegie Mellon. The largest obstacle for me was that I wasn’t sure I would be able to do the research I wanted to at other top schools where I was accepted, like Stanford and Berkeley. It wasn’t clear who my advisor would be at either school (because there is a rotation program for new PhD students; the advisor is not assigned at the time of your admission offer), and relatively few potential advisors at these schools were supportive of deep learning research. I overcame this obstacle by going to Université de Montréal, with Yoshua Bengio locked in as my advisor ahead of time.

Probably the failure I consider the biggest is that I spent most of my PhD trying to solve supervised learning for computer vision using unsupervised feature learning methods, and was caught totally off guard when Alex, Ilya, and Geoff won the ImageNet contest with purely supervised methods. I think that in general wasting time writing papers that turn out to be dead ends is the main way that I fail in my own eyes. Especially now that it’s normal to post papers on arxiv.org, I consider my work a success if it influences other researchers, even if it gets rejected from a conference, and I consider my work a failure if it has little influence, even if it gets accepted to a conference.

7. Is there anything that you feel like you are currently failing at, or you are hesitant about in the future?

For my own point of view: I’ve been working on understanding why neural networks are easily confused by small perturbations to their input (both through my own direct research effort and working to promote interest in the topic to get other researchers to solve it) for nearly 4 years, and still no one knows how to make a model with high accuracy in this setting.

From the point of view of traditional metrics of career success: The reviews of my papers submitted to ICML this year were particularly brutal and I expect most of them to be rejected.

Another thing I think is definitely worth mentioning: The way that I work, I rapidly try out several ideas to see if they show promise, and discard most of them. On a good day when I get a lot of time at my desk, I might code up 3-5 ideas and decide that none of them work. The time investment per idea is small but I can try out a large amount of relatively different ideas. From this point of view, failure of specific ideas is just a constant part of my workflow.

8. When we talk about successful researchers, what do you think about the distribution of weights that are placed on things like publications? Are there some factors that tend to be overlooked?

I think that our metrics for success are causing society to miss out on whole categories of successful people.

For example, we spend a lot of time evaluating work and evaluating people but we don’t spend a lot of time evaluating the evaluation processes themselves. There’s no one whose job is to make sure that conference review processes are fair and accurate. We know from the NIPS experiment that there is a lot of noise in the reviewing process (Eric Price has shown that area chairs disagreed more often than they agreed about how to handle a paper in this process), and yet there is no one spearheading efforts to develop better reviewing processes backed by evidence that they actually work. The research community should value efforts that improve the effectiveness of the community as a whole, but so far we just don’t seem to have any way of putting value on such efforts.

9. Do you think machine learning as a field has a different relationship with failure than other fields? Does this affect different groups of people in different ways?

Machine learning has very high expectations in terms of very rapidly producing a lot of successful work and exerting influence over the firehose of everyone else’s rapidly produced work. For example, Ilya Sutskever has over 50,000 Google Scholar citations, while in mathematics, none of the four most recent Fields Medal winners has over 5,000 citations. It is very strange that in our field success is so explosive. This is probably in part because we use arxiv.org so much rather than being more primarily focused on peer-reviewed publications. To be honest, I don’t know a lot about how it affects different groups of people.

10. What are your thoughts on negative results in machine learning?

I think it’s hard to extract value from negative results in machine learning because it can be so hard to tell what caused the negative result. A negative result might point to something very fundamental wrong with an idea, but it might also just be the result of a very small software bug, the wrong idea of the hyperparameter values to try out, too small of a model, etc.

11. If you could reach all senior academics in ML, what would you tell them?

If I could reach all senior academics / conference organizers / journal editors in ML, I’d tell them: The community needs to have a better way of settling disputes over sharing credit for ideas.

Currently, these are mostly settled by the person who feels they have not been given appropriate credit directly contacting the author of the publication that fails to give them credit. This works if the issue was a simple oversight (author of the new publication wasn’t aware of the old publication) but usually if the two parties disagree it can turn ugly. When there’s no central authority and persuasion fails, individuals fall back to a carrot or a stick, and most people do not have much of a carrot to offer.

This kind of experience is especially stressful if a senior, famous professor demands credit from a junior researcher, such as a PhD student.

As my work has become more well known in the machine learning community, I’ve spent more and more of my time dealing with this kind of conflict.

It would be much better if a conference or journal offered a centralized place to have these conflicts adjudicated efficiently by neutral third parties.

12. What is the best piece of advice you could give to your past self?

I wish I’d used some of those GPUs I bought for deep learning to mine some bitcoin.

How I Fail S01E20: Przemysław Pawełczak (PhD’09, Wireless Communications)

Przemysław Pawełczak is a tenured assistant professor in the Embedded Software group at TU Delft. He obtained his PhD in wireless communications (topic: Cognitive Radio) at the same university in 2009. Before joining the Tenure Track of TU Delft, Dr. Pawełczak was a postdoctoral scholar at UCLA (2009-2011) and a research scientist at Fraunhofer HHI Institute in Berlin (2012). He is the recipient of 2013 NWO Veni grant. You can find him on LinkedIn, GoogleScholar and Twitter.

1. Thanks for joining How I Fail! Can you tell us a little bit more about yourself?

Great to be invited to this series! I’m an assistant professor in the Embedded Software group at TU Delft since 2013. I enjoy my work a lot. The ability to work and interact with diverse, extremely bright and open-minded people at my university makes me wake up every day and go to work. I’m Polish (and very proud of it) but spent almost all of my professional part of my life abroad (Netherlands, USA, Germany), and only at research institutions. This allows me to have a broad (not Dutch-centric) perspective on how things are handled in the academia.

2. Are there any “notable failures” you would like to share? “Most rejections”, or perhaps a failure that hurt the most?

There is no specific rejection that hurt me the most. After you got one rejection and another (and another) you start to get use to it and accept is a natural part of your trade. Of course, every rejection makes you feel bad. It is very hard to dis-associate criticism about your paper/grant application from attack ad hominem, so reading emails starting with the sentence “we regret to inform you” do feels bitter. What I realized, though, is that as you get older (or as you are longer in the business) that bitter feeling is becoming lesser and lesser. When I was a first year PhD student I could not sleep for days after my first-ever paper rejection, but now I embrace it and use it as an opportunity to improve my work and make it better (if I still can).

3. Do you keep track of your failures? Why/why not?

Yes, I do keep track of my failures. For instance, my “H2020 Rejected Proposals” folder has (as of now) eight sub-folders inside. Why do I keep track of my failures? Because it is good to see how much effort you put into papers and proposal preparations. Also tracking this lets you improve your time allocation and strategic decision-making (for instance: which proposals are worth the effort to re-submit and which are not).

4. Do you think this issue is more prevalent in academia than in other fields? What about differences between academic fields or different countries?

My core observation is that academia (anywhere in the world) is no different from other fields. Everyone competes for scarce resources (in academic terms: limited spots in conference proceedings and journal issues) and there are simply to many of us academics for available top spots. So naturally you will be rejected. Embrace it and be persistent. To say it jovially: law of large numbers (resubmit, resubmit, resubmit) will eventually help you.

5. Is failure/rejection in academia something you discuss with students that you supervise?

All the time! I am actually preparing them for rejection from the first day of their work. Statistically speaking there is a much higher chance that any work will be rejected, so it is critical to psychologically prepare students for this (quite common) event. And since I am the person that sees glass half empty (rather then half-full), projection of my defeatism makes it even easier.

6. What do you do when you receive a rejection?

I am a person that over-excites about many things (either failure or rejection), so to keep my head clean I run. I find running more of a meditative ritual, rather than a physique-improving task, I started running during my second year of my PhD and I run ever since. My PR is 48:27 for 10k, which proves I do not run for the records.

7. What about when you receive good news? Any traditions/rituals there?

See my earlier answer. Whenever I can, I go with my wife to a restaurant to celebrate. Also, I thank my students for their efforts (success is never a lonesome adventure) by inviting them to spend some time together outside the university.

8. Are there any opportunities you didn’t take that you wish you had (so you could add them to either your “success” or your “failure” CV)?

I was too late with my NWO Vidi grant proposal writing preparations, so I had just one chance to apply, which ended up in rejection. All the EU proposals I mentioned earlier that were rejected I consider to be a pure lottery. There were naturally some other (less critical) failures but overall I am happy with the choices I made.

9. Can you think of something you did that felt like a success, but you wouldn’t find in a “success” CV?

Success is built on perseverance and daily progress, so the list of mini-successes would be long… but certainly not worth putting in your CV. Would successfully submitting a paper at 5 AM in the morning after a 24 writing streak would be something worth putting in your CV? I’m doubtful.

10. If you could add a compulsory section to all CVs you receive and send, what would this section be called?

If we could measure “grit” (how persevere one is in reaching ambitious goals despite all setbacks) effectively and objectively I would be happy to see it in all CVs.

11. What is the best piece of advice you could give to your past self?

Accept that nobody (including yourself and all academics) is perfect, so embrace that things around you will not as fast and professional as you would anticipate.

Mastodon More Mastodon
%d bloggers like this: