Not-so-supervised learning of academics

Picking up where I left off with blogging last year – this is part two of a write-up of a talk I’ve given a few times last year, part one is here. After talking about algorithms which deal with data that is not fully labeled, in this part I discuss how career choices can be similar, with my own as an example.

PhD 

Doing a PhD was not on my radar until my MSc supervisor suggested that I apply for a position in the group. I liked the group and doing research for 4 years seemed like a good job to me (see my post on being an employee during your PhD). I didn’t have any specific long-term goal and, as I now realize, was clueless about most aspects of academia. 

What I did understand is that it was good to publish papers. I had a few interesting (though not spectacular) results fairly early on, so I wrote papers and sent them to various workshops. I enjoyed these workshops a lot – since there were not that many people, I could meet researchers I’d just been citing, and have good discussions. On the other hand, I spent quite a lot of time writing smaller papers and pushing away the fact that I needed journal publications to graduate. Also, as I discovered later, my grant reviewers have never heard of these workshops, and thus were not impressed with my publication record. 

I also did a lot of service and outreach activities. I had already been doing this type of thing as a student, so I was good at it, I enjoyed helping others, and it was good for my CV, whether I’d stay in academia or not. So I spent time organizing workshops, reviewing papers, giving talks to encourage more girls into science. I did learn something from all of these activities but in retrospect I think I spent a disproportional amount of time on them. 

Postdoc

I doubted a lot before deciding to go for a postdoc. The awareness of the struggle of finding a position after, and all the people telling me I really have to go abroad to have any shot at it, didn’t help. In the end by talking to more mentors, I decided to go give it a try – without leaving the country. 

My plan was to only do one postdoc and then get an independent position – or leave academia.  As I understood to achieve an independent position I needed to do three things: publish on the project I was hired on, develop my own line of research, and get my own funding. I was not prepared to deal with so many different objectives, so in the end, I did all the things poorly. On top of that, I failed to take care of myself, and had to take a few months off to recover.

It was during a particular low point during the postdoc that I started blogging and tweeting more. It started with me publishing my CV of Failures – I thought I would be documenting a story that would end with me leaving academia. The response was overwhelmingly positive, and I continued with the How I Fail series. During all of this I found an incredibly supportive Twitter community, with many others who were going through similar struggles, and it’s been helpful ever since. 

Tenure Track

Much to my surprise (and other feelings), I did find myself in a tenure track position after all.  This is an important accomplishment, but at the same time, the next goal – getting tenure – is coming up in a few years. Again, there is this (self-imposed?) pressure to do all the things, so it is not without challenges.  But, it is a much better experience in several aspects, because I occasionally realize that I don’t have to do all the things all the time. :

  • I occasionally realize that I don’t have to do all the things all the time. I’ve now actually been able to have periods on time focusing on writing, then focusing on teaching etc. 
  • I occasionally (not often enough) realize that I don’t have to repeat the career paths on others to “succeed”. The combination of things that I do, might just be “good enough”, even though it doesn’t fit the typical “successful” CV. 
  • I have a lot of people, online and offline, who share or have shared many of the same experiences, and who have advice, or are just up for having a coffee or a beer when things are tough.

Academia as supervised learning 

Where does the not-so-supervised learning come in? It seems to me that a lot of advice of what we need to do to “succeed” is based on rules derived from previous “successful” CVs – publishing at particular venues, doing a postdoc abroad, etc. Some of these rules we are explicitly told as advice, others we assume ourselves.

But there is a lot of missing from this picture. The “success” label is a function of much more than particular activities, but also the state of the world (number of tenure track positions, number of students, etc), and the state you are in yourself (including anything else you have to deal with next to the job search). These features have not been taken into account when creating the rules. So even if you do follow all the rules you might get a disappointing outcome, and vice versa.

There might also be opportunities that didn’t exist before. For example, few full professors would have been using Twitter during their PhDs and postdocs. As a yet “unlabeled” activity, it probably wouldn’t come up in any rules, but it can be a powerful tool for early career researchers.

Last but not least, it’s important to remember there’s more than one success metric, and why I’ve been writing “success” in a CV sense. Ultimately success should probably involve being happy, which can be achieved through other types of jobs. And perhaps some of these jobs are not even in our dataset yet.

CV of Failures vs Shadow CV

If you’ve been following me for a while, you know that the “CV of Failures” or “Shadow CV” are a recurrent theme on my blog and on my Twitter timeline. In this post I discuss why I think the two concepts are actually quite different, and why this difference is important. 

CV of Failures

The CV of Failures, originally proposed by Dr. Melanie Stefan, is mostly that – a list of things that didn’t work out. Most often I see failures interpreted as “things I tried to do but didn’t succeed”. This category includes rejections of jobs, grants and papers. Although these failures are hard, I think they are not very personal because they depend on both everybody else who applies, as well as everybody who evaluates you. 

Much less common is to include things that are more personal – something you just didn’t do (but should have). It’s often not your fault, because of how academia is structured – but in retrospect, you would have done these differently. This category includes focusing on the quantity over quality, not taking opportunities out of fear and being a bad mentor to others. Even more personal, it’s neglecting your health or people around you – although I haven’t seen many examples of people sharing this. 

Shadow CV

What is a shadow CV then? To me it is larger than the CV of Failures. While the CV of Failures focuses on things you have done (or didn’t do), there are many more things that influence where your CV or CV of Failures are today. It’s all the additional challenges faced by one person, and all the privilege enjoyed by another. 

There are efforts to take parts of the shadow CV when evaluating people. For example, in the Netherlands time off due to parental leave or illness can be listed on a grant application. But this is limited in scope and does not include, for example, chronic illness, financial insecurity or family problems. Even if you are lucky to work in a place where people are supportive – and I have been – these things are invisible to somebody deciding whether you belong to the top 10% of researchers who deserve funding.  

But the shadow CV is not only the challenges. It is also all the things you are proud of but that are not on any CV, like finishing a paper despite having a difficult year or a thank you email from somebody who’s read it. Regardless of what your reviewers say, don’t forget that these are the true successes.

 

My tenure track job search

This topic has come up several times on Twitter, in particular with #TotalToTT. I always participate in these conversations because I remember finding it so important to read stories of others when I was applying for a job. This is a more detailed summary of what I usually try to say regarding my search for a tenure track position. Note that since I was applying in Europe, positions are advertised continuously, so there is not really a “job season” like in the US.

Summary

I applied to a total of four jobs, interviewed for three, and got offered one – the position I have now. This sounds easy especially if you see the stories of people applying to 50, sometimes more than 100 positions.

It didn’t feel easy at the time. I already had problems with my mental health and spent some time on sick leave because of this. At the same time, I was also (unsuccessfully) applying for grants. Although I was not too worried about job security in general, academia already felt as an important part of my identity and I dreaded leaving.

Applying

The first tenure track job I applied for was the Delft Technology fellowship, a fellowship only for women I would compete with other women from all disciplines. I realized this application was a long shot, but since the fellowship only was given every two years, I thought I had to try. I discussed my applications with several full professors in Delft, who encouraged my to apply. But long story short, I quickly got rejected.

I then applied to two jobs in the UK. Although in my field there are quite a few jobs, I was quite selective with where I tried to apply. For example, I chose only universities that were neither too low nor too high on university rankings, only cities where I could see myself living, and only groups where I was getting a good impression about the lab culture. For both jobs, I emailed ahead to ask if it made sense for me to apply, since I wasn’t sure about the fit of my research. The responses were enthusiastic! These lead to informal Skype calls, and then invitations to interviews. This was so important for my self esteem since I felt like I was on the right track.

Interviews

I did a lot of research on each place before the interview – next to general “how to interview for academic positions in the UK” advice, I researched what other people in the lab did, looked at course syllabi and even read the strategic visions of the universities. What I felt was helpful for me, was to write down some answers to questions I was expecting. I felt that overall the interviews went well, however, since Brexit happened, I was myself unsure about willing to move to the UK. I wasn’t offered either job, but I received good feedback so overall I was happy with the process.

My deadline to start applying for non-academic positions was getting closer, when I somewhat by a combination of lucky circumstances heard about my current job. To prepare I did similar things as for the other interviews, but since the position was in the Netherlands I felt like I had an advantage. A day before my deadline I got the phone call that I had the job!

Verdict

I felt excited and relieved, but also scared and guilty about getting the offer. It’s strange for me to think that this is already almost two years ago, because I still do largely feel the same way. I am aware that luck and privilege played a big part in this process.

On the other hand, I do think that the way I prepared my CV and contacted the groups in advance were also helpful. In the end, perhaps having only a few applications was an advantage. I do think this is individual and will vary a lot per field and country. If I could give any advice, I would still encourage people to apply, but talk to others more about how much time and energy you should invest.

Reverse engineering your CV

I recently discovered a great podcast called The Startup Scientist, of which I have now swallowed up all episodes – thanks to Daniel Lakens for the recommendation! The podcast is about treating your academic career as a start-up. I have made this comparison before when giving talks at career orientations but wasn’t able to distill this idea, so I was excited to discover that Dan Quintana did exactly that, so now I can add on to it in this post.

The episode I want to write about is Reverse Engineering your Career Goals. Although I do often say that my career so far is a series of coincidences, I do think this type of reverse engineering has played a part in it.

Towards the second half of my PhD I had a lot of doubts about continuing in academia – I really wanted to, but I was aware that it would be very difficult to get a position. I wanted to do a postdoc, but only if I had a reasonable chance of getting a position afterwards. I was not sure how to estimate this chance, so I used the approach below.

 

1. Find representative data

I first studied the CVs of Dutch professors I knew, and concluded that most of them received independent funding a few years after their PhD. But these professors were already professors for a few years, so I decided their situations did not apply to me.

Through the funder’s website, I found a list of people in related disciplines who received this type of funding that year. Since I was a year away from finishing my PhD, and these people got their PhDs maximum 3 years ago, the gap between me and them narrowed.

 

2. Generalize

After searching for CVs of my “prototypes”, as a good machine learner I tried to find patterns. Although the people were all quite different, and I only had a few examples (I wasn’t doing this automatically, although that would have been an awesome side project), it was easy to spot several things they had in common.

I did this during my pre-Evernote era so sadly I do not have notes on what exactly I discovered, but I do remember two things in particular:

  • All recipients had an impressive (at least for me) h-index. The minimum I found was 6, but the values between 9 and 12 or so were more common. This will vary per field, but for context, the professors I thought were not representative, would be in the 20+ category.
  • Many recipients had an extra “thing” that was different from most others, like their own company, an organization they volunteered for, etc.

3. Multiple time points

Next to looking at people’s websites, it’s also helpful to search for “Name CV filetype:pdf”. Using this method, I was also able to find CVs of the same people, but from a few years ago. This had several benefits.

While the current CVs looked quite intimidating to me, the “time travel” CVs were much more relatable. Instead of learning only about the “value” of a CV, I was now learning about the “gradient”, which would be easier to apply to my own situation.

Of course, it is possible to do this for any CV, by removing all the things from the recent years. But the nice thing about the real “time travel” CVs was that I also saw how people changed the way they presented the same thing, that both CVs already had. For example, a side project that might have been insignificant in the “time travel” CV, was described in more detail in the current CV.

4. Predict?

Now that you’ve estimated your function where the inputs are the CVs at different time points and the output is receiving funding, you could try to fill in what’s missing from your future CV, to get the same output.

Don’t forget that there are multiple solutions (so different inputs will lead to the same output) and noise (so the same input might lead to a different output). Since I’m on a roll with this analogy – there are also lots of other inputs that you might not even be considering yet (life and stuff).

I did not actually create any concrete plans of what I was going to do, based on this informal study. But I suppose that the information got deposited somewhere, and helped me make choices that would later point me in the direction of a solution.

CV of Failure: Things I didn’t dare to try

Image by https://unsplash.com/@tersh4u

What counts as failure?

A recent #withAPhD conversation on Twitter prompted me to write a bit more about my CV of failures.

So far, I have been tracking the “quantifiable” failures, such as paper or grant rejections in my CV of failures, or shadow CV. However, there are a lot of other things that contribute to my experience of failure (and learning to deal with it) which are more difficult to quantify – because I have not tried them at all.

Most of these things can be summarized with the words “impostor syndrome.” I was convinced I would fail, or perhaps even worse, I was convinced “they” would laugh at me for even trying. But time and time again, evidence showed that I probably did have a chance. And even if I had failed, “they” would have thought it was good that I tried. It probably would have been better than always regretting not trying in the first place. So, what do I regret that I didn’t try?

Things I didn’t try

Although originally I thought only about purely academic things, I realized this pattern of not trying goes back much further. A few examples:

  • Talent show in high school. I’ve played piano for 7 years, but don’t consider it “talent show” quality, so I don’t sign up. At the talent show, somebody else is playing piano, but with many hiccups along the way.
  • First year of computer science at university. The student organization has sign up lists for different committees. The committee to organize parties seems fun, but I’m afraid it will have too many people, and I won’t be chosen, so I don’t sign up. Later I become friends with several guys who did join the committee, and realize they would have loved to have me.
  • Internships during my BSc and MSc. Several people are going abroad for internships. I’m afraid to get delayed with getting my diplomas. The projects I do in the Netherlands are all great and I get my diplomas on time, but the gap between the international experience I have, and the international experience that I could have, starts widening.
  • I’m writing papers during my PhD. The most competitive conferences such as NIPS seem to be way out of my league. I submit to good, but less competitive conferences and workshops. My first three papers all get accepted and I have a great time at the conferences. Later, I get evaluated on the quality of my publications – the places I’ve published do not really “count”. I read more papers from NIPS, and realize that maybe, I could have published there, too.
  • I’m finishing my PhD, and read all the regulations for graduating. A part of the regulations describe the requirements for cum laude. This involves a recommendation from the PhD advisors and several external reviewers. I ponder about asking my advisors, but decide against it. After all, even a graduate from our department who wrote several highly cited papers, didn’t graduate cum laude. “They” would find it ridiculous that I even brought the subject up. My defense is a success and the committee members are very positive. Later I confess to one of my advisors about my doubts, and he reassures me it would have made sense to at least try.

I learn from these events by recognizing the patterns and doing things differently the next time around. I did go on to participate in many committees, and even lead the student organization. The internship abroad was possible during my PhD. Although I still haven’t tried submitting to NIPS, I am now getting rejected often (and occasionally accepted) at MICCAI. I approach senior academics and ask for recommendations for fellowships and jobs. I am still scared every time, but past experiences tell me that it’s really much better if I try, than if I don’t.

Or, to quote Susan Jeffers*,

You’re not a failure if you don’t make it. You’re a success because you tried.

* Author of Feel the Fear… and Do it Anyway. The title alone is great advice.

CV of Failure: introduction

Image by https://unsplash.com/@tersh4u

My CV of Failure

Here it is – my CV of failure, or “shadow CV”.

I first found out about the concept of a CV of failure from this article. After a professor from Princeton posted his CV of failures online, shadow CVs have been getting more attention on Twitter, under the hashtags #ShadowCV and #CVofFailures. And it’s getting very popular too – the same professor now added a “meta-failure” of his shadow CV getting more attention than his research.

I already wrote about various successes and disappointments during my PhD (during my 3rd and 4th years). To write those posts, I used an Excel sheet that I normally use for yearly evaluations. Here is a screenshot from my 3rd year as a PhD student:

excel_progress

The one thing you can probably guess is that green is something that was successful, and red is something that was not. Creating a shadow CV would be essentially compiling all the red parts, over the five years that I’ve been doing research. In the era of tracking everything from what you ate to what music you listened to, why not track failures as well?

The experience

Given that I already had all the data, compiling the CV was quite easy. It was exciting – I was curious whether my shadow CV would be longer than a professor’s. It was comforting – the list wasn’t too long after all, and the inevitability of the list expanding in the near future didn’t seem as daunting. I also realized that it was good to start failing early with travel scholarships, because I feel more prepared now for the larger failures that I encounter.

But most importantly, compiling the CV was motivating. I thought about whether anything would have been different for me if I had seen such CVs a couple of years ago. As many other PhD students, I was not very confident. There were many things I didn’t even dare to apply for. Sometimes senior researchers would tell me these thoughts are unfounded, that I should just apply, and that everybody gets rejected. Sometimes I listened, and sometimes got rejected, but sometimes got accepted, which ultimately gave me more confidence. I hope that seeing shadow CVs can help other students do the same: go for more opportunities, fail, and learn from it.

Year in review: final year as a PhD student

This post is a summary of 2014, the last year of my PhD. I am writing it a whole year later due to my difficult relationship with blogging. There are two reasons for this: a recent conversation about blogging on Twitter, with this result, and the fact that the summary of my third PhD year played an important role in me deciding to resurrect this blog.

As 2013 was a year of submitting papers, I expected that 2014 would be a year of paper resubmissions. That guess was quite accurate. But 2014 had more challenges in store for me. The year didn’t start out great for me for personal reasons. I am not sure I will ever discuss the details online, so let’s just leave it at “life changing event”. Up until that point, I was sure I would finish my PhD on time. But, with so many things changing so rapidly, I started having serious doubts about my progress.

Writing and staying motivated

Despite the personal chaos, I continued to work on the revisions of my rejected papers. In February, I resubmitted Paper 1. That was tough, so I didn’t want to touch the other rejected papers for a while. Besides, I had other activities lined up, such as a research visit to Copenhagen, where I wrote a conference paper about the work I had done the year before. The visit was a great experience, both professionally and personally! Unfortunately, I received a rejection, adding yet another thing to the revise-resubmit list. On top of that, I was rejected for the Anita Borg scholarship for the third and final time. But there was also a bright side: for example, around the same time I gave my first invited talks, which was a much-needed boost for my confidence.

In June, I finally received the coveted “We would be happy to publish your manuscript” email about Paper 1. This gave me the needed motivation to continue with the other revisions. In July, I resubmitted Paper 2, and in September, Paper 3, which by then had already been rejected at two different journals. Again, it was very helpful to be involved in other activities, such as organizing a workshop and teaching, to stay motivated.

With one accepted journal paper and two others under review, I again started hoping that I would submit my thesis by the end of the year. The thesis requires at least four chapters, each based on a “publishable” paper. My supervisors agreed, so I spent the last months working on Paper 4. Paper 4 described recent results, and was therefore very refreshing in the midst of all the revising. I finished it on time and submitted it to a conference in December. And then, with three papers “in limbo”, both 2014 and my PhD contract, ended.

Take-aways

My year of revisions had a few successes and several disappointments. However, the more important successes were the things that these experiences taught me. I…

  • …became a seasoned reviser-and-resubmitter
  • …learnt how to stay confident as a researcher despite a lot of disappointments
  • …realized even more deeply how important it is to have colleagues who believe in you, who support you, and who are up for a grabbing a beer (or a Spa rood), whether it is to celebrate or offer a shoulder to cry on.

Year in review: third year as a PhD student

As I mentioned before, it’s important to keep track of your successes and disapointments. Since I do have a list of sorts, I decided to share my summary of 2013 here.

Writing

2013 was definitely a year of journal papers. Or at least, of long overdue journal paper submissions. Here are the totals! I submitted four times in total (one paper twice, and two papers once). Two of these were rejections, one “revise and resubmit” and one still under review. So, 2014 probably will be a year of journal paper resubmissions.

Reviewing

Next to paper writing, there was also paper reviewing. In the beginning of last year, I was getting worried that I was not invited for reviews, but this worry turned out to be unfounded. I guess this goes together with submitting journal papers (and getting into the system) and meeting more people, who have more reviews than you, but are also more busy. I want to believe in review-karma: by writing good reviews, I hope to get good reviews. By good, I mean objective and constructive, not necessarily an “accept”.

Funding

2013 was also a year in which I tried to apply for scholarships to finance my conference visits and the trip to Tuebingen. For the second time (the first time being in 2012), I did not get the Anita Borg scholarship. I did get the ACM-W / Microsoft Research grant to go to a conference in China, which was awesome! The application that I spent quite a lot of time on, for the short-term fellowship from EMBO to go to Tuebingen, unfortunately got rejected (after I returned from Tuebingen already). However, I was able to get some financial support through my university, which was not a competitive application, but very helpful.

Research visit

And of course, 2013 was the year I went on a research visit, for which I have not (yet?) been able to write an overview. In short, the three months went by really fast and I had a great time. What everybody says about research visits is true. It is really helpful to experience a different place and get an idea of how people do research there. I think it’s a must for all PhD students, especially from smaller labs. It probably doesn’t even need to be a lab in a different country to get an impression of “how things are done” and to pick up useful research skills. I already have my next short visit planned, what about you? Did you / will you do a research visit during your PhD?

%d bloggers like this: