Defending propositions: an index for reviewers

This is the third post in the propositions series. If you don’t know what I mean by “defending propositions”, you can read the introduction here.

The proposition

I recently wrote about reviewing papers for the first time, and how happy I was when I recently discovered Publons. Publons lets you track your reviewing, essentially making visible all the effort you put into it. Here is an example of a statistic I’ve extracted from my profile:

Example statistic from publons.com
Example statistic from publons.com

This idea resonated with me because of a proposition I defended a year before, and that I will discuss today.

Introducing an r-index for reviewers as a counterpart of the h-index for authors would lead to more, better and timelier reviews.

Supply vs demand

During my PhD I noticed that the supply vs demand for publications was very different than that for reviews. For publications, supply was clearly higher than the demand: many papers would get rejected, and even if accepted, some papers would never be cited. For reviews, however, the story seemed to be the opposite! It could take months to receive reviews for your paper. And when you would finally get the reviews, regardless of the decision, a few would be very short and uninformative.

I wondered whether this discrepancy could be a result of the reward system. A new publication is rewarded by your name on the paper, by citations, and perhaps even increased chances of getting a position. A new review is – to the outside world – at most a line on your CV. Reviewing is something that is expected to be there, but I doubt it has ever been a deciding factor in a hiring decision. Combine that with well-meant advice from colleagues, “say no to everything, just focus on your publications!”, and you’ve got a supply vs demand problem.

Metrics

The h-index is meant to measure the productivity of a researcher. Although it has flaws, it is being used — consciously or perhaps subconsciously — to evaluate quality. As a result, it is something researchers can try to optimize for. And for a higher h-index, you need start with getting more publications. Do you already see where the supply is coming from?

What if there was something similar for reviewing papers? If such a – yet to be defined – reviewer index became an important metric, more people would want to review papers and submit their reviews on time, just as they do with papers.

Quality is a bit more difficult. One way to do this is a system where reviews could be seen and discussed by others. This doesn’t mean that the reviewers’ identity needs to be revealed. See for example the reviewing process of ICLR.

Or maybe we can step away from reviews altogether, as Ludmila Kuncheva proposes
here. (The post also features one of my favorite stories about publishing before the internet). Reviews would simply be replaced by citations. Then maybe we wouldn’t need new metrics, and could just go back to looking at the h-index instead.

5 easy ways to do more with your poster

How can you make your poster stand out at the poster session, if you only have a limited amount of time? There are many tips out there on how to design the poster and how to structure all the information (a couple of resources I like are here and here), and these are the most important things you should be doing! But if you already have a poster ready and can’t afford doing a complete overhaul, here are three ways to make your poster more memorable:

1. Print your poster on fabric

Print your poster on fabric if you don’t already do so. This not only makes travel easier, but also makes it easier for people to approach you and to start a conversation, and that’s what poster sessions are all about.

2. Add a picture of yourself to the poster

Chances are that your poster will be up for a longer period of time than you will actually be there to present. If people see your picture on the poster and then see you later at a conference, there are again more likely to approach you if they have any questions.

Add a picture of yourself next to the contact information on your poster
Add a picture of yourself next to the contact information on your poster. See the full poster here

 

3. Add business cards next to the poster

These days most people have a smartphone and can take a picture of your contact information. But if you do this regularly, you end up with lots of pictures that you might forget to follow up with. Make it easier for people to remember and contact you by giving them an opportunity to take your card! If you do not have cards, you can consider adding print-outs of the poster (or even the full paper) instead.

Add business cards to your poster for easy exchange of contact information
Add business cards to your poster for easy exchange of contact information

 

4. Add a QR code that leads to your website

This is especially helpful if your website has a URL that is difficult to remember. There are many QR code generators out there, such as this one. Remember to save the image in a large size and resize it afterwards for best quality during printing.

5. Promote your poster on social media

If you have Twitter, take a picture of yourself next to the poster and tweet about it using the hashtag of the conference, and include the location of your poster. People are more likely to like an image if it features a face and might decide to stop by for a chat even if they didn’t yet know about your work.

Presenting my poster at the MICCAI LABELS workshop in 2016
Presenting my poster at the MICCAI LABELS workshop in 2016

That’s all the tips I have tried so far while attending conferences. Please share below if these work / do not work for you and if you know any other tips that might be useful. Thanks for reading!

Bonus tip

To leave a lasting impression, glue your PhD thesis to your poster:

Defending propositions: timeline and the role of graduate school

This is the second post in the Propositions series. The first post is here.

Timeline

Before I dive into the propositions, I want to give a bit of context to how the propositions evolved over time. I show the evolution in the timeline below. The rows are different proposition ideas and the columns are different versions of propositions.pdf I could find in my Dropbox. Red propositions were ultimately rejected. Orange and yellow propositions “made it” but in a modified form. Green propositions are very close to the final versions that I posted last time.

propositions-evolution

As you can see, some propositions survived the whole journey from the first draft in April 2014, to the approved version in January 2015. Today I want to talk a bit more about the proposition on the role of graduate school, which is one of the ideas that I had on my mind the longest.

The proposition

“Graduate school should be a resource, not a requirement”

When I went to university, a lot of things were new, but a few things were exactly how I imagined. You could decide whether to go to class, whether to do an assignment, whether to show up to the exam. It was all my own responsibility, and I loved it! When I got a job – my PhD project, as PhDs are employees of the university – I assumed the same responsibility for my project and my personal development. It was up to me to make sure that four years would lead to a thesis as well as transferable skills for my next job.

But the situation was not the same for all students. In my second year of PhD, the university introduced the graduate school – a program that “ensures and enhances the development of scientific quality along with the needed proficiency for interpersonal skills“. All PhD students could attend the courses, but new PhD students had stricter requirements (compulsory course X, at least 1 course from module Y).

I was happy I could choose from many courses, and signed up for several. Unfortunately my experiences with the compulsory aspect were not as positive. For example, I very excitedly joined a teaching course. In the lessons we would split up into small groups and discuss our experiences with teaching. I shared my experiences with my group, only to hear “sorry, we are only here for the credits” back. You could also earn credits with “on the job” activities like joining a reading group, giving a presentation or collaborating with another student. Taking this to the extreme, you could start a PhD with a list of things to do, and simply tick all the boxes.

Despite my disinterested fellow students, I did learn something in the teaching course: to align learning objectives and activities. But if the learning objective is to become an independent researcher, is ticking boxes the best activity to do?

Defending propositions : introduction

Propositions?

In the Netherlands, along with your thesis you defend a few – let’s say 10 – propositions. Propositions are statements that are “opposable and defendable” and cover a number of topics. The first few are usually about the topic of your thesis, but the others can cover pretty much any topic.
These last few propositions are usually the most interesting, as they resonate with everybody – not just people who are acquainted with your research topic. It is a way to show your personality, by voicing your concerns about a particular topic, or even by slipping in a bit of humor. But apart from being a creative outlet, propositions are also rumored to be difficult to write. In this series of posts I share my experiences with writing propositions, which might give you some inspiration for writing yours.

My propositions

To get started, I present to you my propositions:

propositions
[PDF]

As you can see, propositions 1-4 are about my thesis and pattern recognition in general. Propositions 5-10 are about other topics, but most relate to doing research. It is these propositions that were the most difficult to come up with, but most rewarding to refine into their final form. In the upcoming posts, I plan to share more about a few of these propositions. I will also write about the brainstorming process that I went through, and what I think about this tradition now that it’s all behind me.

While I’m getting the other posts in this series ready, please let me know what you think about propositions. Is it a good tradition? Does it add something to the PhD defense or is it a waste of time? If you are are doing a PhD in the Netherlands, are you thinking about what your propositions might be?

CV of Failure: Things I didn’t dare to try

Image by https://unsplash.com/@tersh4u

What counts as failure?

A recent #withAPhD conversation on Twitter prompted me to write a bit more about my CV of failures.

So far, I have been tracking the “quantifiable” failures, such as paper or grant rejections in my CV of failures, or shadow CV. However, there are a lot of other things that contribute to my experience of failure (and learning to deal with it) which are more difficult to quantify – because I have not tried them at all.

Most of these things can be summarized with the words “impostor syndrome.” I was convinced I would fail, or perhaps even worse, I was convinced “they” would laugh at me for even trying. But time and time again, evidence showed that I probably did have a chance. And even if I had failed, “they” would have thought it was good that I tried. It probably would have been better than always regretting not trying in the first place. So, what do I regret that I didn’t try?

Things I didn’t try

Although originally I thought only about purely academic things, I realized this pattern of not trying goes back much further. A few examples:

  • Talent show in high school. I’ve played piano for 7 years, but don’t consider it “talent show” quality, so I don’t sign up. At the talent show, somebody else is playing piano, but with many hiccups along the way.
  • First year of computer science at university. The student organization has sign up lists for different committees. The committee to organize parties seems fun, but I’m afraid it will have too many people, and I won’t be chosen, so I don’t sign up. Later I become friends with several guys who did join the committee, and realize they would have loved to have me.
  • Internships during my BSc and MSc. Several people are going abroad for internships. I’m afraid to get delayed with getting my diplomas. The projects I do in the Netherlands are all great and I get my diplomas on time, but the gap between the international experience I have, and the international experience that I could have, starts widening.
  • I’m writing papers during my PhD. The most competitive conferences such as NIPS seem to be way out of my league. I submit to good, but less competitive conferences and workshops. My first three papers all get accepted and I have a great time at the conferences. Later, I get evaluated on the quality of my publications – the places I’ve published do not really “count”. I read more papers from NIPS, and realize that maybe, I could have published there, too.
  • I’m finishing my PhD, and read all the regulations for graduating. A part of the regulations describe the requirements for cum laude. This involves a recommendation from the PhD advisors and several external reviewers. I ponder about asking my advisors, but decide against it. After all, even a graduate from our department who wrote several highly cited papers, didn’t graduate cum laude. “They” would find it ridiculous that I even brought the subject up. My defense is a success and the committee members are very positive. Later I confess to one of my advisors about my doubts, and he reassures me it would have made sense to at least try.

I learn from these events by recognizing the patterns and doing things differently the next time around. I did go on to participate in many committees, and even lead the student organization. The internship abroad was possible during my PhD. Although I still haven’t tried submitting to NIPS, I am now getting rejected often (and occasionally accepted) at MICCAI. I approach senior academics and ask for recommendations for fellowships and jobs. I am still scared every time, but past experiences tell me that it’s really much better if I try, than if I don’t.

Or, to quote Susan Jeffers*,

You’re not a failure if you don’t make it. You’re a success because you tried.

* Author of Feel the Fear… and Do it Anyway. The title alone is great advice.

Firsts: reviewing a paper

After a conversation on Twitter about when to start reviewing papers, I decided to post about my own experiences with reviewing, when and how and when I started, and what I wish I would have known in advance / done differently.

I was in my third PhD year that I started wondering about being a reviewer. I heard of another PhD student being asked to review, and got worried why that has not happened to me yet. But shortly afterwards, I was invited for two journals, and then more and more invitations (including more important journals where I have not published) followed, so the worry disappeared. In this post I share some experiences about what helped me with reviewing, and what I could have done better.

Getting invited

  • I discussed the issue with more senior academics, who were already reviewing for a while. They gave me some pointers about how they got started, but they also made a mental note of me being a potential reviewer. Then, if they were an editor, a program committee member, or simply unavailable to review themselves, they could mention my name.
  • I started submitting journal papers, which meant that I had to create accounts with publishers. Editors were then able to link my name to different keywords, even if my own papers were not published yet. In fact, most of the invitations I received were from journals that I had never submitted to, but that shared a publisher with a “submitted-to” journal. In retrospect, I could have created the accounts even before I was ready to submit, and indicated that I was available to review.
  • I maintained an online presence with the page at my university and Google Scholar. My guess is that editors who did not know me personally, would scan these pages to get an idea about my research and experience. Having a blank webpage (with only your name and contact details) might send the wrong message in such a case.

Accepting the invitation

Once I got an invitation, I accepted it! The paper was on the topic of my PhD and the invite gave me a feeling of accomplishment (and relief), so I felt ready to do it. It could also happen that you do not feel ready, for example because it is unusual for PhD students to review papers. My advice is to go for it anyway. Somebody else thought you have the expertise you do it, so don’t listen to the impostor inside you, and give it your best shot! Reviewing taught me a lot about my own writing, so I’m happy I didn’t wait until I was “grown up” to start doing it.

Reviewing

Then it was time for the actual review. I spent quite a lot of time on this, because I was doing the review by myself, and I wanted to make sure I didn’t miss anything. I thought the paper was not very strong, but I felt I couldn’t rely on just my opinion to say that. So, I discussed the approach with my supervisors, who agreed with me.
Then I read a few “how to” guides and started writing what I wanted to be a constructive review. I also went over the reviews I had received for my papers, as both good and bad examples of reviews. This was more insightful than the guides, so in retrospect, I should have asked colleagues for more examples to learn from.

An alternative would have been to ask a supervisor or colleague to go through the review process with me from start to finish. At the time, I probably thought this would be asking too much, but now I feel that many academics would be happy to do this.

Keeping track of reviews

I created a folder called “Reviews” with a subfolder with the name of the journal, and stored the submission and my review of it. This seemed sufficient at the time, as most people listed only the names of the journals on their CV. But it might be a good idea to keep track of what happened to the paper afterwards, should you need that information later.

Update 4th of October 2016: I found an excellent place to keep track of reviews, share reviews and even rate reviews written by others: Publons! I am still in the process of adding reviews to my profile, but you can already check it out here

CV of Failure: introduction

Image by https://unsplash.com/@tersh4u

My CV of Failure

Here it is – my CV of failure, or “shadow CV”.

I first found out about the concept of a CV of failure from this article. After a professor from Princeton posted his CV of failures online, shadow CVs have been getting more attention on Twitter, under the hashtags #ShadowCV and #CVofFailures. And it’s getting very popular too – the same professor now added a “meta-failure” of his shadow CV getting more attention than his research.

I already wrote about various successes and disappointments during my PhD (during my 3rd and 4th years). To write those posts, I used an Excel sheet that I normally use for yearly evaluations. Here is a screenshot from my 3rd year as a PhD student:

excel_progress

The one thing you can probably guess is that green is something that was successful, and red is something that was not. Creating a shadow CV would be essentially compiling all the red parts, over the five years that I’ve been doing research. In the era of tracking everything from what you ate to what music you listened to, why not track failures as well?

The experience

Given that I already had all the data, compiling the CV was quite easy. It was exciting – I was curious whether my shadow CV would be longer than a professor’s. It was comforting – the list wasn’t too long after all, and the inevitability of the list expanding in the near future didn’t seem as daunting. I also realized that it was good to start failing early with travel scholarships, because I feel more prepared now for the larger failures that I encounter.

But most importantly, compiling the CV was motivating. I thought about whether anything would have been different for me if I had seen such CVs a couple of years ago. As many other PhD students, I was not very confident. There were many things I didn’t even dare to apply for. Sometimes senior researchers would tell me these thoughts are unfounded, that I should just apply, and that everybody gets rejected. Sometimes I listened, and sometimes got rejected, but sometimes got accepted, which ultimately gave me more confidence. I hope that seeing shadow CVs can help other students do the same: go for more opportunities, fail, and learn from it.

My relationship with blogging, part 2

As I wrote a few months ago, I have a difficult relationship with blogging. In short: I start, after a few months I think what I wrote is silly, and then I get rid of it. This time I promised myself and the internet that I wouldn’t do this. I made no promises about posting great content or posting often, just that I would accept my posts they way they are.

I dare to say that it’s actually going quite OK! I think the thing that is different this time is being on Twitter:

  • I read more blog posts in general, which helps me improve my writing and gives me ideas on what to write about.
  • I realized there are a lot of people struggling with writing, and that ways to improve your writing (such as doing it, as I am doing now!) are a good thing.
  • I connect with more researchers, and am slowly starting to realize my posts might be useful to others

As part of “blog relationship therapy”, I decided to also be more accepting of posts from my earlier blog – the posts I decided were silly in one way or another, and eventually led to that blog’s doom. I’ve resurrected a couple of them. They are mostly about “first” experiences as a PhD student, such as preparing for a lecture or writing a proposal. Enjoy!

On choosing to do a PhD

This week I gave a talk at the career event of my former student society for mathematics and computer science students, ‘Christiaan Huygens’ (CH). All the speakers were asked to talk about their work, and the choices they made to get there. As it wasn’t always my goal to do a PhD, I thought it would be good for the students to hear about the doubts that I had. And now, for the purpose of sharing N=1 experiences, I’d like to share these thoughts with a wider audience.

Choices

One of the courses I followed during my masters was Pattern Recognition (IN4085, for the readers from TU Delft). I was immediately sold. It seemed magical that reading licence plates, recognizing faces, or predicting a patient’s diagnosis, were all based on the same underlying principles. I followed all the courses I could find on related topics, and did my graduation project on a pattern recognition topic (here is proof).

I was convinced that my job had to do something with pattern recognition. But, I also wanted to somehow apply all the other skills that I had developed during my student union time. As many of my classmates were choosing consultancy jobs, I was excited to find out that there was something similar there to match both my interests: a data analysis consultancy job at a large company in Amsterdam. Note that this was 2010, and nobody was hiring “data scientists” yet, at least in the Netherlands.

Everything was going really well with that company. I had visited them to find out more about their projects and meet the team. I really loved the new environment, and I was convinced I would be hired after an interview. Shortly afterwards, an opportunity to do a PhD in Delft came up. That was also an environment that I loved, but it was entirely different from the job in Amsterdam. It was safe and familiar. After all, I thought that during my MSc project I had already figured out what research was about (cough cough). Fueled by the belief that you should always do new and challenging things, I felt that a PhD would be seen as an inferior choice.

Doubts

But, doubts also started to creep in. I could not define them fully at the time, but now I am quite sure the doubts were: (i) I wasn’t sure I would be challenged enough by the technical challenges of the job in Amsterdam and (ii) I thought I would be challenged too much socially — always looking professional, always interacting with people. All things that I enjoy, but perhaps not every day, and not coupled with a long commute.

Perhaps my biggest problem was that at the company, it would be part of my job to oversell things — make things seem more impressive than they are to clients, and come up with convincing arguments on the spot. This ability was considered very valuable among my classmates, and I imagine quite sought after by companies. I of course felt honored by the company’s belief I had this ability, because it wasn’t something that came to me naturally. So while I was excited and challenged by the prospect of developing this ability, it also felt like I would be betraying myself a little bit.

Best advice I received

One thing that helped me in my decision is a conversation I had with a former student. He had just finished his PhD and was switching to an industry job. The advice was to

  • compare concrete opportunities (rather than PhD vs not PhD)
  • consider the people I would work with and the daily tasks I would have to do
  • whether I could be myself, and how the position would fit in with the rest of my life

This cleared things up for me, and once I had accepted the PhD position, it felt like a huge weight off my shoulders.

I didn’t regret my decision for a second afterwards. I had a great time during my PhD because of the people that I worked with, not only inside the lab, but also other researchers that I met at conferences. I discovered I was very wrong about knowing what research is about! There are new, exciting and challenging things in my job every day, but challenging in a way that is meaningful to me.

I also had quite a lot of freedom, both in the ideas I pursued and in managing my time. I was in the office during regular work hours, but I appreciated being able to take a day off, go on holiday, or work at home if I wasn’t having the best day. I felt that I was being valued for my ideas, rather than for showing up. And isn’t being valued one of the most important components for job satisfaction?

Keep in mind

That is not to say that you should always choose a PhD when in a similar situation. There are huge differences in PhD positions as well! Not all researchers are nice people, and not all projects offer the same freedom that I had. I hope that this strategy — considering concrete opportunities, and staying true to yourself — should help you with the answer.

Why you should post preprints on arXiV

Recently on Twitter I saw a lot of discussions about preprints, such as under the #ASAPBio hashtag, which originated in the biology community. My guess is that preprints are more or less common in different fields, and I thought it was normal for Computer Science to do it, so I couldn’t contribute anything to the topic. But I’ve encountered some doubts when I encouraged other CS students to upload their work to arXiV, so I thought I’d share my N=1 experience with preprints.

Long story short, I spent a good part of 2013 writing journal papers. I submitted three of them that year, and directly uploaded the submitted versions on arXiV. You can see my page on arXiV here.

I spent 2014 revising these papers. One paper was accepted in 2014, and two others only in 2015, when I was already a postdoc. One of the accepted papers is still in press, even though it is already 2016. I imagine it will be three years (!) between my initial submission — that really isn’t that different from the revised version — and the published version. And this is in Computer Science, a fast-moving field!

As a PhD student / postdoc / aspiring researcher, you can’t really afford such a time lag. And that is where preprints have been immensely helpful to me in different ways:

  • Two of the papers were based on earlier conference papers. When I was discussing that work with other researchers (at conferences, via email), I could send them the preprint, which contained more detailed results.
  • The third paper (a type of survey) was completely new, and I was a bit scared that somebody would publish something similar before me. The preprint was actually a way to assure myself that it was now documented that I came up with the idea. Again, I discussed this work with other researchers while it was already in arXiV, and even got some valuable comments, which helped me a lot when revising the paper.
  • The preprints were cited (mostly by myself, but also by other researchers). After publication, I merged each preprint with the published version in Google Scholar. I don’t really have a lot of citations, but I would have had even less if the papers only became available in 2015 instead of 2013.
  • I didn’t apply for jobs while I had any unpublished preprints, but if this was the case, I could put the preprints on my CV, which is more informative than simply listing the paper title with the comment “manuscripts in preparation”.
  • Most journals allow this! You can check on this website what your journal’s policy is

If you are a student in Computer Science (or anything, really) and you are doubting about uploading a preprint of your recent work, I hope this might change your opinion a little bit.

%d bloggers like this: