 |
For this post of How I Fail I’m interviewing Michael Ekstrand, a computer scientist studying human computer interaction and recommender systems. You can find more about him on his website or follow him on Twitter. |
1. Thanks for joining the How I Fail series! Please introduce yourself
I’m Michael, an assistant professor of computer science at Boise State University in Idaho (northwest USA). My training is in human-computer interaction, and I do most of my work on recommender systems with broad interest in algorithmic assistance or mediation for information access (information retrieval, a lot of machine learning and data mining, etc.). One way I think of my work is that I smash people and intelligent information systems into each other and see what happens. I get to do this work with a lot of great people, particularly my colleague Sole Pera and our students in the People and Information Research Team, and my collaborators at Minnesota, Eindhoven, and elsewhere. I also maintain the LensKit recommender systems toolkit.
I joined Boise State in 2016; I received my Ph.D at the University of Minnesota (with the GroupLens research group) in 2014, and then spent as an assistant professor at Texas State University. I also spent a summer as an intern at Autodesk Research in Toronto while working on my Ph.D.
2. Do you keep track of your failures? Why/why not?
Not formally, although it is starting to emerge as a part of the record-keeping process to make annual review materials and eventually my tenure application easier to prepare. This isn’t the result of a conscious decision one way or the other – I have a mental list of failures that led to most of my CV lines, but I haven’t yet had the time to think about whether and how, strategically, I want to contribute my failure record to a healthier academic culture. This post is a start.
3. What do you think about sharing failures online?
It is an interesting idea that has a lot of potential merit. I have thought about doing it; the main reason I have not is – as indicated above – I haven’t decided to.
There are a couple of downsides that I see. One is Hal Daumé III’s point that they can come off as a humblebrag. I completely agree with that concern. Another thing I think about is the insidiousness of comparison. A reader will inevitably compare their own record with the one they are reading (or worse, their faulty memory of their record that may have an inflated reject count because they’re at a mental/emotional low point).
We need to do something as a profession to normalize failure and rejection, to bring out in the open that everything isn’t always as cheery and bright as it looks. I see promising trends in that direction, and that is very heartening. To the extent that publishing failure CVs helps that end, I am all for it.
4. What do you do when you receive a rejection?
I don’t have much of a ritual. I usually read the reviews, get grumpy for a bit, and then come back to them after a few days. I generally copy reviews – accept or reject – into a Google doc where my co-authors and I can mark them up, identify their discrete points, and discuss how to respond.
I often already have a secondary venue in mind when we submit a paper, so rejection becomes ‘execute on the backup plan’. Sometimes the feasibility of a fallback plan (coupled with my estimate of the odds of success) becomes a factor in deciding where to send something. This helps with rejection, I think – it’s just another possible outcome of the publication process that I’ve already been planning for.
Depending on the nature of the work and the target venue, I may tweet about the reject.
5. What about when you receive good news?
I usually share good news with my spouse, co-authors, close colleagues, and Twitter, in whatever order I talk to them next. I usually tweet acceptances, and occasionally rejects. Rewards often involve pizza and/or a good drink. Or something else nice, like good fish & chips.
6. If you would have a CV of failures, how would it compare to others in your field?
It’s hard for me to reason about the overall failure rate, but I can try to estimate it per category.
I think my rejects-per-published-paper rate is probably a little lower than average. There are a number of potential contributing factors, none of which involve me being a superior academic. Lab prestige is likely one factor, particularly before RecSys went double-blind. I’ve also targeted RecSys with the bulk of my work; I know the community well, and that probably enables me to more effectively target the audience with my writing. I feel like my overall submission rate has also been low, so there haven’t been as many chances for rejection as some of my colleagues. I’m working on fixing that.
My grant reject rate is probably at or a little above average. So far it’s 0 for 6 (or 7?) with the NSF (current success probability estimates are at around 10%), and 0 for 3 with private-sector grants. I’ve had one successful internal grant proposal. Boise State has fantastic grant writing support, though, and my ideas are getting clearer and better, so I am hoping to get a hit sometime soon. I find that writing support helps a lot: skilled writers with a good eye for making the proposal responsive to the solicitation make it so much better. The last proposal I submitted is the best I’ve ever written, and I hope to continue improving that skill.
7. Can you share some examples of failures which hurt the most, and why that was?
The parade of rejects when I was applying to graduate schools hurt at the time. I am extremely glad that I went to Minnesota, and would not want it any other way: working with GroupLens was one of the best experiences of my life, and it’s hard for me to imagine better advisers and colleagues. But at the time, I didn’t know that would be in my future. I wanted to do programming languages at Rice or Penn State, and they were saying no. I didn’t know what was next.
Seeing my software not take off stings a bit sometimes, particularly when I see other projects give ‘LensKit was too hard/impossible to use’ as a motivation for their work. I’ve put a lot of work into the tool, and want it to be a useful resource. I hope that it can still be one, and we’re doing work on trying to address a number of the particular pain points, but seeing it rejected can be frustrating.
And then there’s the 3rd conference reject for one of my software papers. It was pretty clear that one of the reviewers had also participated in the first round of reviews, and is invested in a competing technology; that’s all fine. What was frustrating was that their review sounded like they were reviewing the first version of the paper again, so it felt like all the work we had been doing to improve the material wasn’t making a difference. We went on to publish the work in a journal, with a Major Revision round bringing the paper back around to an overall state that was remarkably similar to the original paper (though with many important improvements).
8. Can you think of something you did that felt like a success, but you wouldn’t find in a “success” CV?
It shows up on my CV, but does moving to a new position count as success or failure? Success, because I did two job searches that each concluded with accepting an offer? Or failure, because the best option my end-of-Ph.D job search produced didn’t wind up being a very good fit?
It feels like a success, because I started at Boise State with a much clearer picture of what I wanted to do, two years of experience, a full tenure clock, and a fantastic batch of new faculty.
9. Is there something we can all do to improve how failure affects others in academia?
Talk about it openly. There are a number of forces that resist this; these include general societal, and academic, tendencies to put on a happy face and push failure under the rug; and rejection of work that we don’t feel we can yet publicly discuss. I find the increasing interest in this conversation quite encouraging. We need to normalize failure and rejection, and destigmatize reaching out for help.
One of the most helpful things I read as a first-year assistant professor was Matt Might’s article on how to get tenure, particularly this quote:
“My first year as a tenure-track professor cannot be described as anything other than an abject failure. I was so desperate to publish and raise funds that I began thin-slicing my research and submitting lots of poor quality papers and grant proposals.”
Having this high-impact professor, who I greatly respect, write that he had a terrible first year helped me realize that maybe there was hope for me too. We need more writing like that. We need to get the message out that there will be down times, both during and after grad school, and that your first year as a faculty member will probably be pretty bad.
Increased efforts on promoting diversity in the academy – and with it, the compassion and empathy that are needed to ensure that people with a wide range of experiences and needs can succeed – help drive some of this, I think. Having people who ‘make it’ tear down the curtain may help reduce impostor syndrome and related effects in their peers. Promoting a culture where it’s OK to fail, get rejected, and generally not be OK makes it easier for people to reach out and get the help that they need.
The movie Wreck-It Ralph has a great scene where a group of video game villains are attending a Bads Anonymous meeting and say the group affirmation “I’m bad, and that’s good. I will never be good, and that’s not bad. There’s no one I’d rather be, than me.” We might need something like that for academics. “Things are bad, and that’s ok. They won’t always be good, and that’s not bad.”
I think we also need to find ways to talk about these things with those who aren’t yet in research academia, such as undergraduate students who may (or may not) be thinking about graduate school.
10. What is the best piece of advice you could give to your past self?
Don’t rush things. If a problem is interesting and important, keep thinking about it from time to time, and eventually you may crack it. There’s a thorny issue in recommender system evaluation that I’ve been thinking about, and talking about, for at least 5 years now; in the last year or so, a path forward finally clicked and turned into a grant proposal, a forthcoming paper, and at least one more paper in progress.
But with that, make sure there are things that you can move forward on even while big ideas are in the slow cooker. I’ve been working the last couple of years to keep my research pipeline flowing, and would have benefited from thinking strategically about it earlier in my career. Some things aren’t primarily a pipeline problem, but maintaining a steady stream of research output is.
***
If you want to read more about failure, sign-up for the weekly newsletter so you won’t miss any posts!