I only wrote about how I use Kanban for managing research projects quite recently. Here I described a physical Kanban board as well as “Kanban-like” workflows in various apps. What I didn’t know at the time, is that Todoist was working on a boards feature, which I first used in a beta version, and which has now been released!
Todoist was kind enough to feature one of my boards in their first blog post on boards, but in this post I give a few more details on how I now use the boards feature!
Research project overview
The board that I wrote about before, and that Todoist featured, is my overview of research projects. This was a picture of my physical version from a few months ago:
And here is how the Todoist version looks like (image credit: Todoist):
The columns are similar to the physical version, going left to right in chronological order (i.e. ideas on the left, published papers on the right). Several cards link to the projects for those specific papers.
Blog posts / How I Fail
Other boards that I use are for my blog posts and How I Fail interviews. The How I Fail board is especially important to emphasize, since multiple people are involved. Since I do not want to reveal the names of people I haven’t interviewed yet, I cannot post a screenshot, but I have several columns here:
Invited, waiting for response
Accepted, plan interview
Schedule blog post
Most of these are self-explanatory – if I invite somebody they can either decline, or accept, and if all goes well the card move to the “published” section. “Dropped” is for people who accepted but then stop responding.
An important thing about such boards is that I would still like to see the topics/names that are “done” but I should not address again, so I don’t want to check-off the todo item. To deal with this, I use priorities and filters, but it is not an elegant solution.
Experience Points (XP)
I also have a board with various tasks, that just need to be done, but are not necessarily part of a project. This often includes scheduling “life admin” appointments, filling in reimbursement forms etc. As somebody on Twitter suggested, “XP” (based on games) seemed to be a more fun name for these tasks :).
Whereas the previous boards have a logical ordering to the columns, this board do not have a specific ordering. Rather, each column is a particular category (stuff to buy, invoices to pay, etc). Once a column gets too long, I try to clear it by doing several similar-type things at a time.
I am enjoying the boards functionality in Todoist, and imagine I will be using it more. What I am still missing is being able to define your own rules for how a board works, for example:
Cards in “maybe” column do not show up in as “open tasks”
Cards in “waiting for” remind you to check the status, at a specific time interval
Cards in “done” are visible, but do not show up as “open tasks”
If you are also using Todoist boards, I’d love to see more examples!
Although I have only supervised a couple of students during my tenure track, I already found often saying the same thing during each meeting – in particular, what are good papers to start reading about a particular topic. Since I was already an avid Evernote (get 1 month premium for free here) user, I decided to see if shared Evernote notebooks could be the solution to share papers with students. This might be also an option if you are organizing a journal club. Read on for the solution!
Remember that Evernote is not a reference manager, but it is where I store the paper PDFs and notes about the papers. Jabref is where I store the references. The only link between the two is the Bibtex key, which is how I name the note in Evernote.
This is my paper collection i Evernote – 913 in total – and each note is a paper (or report, etc).
Each note is at least the PDF I saved (below), and perhaps some notes I made about the paper. And this is how it looks like in Jabref:
Since there is no direct link, I might have a paper in one place but not the other, but papers that I cited in my own research in the last few years, are definitely in both.
Sharing your paper collection with others
Since Evernote allows sharing notebooks, to have a shared collection of papers all you need is to share the notebook with the people involved. For the students I was supervising, I used the “can edit” as permissions so they could also add new notes, annotate papers etc. But you could also choose “can view” option if you prefer.
Sharing a collection of 900+ papers is probably not effective 🙂 But what helps here a lot, is the tagging system of Evernote. When I add a paper to this notebook, I add several types of tags:
Type (paper, thesis, etc)
Topics (specific types of machine learning, applications etc)
Projects (a specific project where I might want to cite this paper)
“Priority” (p1, p2, p3 or p4)
I have been using the type, topics and projects for a while, but the priority was an addition after I shared the notebook. Roughly, the priorities translate as:
p1 – everybody in the lab should read this
p2 – important paper for many projects in the lab
p3 – relevant to some projects
p4 – not related to our research but more “general interest”
With these tags, you can then do queries on topic & priority. So for example if your project is on transfer learning and you want to find all papers I might suggest, the query “tag:ml-transfer & tag:p2” gets you 43 results. Still a lot, but now it’s doable to screen the results and narrow them down.
It’s good to mention that since the notebook is originally mine, only my tags can be used within the notebook. So somebody with edit permissions would be able to add more of the tags that I use, but not add entirely new tags.
The system is easy to use, paid account only needed if you want a lot of storage
Saves time both for me and for students
Less chance to miss a relevant paper
Everybody can use their own reference manager if they want
Could limit the way students explore literature
Limited commenting possibilities (notes from everyone appear the same by default)
No true integration with a reference manager
This system has been quite helpful for me with several student projects. However, there are many things I am still missing, such as creating your own fields for each paper, and interacting with the annotations through a spreadsheet. (This is possible in Notion, but that is something for another post…)
However, an important quality of any system is that you actually use it. Since I already use Evernote on a daily basis, it works for me. But I’d love to know what everybody else is using for sharing literature with others – please leave a comment below or let me know on Twitter!
Following up on the post about organizing student projects, I wanted to explain a bit about how I keep track of my own projects on a slightly higher level of abstraction.
When I started doing research, I was working on one, maybe two projects at a time. But as time went by, this number can increase quite quickly. I get easily excited about new ideas and starting projects (the “shiny object syndrome”), as well as joining projects by others. Over time this led to several situations where I had more projects than I could handle, leading to delays or abandoning the project altogether (see these 9 ways to fail a project for more on this!).
My solution has been to “just” limit the number of current projects. In this post I explain the tools I use to keep track of my projects effectively.
Visual overview of all projects
The tool that I’ve found the most helpful, is to use a Kanban board. The idea behind Kanban is to “manage work by balancing demands with available capacity” (Wikipedia) – sounds exactly like what everyone needs, right?
Here is how the Kanban board in my office looks like. A card is a project/paper, and it can belong to these categories: Idea, Incubator, Doing, Preprint/Revise, Under Review, Published.
Here is how I use the categories:
Ideas are just that – ideas. Perhaps I read a few papers on a topic and thought “I should do something about this”. Ideas can be good for starting student projects, since I probably won’t have time to get to this topic myself.
Incubator is a category for projects that are a bit further than ideas (for example, there is a preliminary experiment), but that I do not want to focus my attention on just yet.
Doing is a category for current projects, that you want to advance every day or week. There should be as few as possible projects in here!
Preprint/revise is for “mostly done” projects, but that still need a bit of time investment to complete
Under review are papers under review, that might return to the “Preprint/revise” category in a few months
Published are accepted papers!
(Bonus) Graveyard is for projects I decided NOT to continue, you can see it in the bottom left of the board. I thought “graveyard” sounded more dignified than “abandoned” but am open to other suggestions.
Next to these categories, I use the color of the card to indicate the type of project. Green are research papers, yellow are education projects (such as my portfolio), and red are grants. You don’t see any red right now, because this is already after I decided to leave my tenure track position :). I do not include various recurrent responsibilities on this board, but you could decide to do so.
Finally, I have a horizontal divider between projects that I’m leading, and projects I’m participating in. Overall, this gives a nice overview of all research projects I’m involved in! If you want to do the same with the board in your office, you might want to get some dry-erase magnetic cards, such as:
Project overview in apps
Although most of my systems are digital, I like this visual overview in my office (or at least, before the pandemic). But this is just one way to organize things, and it might not be sufficient for you if you get distracted easily.
Fortunately, there are various ways to implement the same idea in different apps. You can have the same type of Kanban board in apps like Trello or Notion. But even apps which are not organized like a board, are suitable.
Here is an example for Todoist which I use for getting things done. Here you can group projects under other, top-level projects. If you call your top-level projects “Idea”, “Incubator” etc, you can easily see how many projects you are handling at any one time. Similar to my board, you can use the color of the project to indicate research, education etc.
But for example, even in Overleaf assigning a tag to a paper can help you achieve the same. Here’s mine, with slightly different categories.
You can see that the board and the Overleaf are not 1-to-1, because some projects can have multiple Overleaf documents, and because I’m bad at updating tags 🙂 But, at least I’ve succeeded at not putting everything in “Doing”!
I’m happy with this system overall, and imagine I will continue using it both for work and personal projects.
A feature I am still missing, is to have an indicator of time commitment per project, and for “what’s already there” on your calendar. For example, I could imagine having actual “slots” in the Doing category, and having larger projects take up multiple slots. And when you already have many things on your calendar, the number of slots decreases. So if you hear of an app like this, let me know 🙂
Project supervision is one of the many things you do during a tenure track. Since I was already interested in project organization, I did some research, which inspired my own lab scrum setup. I discuss why and how I used scrum to organize student projects, and my take-aways from the experience. To find out more, read on!
At the start of my tenure track I did a bit of research about what others had recommended, and came across several interesting papers about project organization. This is a whole post in itself, but for today’s topic, here are a few papers that I found helpful:
The last two papers are about a technique called scrum, which is a type of process frequently used in software development (more background here). Traditionally in this process, a team is working on the same project. This is different with several students working on different projects. Another difference is the timing, which might be slower in a research setting. Nevertheless I was inspired by the ideas in these papers and decided to try it out.
Setup with Kanban board
Although I was excited by the idea of trying a different type of organization, I had no previous experience with scrum, and didn’t want to introduce too many things that would be overwhelming for everyone. What follows is the setup we (myself, 4 MSc students and 2 PhD researchers) used for 6+ months, where some things are loosely based on scrum, papers I read, etc. This setup has advantages and disadvantages, which I discuss later in this post.
The main idea was to keep track of all projects jointly, via a shared Kanban board and two weekly meetings with everyone there. Typically we did the following meetings:
Tuesday/Thursday – individual meetings in time slots as needed (30 min each)
When planning tasks, we added “post-its” (I bought these reusable magnetic ones which are pretty awesome) to the shared scrum board. We initially used different colors for different types of tasks, but using different colors for different people might be more logical.
For me it was important that everybody created actionable, finite tasks. So, “literature research” is not OK, but “summarize 10 papers on topic X” is. When students had exams, they included studying as a task. We didn’t have guidelines for how small or big a task could be, although in practice they were probably things that could be done in days, rather than hours or weeks.
New tasks always started in the backlog section of the board. On Tuesdays, tasks can be moved to the “in progress” section. The idea is not too have too many “in progress” tasks at the same time.
Every group meeting was essentially a longer “stand-up”. Each person (including me!) would briefly say something about their “in progress” tasks. This involved saying something about what was done since last time (and if the task was completed, still in progress, or deprioritized), and any problems that came up. Suggestions from others about things to try usually followed. When it was clear that I needed to spend more time with the student, or some students could help each other, additional meetings were planned. This way this meeting was an hour at most, but usually closer to half an hour.
Everyone could plan an individual meeting with me via a shared calendar with 30 minute time slots. In practice, about 4 slots would be filled each week, so I would see each person at least once in two weeks (next to the group meetings).
Alternative with Google Slides
While the initial setup had many positive points, there were two main things missing. The first was more of an overview of what has happened / is happening in the period of a few weeks. The second was the ability to show something, such as results (bugs, etc).
For these reasons, we switched from the Kanban board to a Google Slides presentation, where each person had two slides, one for results, and one for a 6-8 week task planning and progress. The slides had to be prepared before the Tuesday meeting. Otherwise the meeting setup was mostly the same.
This setup provided more overview, but I also missed the structure the Kanban board provided. In the end, I was thinking about a system that would have both features, but I didn’t get the chance.
I’ve already mentioned a few advantages that this system had, but here is a recap.
First, I think this is a great way to have a “lab feeling” if you are in a similar situation to me, and do not have funded projects with multiple students or physical lab space. Although the students all did distinct projects, it did feel like a team. Getting coffee, bringing cake etc also helped of course 🙂
Second, I saved time by not having unnecessary meetings, but without compromising my availability. Further time is saved by less repetition when explaining something, and by identifying similarities across projects, where students might be in a better position to help each other.
Third, I think this setup improved everybody’s planning skills, but also their awareness of how planning is hard. I also participated with my own projects, and I typically got the least done because of other responsibilities. I think this is important for students to see. Students seeing each other’s project plans likely gave them more examples to learn from, and perhaps a bit of accountability.
The disadvantages of this system, from my point of view, mostly have to do with implementation. First, it takes a while to figure out how to do everything, if you try to adapt a system to fit a different situation. There is also time involved in figuring out how/where to meet (if you don’t have a dedicated space) and/or selecting which apps you want to use.
Second, your adaptation may miss parts that you want to have. We did not have a clear separation of meetings (such as planning only, retrospective only) or project roles (such as scrum master). Perhaps these things might have felt silly at first, but I do think they would have been beneficial.
It’s possible that this setup might not be the preferred setup for some students, who want to keep everything about their project private. I do not have specific advice for this situation. But ultimately different labs are organized in different ways, and it’s OK that this might not be for everybody.
Overall I would say that doing this is a worthwhile experience! Do spend more time thinking about the exact implementation beforehand, particularly what meetings there will be, who will do what, and where all the plans/tasks/results will “live”. Once you have this in place, help people stay with the process for a least a month or two to evaluate if it’s a good idea.
This post is inspired by a discussion on Twitter, started by Antony Caravaggi and continued by Christian Baumgartner, who also sent me several follow-up questions – thanks! I’d also like to thank everyone had first-hand experience of my lab organization ideas 🙂 – Ralf Raumanns, Ishaan Bhat, Tom van Sonsbeek, Rumjana Romanova, Colin Nieuwlaat and Britt Michels. Thanks a lot!
However, last year following a period of illness I’ve reconsidered the tools I use. In this post I explain why I switched from Todoist+Evernote to only using Evernote, and why I later decided to go back to my trusted system.
Downsides of Todoist
My main problem with Todoist is that it is too easy add tasks.
That might sound a bit weird. Of course the adding tasks functionality of Todoist widgets is great, and it is easy to capture all the little things you need to do. But since all tasks have the same “weight” (even if you give them different priority), your overall task list becomes too focused on not-always-important, little tasks. Although I was regularly organizing my list, just having all the other tasks there was kind of weighing on me.
A related problem is that when you add a task, you don’t see what tasks you already have scheduled. So you can be too optimistic when adding a task for “tomorrow” when you already have various meetings and other tasks scheduled.
Finally, Todoist has a desktop app, but it doesn’t work if you start it when you are offline.
Evernote as your todo-list
Evernote is not a specific todo-list app, but it is possible to use it as such. You can see notes as individual todos, and then organize them via notebooks or tags, or you can create a checklist in a single note. I decided to go with the checklist approach, and created two notes – “Current” and “Maybe”. “Current” was for anything that was coming up, and “Maybe” for projects that I might or might not do.
Most of the time I worked with the “Current” list, where I made a table with one row for each week, and columns for different types of tasks. I started with “work” and “home”, but later split these up into more categories, based on priority.
This system had several advantages that I missed with Todoist. When adding a task, I had to add it to a specific row, so I would already see what other tasks I had planned for then. Also, I became more aware of the weight of the different tasks, and I feel that overall my todo list became more balanced.
This way my todo list was also accessible offline, and it was in the same app as my other project-related notes.
Downsides of Evernote
Unfortunately, there were a few disadvantages as well, that made me miss Todoist.
The main thing I missed was the integration with Google Calendar – in Todoist I would enter a date and time, and an event would show up on my calendar. Now I had to create a separate “Planning” calendar, and add tasks manually – which I didn’t do consistently.
Another problem was recurrent tasks, which I did once a week or once a month. In Todoist this is basic functionality, but Evernote does not have this feature – you can set a reminder for a note, but when it’s time, you have to reset the reminder yourself.
After 2-3 months of using Evernote only, it felt good to create a list in Todoist again. I’m more mindful of the downsides and am trying to manage them better, for example by using filters for my tasks and scheduling tasks for next week on the calendar. It’s not yet the ideal system I wish I would have, but I think using it consistently does help in the long run.
Do you have any tips of how to create a better todo-list / calendar system? Let me know in the comments!
As I get more responsibilities and work on more projects, I find myself increasingly using checklists. Especially with things that do not occur too frequently, I have to think “how did that go again, what should I not forget?”. After reading The Checklist Manifesto, where Atul Gawande discusses checklists in airplanes and in surgery – where you really don’t want to forget anything, I decided to try it out for myself. So far I’ve made myself checklists for the following:
Student starting a project
Planning a conference trip
Giving a talk
Having a paper accepted
I keep the checklists as templates in Todoist. I’ve broken down each of these into different stages (for example, before the trip and after the trip), with various one-off tasks I need to do, such as booking travel, or filling in reimbursement forms. Sometimes I add links to the tasks, which will take me to the website or Evernote note I need to complete the task.
Then as soon as one of these events comes up, I copy the template to a new project, and fill in dates for each task which are suitable (this could be automated somewhat, but I prefer to have control over this). This way I will never forget all the details that need to get done.
I like this approach and would like to apply it to more things I do in my job. For example, I’m thinking about making teaching each class into a template. Although my materials are prepared from the year before, I still need to go over the materials, post all the details on the learning environment, grade exams etc. Since I already know all these things are coming up, I might just as well add them to my todo list (and reserve time for them!) with a few clicks. As I’m trying to improve estimating the time I need for a task, I can become more and more specific with this.
It would be great to have such a checklist for every new project. I can imagine such a checklist would contain, for example, creating a directory structure for the code. These tasks are of course much easier to estimate than actually working on the project, but perhaps some day I will get there as well.
However, there are other things I do in my job that I can plan in advance. For example, this year I have also been documenting how long I needed to prepare for lectures and to grade assignments. I can use this information to create a checklist for each course, and just repeat the checklist every year. For activities that do not take place on fixed dates, for example reviewing papers or giving talks, I could already budget hours, and move them around as needed. This would probably also help with saying no to more things.
Do you use any checklists? Or is this an overkill? Let me know by commenting below or on Twitter!
After a conference most academics probably face a fairly full inbox. In this post I share a few tips I’ve found helpful with managing my email. I am by no means an expert, but I’m happy with some of the strategies I use, which I share below.
The idea of inbox zero is exactly what it suggests – once you process your email, ideally you should have ZERO emails left in your inbox. Following the “getting things done” system, I try to either handle an email immediately (if I can delete it or if I can reply quickly), or put it on my todo list for later (if I need to look things up first, can’t take action immediately, etc). Once an email is on my todo list, I archive it, so it’s not just sitting there staring at me. I can’t imagine how stressed I would be without this strategy.
I have a few things to improve though. A few emails do not fit into either category, for example if I read my email on the phone, but need to use my laptop to reply. This is not helpful since I am revisiting this email several times, instead of once.
I try not to email on evenings and weekends. The goal of this is to enforce boundaries on my work time and manage expectations of others, both with regards to when they can expect to reach me, and what I expect of them. I appreciate that people might have different working hours. Therefore, when I do email outside of my normal hours, I usually schedule the email to be sent later. I use the Streak plugin for Gmail, but there are others, like Boomerang, and Outlook has delayed sending functionality as well.
Some of the emails I send are very similar to each other, for example with information about student positions. To avoid typing the same information over and over, I use the Snippets feature for Streak. The same functionality is possible with Canned Responses by Gmail, but I like the user interface of Streak more.
For mail that’s not urgent to read, I have filters that skip the inbox, and deliver it to a folder I call “Snooze”. For me these are typically newsletters and announcements that are not personally addressed to me.
Although this type of functionality is offered by Gmail with automatically labeling emails, I prefer to define my own rules of what is important or not. This means that there is an initial time investment, every time I receive a newsletter, to create a filter for it.
These are the main strategies I use, but I would love to hear more of what has worked for other people – let me know in the comments below or on Twitter!
I recently discovered that one of my biggest dreams – being able to link all my online LateX projects to a single .bib file (but without using Mendeley) – is possible!
Previously I had “solved” this problem with ShareLateX projects, by writing a script that copies my main .bib file to several project folders every hour. However, this requires Dropbox sync for ShareLateX, which is a premium feature for new users. Not to mention, it’s not ideal to do it every hour, and to have to update the script when you want the .bib file to be copied with an additional project.
Now a much simpler solution is possible with Overleaf. It’s probably been there for a long time, but I only realized this now. When you add a new file to an Overleaf project, there is an option “Upload from URL”. I thought this option would do just that – get the file from the URL and upload it. But what it actually does, is remotely link to the file. That’s the solution right there! (Thanks to Overleaf’s Dr. LianTze Lim for pointing this out!)
Here are the steps to get this to work:
1. Put your .bib file in Dropbox, set the sharing settings so that it’s accessible by anyone with the link. Copy the link
2. In your Overleaf project, Go to “Files”, then “Upload from URL” and paste the link here. With Dropbox, this link will end in “dl=0”. Change this to “dl=1”
If this works correctly, you should now see your .bib file in your project, but with a “linked” icon next to it
3. Proceed as you usually would with a bibliography file
If you are collaborating with others, the best way is probably to have two .bib files – the linked one (not writeable from Overleaf) and another one that is local to the project, for any new references. This way at the end of the project, you could move all the new references to your main .bib file.
I also tried to do this in ShareLatex, but couldn’t find this type of option. Although ShareLateX has free Dropbox sync because I’m an early user, this feature of Overleaf could have convinced me to switch (even if losing the sync). But Overleaf and ShareLateX are merging, so I’m hoping I might get to enjoy the benefits of both.
If you are a reader of this blog, it’s probably no surprise that I love Evernote. In this post I summarize all my favorite ways to use Evernote – a few that I have written about before, and a few that still need to become blog posts. Enjoy!
I write my blog posts in Evernote before transferring them to WordPress. I like this because I can use Evernote online, so I can write when I don’t have Wi-Fi, and I get less distracted in general.
2. Weekly review
I use Evernote in my Getting Things Done system, which I’ve blogged about here. During the weekly review, I use Evernote to write down a summary of what I have done each week for different goals, such as writing or exercise.
3. No list
Also during the weekly review, I write down what opporunities I have said yes to, and what opportunities I have said no to. This has made me more aware of how many request I get and accept, and easier to say no.
For the things I did say yes to, I might forward the invitation to Evernote and tag it with “my_CV”. When I need to update my CV, I can add all these items based on the tag. I do the same for any media that mentions me, just in case I need this for an annual review or a grant application.
5. Read later
The first thing I started using Evernote for was to save blog posts or articles to read later. I wrote about this in the “capture’ part of the GTD process. The saved posts go to my Evernote inbox. During the weekly review, I briefly look at the posts and either delete them, or add tags and move them to my ideas notebook. When I need information on a specific topic, I just search the ideas notebook for that tag.
6. Organizing literature
Evernote is great for saving PDFs, so I use it to organize the papers that I’m reading. I find the tags very helpful in finding the exact paper that I need. Read more about my system here.
7. Organizing travel
When travelling for a conference, I keep all the reservations and other important information in a notebook for that specific trip. Read more in this post.
8. Happy thoughts
I forward emails with that made me happy to Evernote and tag them with “happy”. This could be anything from getting a paper accepted, to a comment from a blog reader that my posts have helped them. The idea is similar to the “jar of happy” – review these notes when you are feeling down. Similarly, you could add photos, drawings, handwritten notes – anything that can remind you to appreciate life.
I have used Evernote as my 5-minute journal, although I have now switched to paper so that I don’t use my phone just before bedtime.
10. Text snippets
I have several notes where I store pieces of code I often use for my blog, such as:
hiding an image (great for attaching Pinterest graphics to the post)
shortcodes for WordPress
embedding newsletter sign-up forms
Of course I can always find these snippets from within WordPress, but I find it easier to keep these snippets in a single note.
11. How-to guides
I have started writing guides for myself, on how to do things that I have to do from time to time, but not often enough to remember all the steps. For example:
Installing essential WordPress plugins for a new website
Installing and configuring software if my laptop dies
Creating a yearly financial report
12. Meal planning
Evernote is great for keeping track of your favorite recipes! I have two notebooks (both shared with my partner) to plan my meals – one with recipe ideas, and one with recipes where we drag and drop recipes for the upcoming week. This way it becomes really easy to create a shopping list!
I do not really use Evernote as an address book, but I might keep important information about people I know, like food they don’t eat, their favorite beer, what gift I gave them last year, etc.
When people recommend me books or podcasts, I tend to add a short note to Evernote about it. This way if I check out and enjoy the recommendation, I can thank the person later!
14. Buy later
When I think I might want to buy something online, I often save the item to Evernote first and tag it with “buy”. From time to time, I go through this tag. Often I’m able to delete a lot of items – a few that I bought, but most that I decided I didn’t need. Great for keeping your place uncluttered AND saving money.
15. Gift ideas
No “best ways to use Evernote” list is complete without gift ideas! Similar to the “buy later” section, I save items that could be good gifts to Evernote, and tag them with “gifts”. The difference is that I don’t delete the note once I’ve bought the gift. Instead, I can edit the note to include who I gave the gift to.
If you like these ways to use Evernote, check out my Pinterest board where I’m always collecting more ideas:
I’ve been using ShareLateX, which lets you edit your .tex documents collaboratively online, for years now. I’m happy with it, except one thing: you have to have a different bibliography file per project. In this post I explain a workaround I created for this problem. It’s not beautiful and requires that you use Windows and have ShareLateX premium, but if you have a similar problem, read on!
A better workflow
If you are like me, you generally “manage” your bibliography by the following method:
Copy-paste .bib file from a recent project
Copy-paste new references from Google scholar while writing
Clean up all the missing fields etc, when paper is ready for submission
Forget you did this, and do this all over again for the next project
Sure, it’s possible to have a single master bibliography offline, and then every time you add a reference, to export the bibliography, and import it in your current projects. But this is a bit time-consuming, so I decided to find a way to automate the process a little bit. There are four steps to this:
Create a clean master bibliography in Dropbox
Enable syncing between ShareLateX and Dropbox
Set up script to copy master bibliography to ShareLateX folders
Schedule the script
Master bibliography with Jabref
The first step to a better system is to actually have a cleaned up bibliography file that you will WANT to use. I do this in Jabref, because it is as close to “just editing .bib files” as I can get, has everything I need, and is free/open source.
To finally create a single bibliography, I copy pasted the bibliographies from my MSc thesis, my PhD thesis and my papers since my PhD into a giant .bib file. This created lots of duplicates, but these can be edited in Jabref. This was quite neat, since it allowed to me to choose the “most cleaned up” version of the reference. This was a time-consuming process, but hopefully with this system I don’t have to go through it again. My master bibliography now lives in a Dropbox folder called “Bibtex”.
ShareLateX – Dropbox sync
The next thing that I needed is syncing between Dropbox and ShareLateX. Unfortunately this is a premium feature at this time.
If you have the syncing enabled, each paper lives in its own Dropbox folder, under Dropbox/Apps/ShareLatex. Here are mine:
Each of these folders has a .bib file. Although locally I could ensure that the projects link to my main bibliography file in Dropbox/Bibtex, I cannot do this in the cloud. (Or, there is a way to do this but I haven’t found it, which would render this blog post obsolete).
Copy bibliography to project folders with a script
My “genius” solution is to copy my master bibliography to the individual project folders. Of course I don’t want to do this manually – I want an “if this then that” solution that does it for me when the bibliography is updated.
In the end I settled for a slightly simpler solution of “copy every hour between 8 and 17” (when I’m most likely to be writing). Not as elegant, but (at least in Windows) it’s the 20% of effort that gives me 80% of results.
To copy files automatically, I used a Powershell script that says what to copy and where, and the TaskScheduler, which actually runs the script.
The Powershell script is called jobBibtex.ps1 and has the following contents (you can just create it with Notepad, but be sure to save it as “other” and not as a text file):
The script now has hard-coded in it which projects it should copy to. A better version would go through all the folders in Dropbox/Apps/ShareLatex – this is the next improvement on my list.
Schedule the copying script
The last thing is to schedule the copying script. I used this tutorial to get this done, but here is a short summary. From the start menu, search for the TaskScheduler program. Once there, go to “Actions” and “Create basic task”
In the menu that comes up next, you need to select a name for your task (“Copy bibliograhy”) and select a trigger, for example “Daily at 8AM”. As the action, select “start a program”.
The program that you are starting is Powershell.exe, and you need to add the following argument (replace the path with wherever you saved your .ps1 script:
Save the task and run it to see if your .bib files were really copied!
Clearly this is not a perfect solution.
This is a one-way sync, so editing a copied .bib file on ShareLateX will not reflect in your master bibliography. This is problematic for collaborative projects. My solution would be to have two .bib files in each project – one with the master bibliography, one with additional files your collaborators want to add (which you could later add to the master .bib file).
The projects which to copy to are now hard-coded in the script – it would be better if if the script would work for any new folders in Dropbox/Apps/ShareLateX
Copying every hour is too much, a “when updated” copy would be more neat
But, it is the 20% solution that gives me 80% of the results I wanted, and hopefully will save me time in the future.
Do you have similar solutions in place? Leave a comment below!