Firsts: designing an undergraduate project course

One of the new things I had to do during my first year on the tenure track was to design a course. I have not designed entire courses before, but this was a great experience that I learned a lot from, and even managed to integrate my research with my teaching. As I am writing about this in my teaching portfolio, I thought I could share some of the insights in a blog post as well, that could be helpful to others in a similar situation.

Goal

The goal was to design a project course on image analysis for first year students. As most courses at my university, this would be a 5 ECTS course and run for 8 weeks (+2 weeks for evaluation/exams). A project course meant that many of the learning goals focused on (already defined) project skills. My job was to create an assignment on which students could work together. A first year course meant that I could not assume a lot of prior knowledge of the students. I also had to align my course with the other existing project courses, connecting theory, modeling and experiments. And of course, I wanted to create a course that was fun.

Brainstorming

I started designing the course early on – as I started my job in February 2017, while the course would only start in November 2017. I used the other project courses in our department, and other courses I could find online as inspiration. I also searched for information about how to organize group projects, and what aspects of projects students like or dislike. I saved all of this Evernote and later used these notes during brainstorming.

During brainstorming, it became clear I wanted to add real-world components to the course, such as having a client for the assignment and gathering data. I also wanted to design the course in such a way that success was not too dependent on programming skills. So, I needed an idea which had all these components, and somehow involved analyzing medical images.

The project I settled on was extracting visual properties, like “asymmetry”, from images of skin lesions. Dermatologists look at such properties when making a diagnosis, and by automatically measuring such properties, we can design machine learning algorithms (which students will come across later in their studies).

Real-world components

I found a client for the assignment – the developers of the app Oddspot, which asks the user questions about the lesion, and then calculates a risk score. The developers could be interested in extending the app with imaging, and the students’ assignment was to investigate the possibilities. This way I had the basics for the theory – which features to measure in images, the model – an algorithm that actually does this, and the experiment – testing whether the measurements were effective.

I thought that another real-life component would be for the students to gather data. My first idea was to gather images of skin lesions with smartphones. But my own phone was not good enough to produce good quality images so I doubted this would work. Instead, I decided to use a public dataset from the ISIC melanoma detection challenge.

To still have a data gathering process, I asked each group of students to visually assess the features they were planning to measure with the algorithms. This way, even if a group would not be able to get their algorithms to work, they could still perform experiments – for example, by looking at interobserver agreement.

The project courses are assessed with a final report and a presentation. I decided to replace this traditional presentation with a Youtube video, aimed at a more general audience, such as prospective students. I thought this would allow for more creativity than a traditional presentation, but also build in some accountability, since the presentation could in fact be watched by other people.

As I was thinking of all these things, I was writing the guide that the students would get from me, to try to understand if any important information was missing, and filling in required documents related to the design of the course – for example, which learning goal would be assessed in which assignment.

During the course

Since this was a project course, I actually gave only one lecture to the students, where I talked about image analysis, measuring features in images, and of course explaining the assignment. After that, the students met in groups, together with teaching assistants (TAs). The role of the TAs in this case is to oversee the project skills part is going well – they are not required to have any background in image analysis and are not supposed to help the students with the content of the assignment. During these weeks, I would meet with the TAs and the study coordinator, who took care of all the logistics of these project courses, to discuss the progress of the groups. I made notes during these meetings, to take into account when updating the course next year.

After having read lots of “advice for tenure trackers” types of blog posts, I was afraid that teaching a large course would leave me overwhelmed with email. So, both in materials I gave to students and during the lecture, I asked the students to ask all questions related to the course content via discussions on Canvas. Of course I still got emails, but I redirected those students to Canvas and then answered their questions there, so that the answers would be visible to everyone.

What this system achieved was that (i) I didn’t get any repeat questions (ii) all students had the same information, so it was more fair and (iii) students could learn from each others questions/answers. Another advantage for me was that I would get a digest from all new questions in Canvas at the end of the day, so I could schedule times I would go through them, rather than multi-tasking during the day, as happens with email.

Another thing I have to emphasize is that a lot of the logistics were handled by the study coordinator, who found the TAs, checked which students were absent too often etc. Meanwhile, I could just focus on the content of course, which takes a lot of stress away from the experience of teaching for the first time. So, hats off to my department for setting it up this way.

After the course

The course grade consisted of project skills, which were assessed by the TAs, and the content part, based on the report and the Youtube videos. Although I gave general criteria for how I was going to assess these and made an “assessment matrix”, in the process I decided I needed a more detailed rubric to keep grades fair, so grading and then re-grading took quite a bit of time.

The students really surprised me with their Youtube videos (in Dutch), which were all very well done. I even tweeted this:

At the end I had a short meeting with each of the groups to give them feedback on their assignments and get more input for the course. For example, I asked them what they found the most surprising and the most difficult (of course, recording this in Evernote). I also brainstormed a bit with them how to update the assignment next year.

Next year

I’m happy to report that overall I got good feedback about the project. The students said they particularly enjoyed that it was a real assignment and not something that was already done many times. This is great, but of course also means that I will need to update the assignment each year, so that the students are building upon each other’s work, and not doing exactly the same things.

I worried about the programming part being too difficult. During the course, the students did find programming challenging, but at the same time it was clear they were figuring things out. And all groups did submit code which was of sufficient quality. Most students indeed complained about the level of difficulty in the course evaluations, although a few students commented that they liked having to figure it all out themselves. This is definitely something I will address next year.

Finally, I of course also received course ratings. I know I should take these as a grain of salt, since group projects probably get higher ratings overall, student evaluations are not correlated with learning, but… it still feels pretty great to have a success in the middle of all my rejected grants and unfinished papers.

Integrating research and teaching

Remember all those visual ratings the students had to do? I’m not sure I realized it at the time, but in fact, I had just crowdsourced a lot of annotations for medical images. I am now using these results in my current research, and recently I submitted a paper about it, where I acknowledge the students who took the course. Another real-world component?

Take-aways

My take-aways from this experience would be:

  • Take a lot of time for brainstorming
  • Find examples of other courses
  • Evernote is great for keeping track of ideas, feedback, etc.
  • Use the learning environment to reduce your email load
  • Think how large classes of undergraduates can still participate in your research
  • Having teaching support is absolutely the best and made this potentially stressful experience very enjoyable

Acknowledgments

I’d like to thank Josien Pluim for brainstorming about the course, Chris Snijders for participating with Oddspot, Rob van der Heijden for coordinating a LOT of things, Nicole Garcia, Jose Janssen and Nilam Khalil for administrative support, Maite van der Knaap, Femke Vaassen, Nienke Bakx and Tim van Loon for supervising the student groups, and last but not least, students who followed 8QA01 in 2017-2018.

Firsts: preparing for a lecture, part 2

“>Here I started preparing for my first lecture and describing my experiences. Now that the lecture is over, I have a few more things to share about the whole process.

In the end, I think I spent 20 hours preparing (not 30 as I estimated previously). At some point I felt I was satisfied with my story and I didn’t feel I could change anything until actually trying it out. In terms of the “how to”, I found a teaching course on Blackboard, and looked through the first teaching lesson. There are some useful tips there on keeping the attention level of the students high, and on structuring the lecture, such as providing summaries. With this in mind, I revised my slides one final time, for a total of around 30 slides for each 45 minute lecture.

During the lecture, a few things went differently than I originally planned. For instance, I included short summary/overview slides for transitions between different topics. However, every time I felt that I already explained this twice, so I did not put too much emphasis on it. In the end, it turns out that I should have done it anyway, against my intuition :). For other slides, I realized that I could have explained something in a different way, or linked the topic more to something students are already familiar with. I received some helpful tips about this from my supervisor, so I hope I can apply these in the future.

Another thing that didn’t go exactly as planned was the timing. I am very happy that I was using the presenter mode in Powerpoint where you can see the time and your notes on the laptop screen. I was too involved with the lecture to actually look at my notes, but the time was very helpful. I realized that I started too quickly, so later on in the lecture I tried to go a little bit slower. In the end, both lectures were finished in under 45 minutes.

Overall, I think the lecture was quite a nice experience! I was not too nervous about it, and although there were some silences (such as after a “Are there any questions about this”?), there was also student participation after some of my other questions and examples. That was very nice, especially because students started thinking of how to apply the method to their own PhD research. And that’s what the course is all about!

Firsts: preparing for a lecture, part 1

I’m very proud of it and very scared at the same time – next week I’m going to give a lecture for the first time. The lecture is a part of the Advanced Pattern Recognition course for PhD students and my own lecture will be about the dissimilarity representation and multiple instance learning – topics I should be familiar with 🙂

Right now I’m spending a lot of time in preparing for the lecture. I’m guessing that 30 hours will be a good estimate for how much time I will spend in total. Right now I will try to explain my progress and how many of these hours I am spending where.

I spent an hour or two searching for information on how to prepare your first lecture. I found helpful tips here and here. The main messages for me is: pick a few core topics and explain them well, rather than skipping over all the possibilities.

With that in mind, I started thinking about the actual content. Although the topic is very related to what I’m doing in my PhD, I want to talk more about the general techniques rather than the specific parts that I am doing. Therefore, I could not use the typical structure of my conference presentations. I started out with a mind map (or at least, a bunch of words with arrows between them) of both topics to see what exactly I would need to cover. In my head, I was already preparing the connections between different topics and thinking of nice examples, so in the end, this process costed me about 2 hours.

Then I looked at which topics I feel comfortable explaining (most related to my own research), and which topics I don’t have experience with / haven’t tried explaining to others. For instance, with a dissimilarity representation, there are two main possibilities to improve upon nearest neighbor classification: embedding the dissimilarities, or training a classifier in the dissimilarity space. In my research, I only do the latter, and although I understand the concept behind embedding, I don’t feel as comfortable with it. Yesterday I spent most of the day reading about it and at the same time trying to revise last year’s slides so I could actually use them in my own explanation. This turned out very time-intensive (+/- 7 hours), but also very helpful.

I still need to prepare the slides for my more “comfortable” topics, revise the whole story and practice. I’m not really into practicing the whole thing before presentations, but here I’m especially worried about the timing, because I have never talked for 1.5 hours before. Also, as my lecture is only on the fourth day of the course, I plan to attend the other lectures and see how the experienced people are doing it. So, probably I will revise a few things after that as well.

The last, somewhat more optional part, is to go over the exercises that “go” with my lecture. Because the content and slides that I’m using changed from the previous years, I have to check whether the exercises are still useful, and update them if necessary. I’m actually very looking forward to this, but I’m afraid I won’t have the time to come up with my own exercises, test the code, etc, so I might have to leave that for next time 🙂

To be continued!

%d bloggers like this: