Portfolio Learning - Themes from AMEE 2011

I attended many, but not all of the portfolio related talks at this meeting. In part, it's my duty to do so, because portfolios are a large part of my day to day responsibility. However, it's always been fascinating to me to see how this technique to promote reflective learning has been adopted in different ways around the world. At previous meetings I've been astounded to see the contrast between some countries, where the results of using portfolios to teach subjects like biochemistry (!) are presented, to others where portfolios are used to promote reflection on the larger issues of training and evolution of one's professional identity. And then there have been the North American programs, which by and large have a much less established use of portfolios within their educational systems.

This year, by contrast, I saw a much greater consistency in the discussion amongst the poster presenters and some of the abstracts, which seemed to be tackling some of the deeper, more fundamental issues in portfolio use in medical education. Here are the themes that I saw (of course, fully coloured by the lens I look through, as the director of a local nascent program).

1. Participating faculty need to be trained to support the goals of the portfolio program. Many presenters mentioned the disconnection between the usual set of teaching skills that many faculty members have developed and use regularly, and those necessary to support reflection or assess it. It is universally the case that portfolio programs require large amounts of faculty resources, both in terms of time and numbers of people. We certainly have found this to be the case in Toronto in our first year. In fact, when starting up a portfolio program, one is really starting two parallel educational tracks: one for students and one for faculty. There should be equal emphasis on training faculty as on educating students. This is very daunting when just setting up a portfolio is such a huge task!

2. Getting students engaged is problematic. A recurrent figure seems to be that 30% at maximum of students seem to participate at the level expected of them. I heard much discussion of how students don't seem to understand how good this is for them. I also heard quite a bit about the numerous factors which work against student engagement with a portfolio program. Of these, the fact that portfolios (especially the ePortfolio for Foundation year 1 in the UK) are assessed summatively (for progression) deserves its own theme below. Otherwise, a lack of coaching and examples, and a disproportionately high effort as perceived by learners relative to their benefit, were key themes. One gets the sense that getting learners to do this is like getting them to take their medicine, or eat their vegetables! We know it's good for them so we press on... I don't pretend to have any special answers to this, but my instinct is that the guidance of a mentor, or tutor with a longer term relationship with the student, may coax them along at first, and then if the light goes on in their heads, students may feel they can get more out of this type of work. But of course, we must pay attention to the other side of the ratio - reflection, when well done, is NOT easy, and growth is a struggle. Given the fully stuffed schedule that our learners must maintain, we must pay attention to whether our reflective assignments are too demanding in the overall learning context.

3. The a-word (assessment). The catch-22 came up again: if portfolios aren't assessed, they aren't done, since students don't want to do all this work and have no one look at it. On the other hand, many students resent having their private reflections assessed, and fear the consequences of being honest in their reflections since their faculty will see them. Some presenters showed data that students are happy enough to reflect, verbally and in person, but don't see the point or the need to create something written. The writing part seems like a make work project, which has little to gain for the student, apart from leading to... an assessment.

There are several important threads in this. If assessment is to be valid for the learner and for the program, it must be clear what is being assessed, and there must be a clear reason why this must be assessed. In the case of portfolios, simply asking students to compile their evidence of learning and reflect on it may not meet this standard, if the students have not experienced the benefits of rigorous reflection, and accompanying feedback. The "why" of it all may be really unclear to them. Likewise, the "what" of the assessment, that is the target of the assessment, must be made crystal clear by the program. In our case, we are specifically targeting the development of reflective capacity in relation to 6 of the 7 CanMEDS roles. We are doing this because we assert that developing reflective capacity as a practitioner is a crucial step, and that we believe that students should be able to indicate what they think a given incident means to them. This is a fundamental analytic skill, that is as essential to their professional career as a good history and physical examination. That's why we assess it.

The other key part of this is the idea that reflections may not have to be written. This is certainly true, as John Sandars showed at the previous AMEE meeting in Glasgow with his digital storytelling workshop. However, I argue that a reflection needs to be 'created', whether that creation is in textual or another form. A simple discussion is a good start, but to critically examine one's biases, assumptions, and automatic behaviors requires concentrated effort, and when thoughts must be committed to page (or .mp3, .jpg, .mpg, powerpoint etc) one is more likely to make the effort. We have found, in general, that students go quite a bit further in written portfolio sections than they do in conversation. So, because we value the ability to reflect on personal experiences, and have created an assessment for it, we insist that students create reflections in the same way that they have to create a written up history and physical examination for assessment of their clinical skills.

I hope you'll comment on the above. What do you think of these issues in portfolio and reflective learning?

Back home at AMEE in Vienna

Well, a 7 hour flight with a whole bunch of irritable children, a quick change of clothes and I arrived at AMEE2011 on Monday August 29.

This meeting is perhaps the pre-eminent international forum for sharing with and learning from medical education colleagues from around the world.  A welcoming environment is promoted and it's a great place to showcase your work and hear what others are doing in your field.  This meeting once again was true to form! 

Check the link to the picture gallery for a few highlights of AMEE.  I'll highlight a few key themes in subsequent posts over the next few days.