This article was downloaded by: [Aston University]

On: 30 August 2014, At: 20:50

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered
office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Innovation in Language Learning and

Teaching
Publication details, including instructions for authors and
subscription information:
[Link]

Whose learning is it anyway? Problem-

based learning in language teacher
development
a
Freda Miriam Mishan
a
School of Languages, Literature, Culture and Communication ,
University of Limerick , Limerick, Ireland
Published online: 07 Apr 2011.

To cite this article: Freda Miriam Mishan (2011) Whose learning is it anyway? Problem-based
learning in language teacher development, Innovation in Language Learning and Teaching, 5:3,
253-272, DOI: 10.1080/17501229.2010.548558

To link to this article: [Link]

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the
“Content”) contained in the publications on our platform. However, Taylor & Francis,
our agents, and our licensors make no representations or warranties whatsoever as to
the accuracy, completeness, or suitability for any purpose of the Content. Any opinions
and views expressed in this publication are the opinions and views of the authors,
and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content
should not be relied upon and should be independently verified with primary sources
of information. Taylor and Francis shall not be liable for any losses, actions, claims,
proceedings, demands, costs, expenses, damages, and other liabilities whatsoever
or howsoever caused arising directly or indirectly in connection with, in relation to or
arising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any
substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,
systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &
Conditions of access and use can be found at [Link]
and-conditions
 Innovation in Language Learning and Teaching
Vol. 5, No. 3, November 2011, 253
272

Whose learning is it anyway? Problem-based learning

in language teacher development
Freda Miriam Mishan*

School of Languages, Literature, Culture and Communication, University of Limerick, Limerick,

Ireland
(Received 29 June 2010; final version received 14 December 2010)

This article describes the piloting of a problem-based learning (PBL) approach in a

Downloaded by [Aston University] at 20:50 30 August 2014

teacher education context. Originating in medical education in the 1980s, PBL is

now applied in the teaching of a broad range of disciplines. While increasingly used
in teacher education, however, PBL has not been applied, to the author’s
knowledge, in the area reported on in this paper
language learning materials
development. Problem-based learning is rooted in constructivist philosophy, which
holds that knowledge is actively constructed within the mind of the learner and
influenced by his/her interactions with peers and with the environment. Further-
more, constructivism holds that learning is spurred by ‘the problematic’ (i.e.
cognitive conflict). In PBL, cognitive conflict is ‘concretised’, in that a real problem
is used to trigger the learning process. This article reports on the piloting of PBL on
a materials development module in a Masters in English Language Teaching
programme in Ireland. It presents the students’ and tutor’s reflections on the
approach. These are largely positive as regards the development of professional
skills especially teamwork, leadership and achieving compromise. It concludes
with recommendations for further research on the use of PBL in language teacher
education programmes.
Keywords: problem-based learning; teacher education; constructivist approaches;
language teaching; learner centredness; teaching materials

Theoretical framework for problem-based learning (PBL)

‘Problem-based learning appears to be more than a passing fad in education’
(Savery 2006, 17). The increasing popularity of problem-based learning (PBL) in a
range of pedagogical contexts today, indicates how well PBL reflects the current
pedagogical ethos, i.e. the shift of focus in pedagogy from ‘teaching’ to ‘learning’, and
the skill-building priorities of today’s independent, digitally literate learners. It is
also a factor of its solid foundation in educational philosophy, for PBL derives
out of the constructivist theories of the early twentieth century (identified with names
such as Dewey, Piaget and Vygotsky), which hold that learners actively build
(‘construct’) knowledge within, and as a factor of participation in, social environ-
ments. Constructivism views language and communication as the means through
which learners process and ‘make sense of’ incoming information. Learners are
seen as assimilating new information into already understood structures or sche-
mata (nobody is a tabula rasa), or accommodating their existing knowledge to new

*Email: [Link]@[Link]

ISSN 1750-1229 print/ISSN 1750-1237 online

# 2011 Taylor & Francis
[Link]
[Link]
 254 F.M. Mishan

information (altering or broadening known concepts so as to accommodate

unfamiliar ones), thereby ‘constructing’, acquiring knowledge. From the constructi-
vist perspective, therefore, learning is construed as ‘a social process of making sense
of experience in terms of what is known’ (Tobin and Tippins 1993, 8). The influence
of this philosophy on PBL can clearly be seen in the constructivist emphasis on
cognitive conflict (‘the problematic’, Dewey 1938) for stimulating thought and
learning: ‘the educator who associates difficulties and effort with increased depth and
scope of thinking will never go far wrong’ (Dewey 1940 in Garforth 1966, 177).
Thence comes the integration of self-reflection, ‘an individual’s attempt at objectively
and explicitly representing reality’ (Rieber 1993, 198), as an essential element of PBL.
Problem-based learning can thus effectively be construed as ‘philosophy in action’,
with ‘the problematic’ concretised as the problem that is at the heart of a PBL
activity, ‘social interaction’ formalised as the PBL group (see ‘Pedagogical frame-
work’ below). The correspondence of the key principles of constructivism to the
Downloaded by [Aston University] at 20:50 30 August 2014

characteristics of PBL can be schematised (see Table 1).

Pedagogical framework for problem-based learning (PBL)

The pedagogical framework for PBL involves, self-evidently, a suitable problem (or
‘trigger’ as it is called in some of the literature). The design of the problem/trigger is
vital to the success of the PBL activity since the metaphorical ‘space’ between the
problem and its solution is where the learning takes place. This is the space in which
an accommodation is reached between existing and new information; in other words,
the zone of higher order thinking skills, most particularly, critical thinking and
problem solving.1 In order for this learning space to be accessed, the main thrust of
the PBL problem has to be the stimulation of these higher order thinking skills. With
this in mind, one aspect that is frequently stressed in the literature (e.g. De Simone
2008, 182; Weiss 2003, 27) is authenticity; this is seen as essential for three main
reasons. Most fundamentally, as has been argued elsewhere (e.g. Mishan 2005, in
relation to language learning tasks), authentic tasks, in the sense of being ‘real-life’,
are far more motivating than contrived ones and motivation is, of course, crucial to
learning. Secondly, because PBL is largely chosen as an approach to use in practical
disciplines, the problems need to touch on students’ current or future experiences
(e.g. Weiss 2003, 27
8). Thirdly, only authentic problems can generate the ‘fuzzy’
solutions usually reached in real life. This brings us to an essential characteristic of
the PBL problem, that it is ‘ill-structured’ or ‘ill-defined’ as it is variously termed
in the literature (see, for example, De Simone 2008; Savery 2006; Weiss 2003); that is,

Table 1. Constructivism and problem-based learning.

Principles of constructivism Characteristics of PBL

Knowledge is actively constructed by the learner Learner-centred

Cognitive conflict (‘the problematic’) stimulates learning ‘The problem’
Learning is a social process Group work, assigning
of roles
Learning depends on use of language in communication Group work
New knowledge is built on the foundations of previous Investigation, research
learning (‘accommodation’ and ‘assimilation’)
Self-reflection Self-reflection and
peer-assessment
 Innovation in Language Learning and Teaching 255

one in which the initial scenario does not necessarily provide all the information
needed to develop a solution, and there is neither one correct way to solve the
problem nor one ‘correct’ solution
hence, the solution is ‘fuzzy’.
Operationally, PBL involves students working in small tutorial groups with
minimal and, at times, no tutor intervention. The tutor’s role is classically trans-
muted to that of ‘facilitator’, guiding the group discussion ‘from the side’ by
using stimulating questions or prompts when s/he deems they are required. A crucial
aspect of PBL group meetings is the assuming or assigning of specific roles: ‘the
discussion leader’, who effectively chairs the meetings; ‘the recorder’, who minutes
the discussions; and ‘the observer’, who observes and evaluates the group dynamics.
This structuring is seen as so essential because it concretises the theoretical
underpinnings of PBL
the importance of student self-direction, of the processing
of knowledge within a group context and of reflection upon the knowledge-acquiring
process. Students are presented with a problem pertaining to their discipline, then
Downloaded by [Aston University] at 20:50 30 August 2014

working within the group as structured above, they identify what they need to do to
solve the problem in terms of learning and set up an action plan in which students are
assigned separate specific tasks to be done individually or in sub-groups. Tasks might
involve sourcing materials from the library, web or databases, carrying out surveys,
etc. Once the required research has been done (or the first stage of the research), the
whole group reconvenes to share their information, peer teach and (if necessary)
iterate this cycle until a solution is reached. The final stage of the PBL process is a
reflection on what has been learned and on individual students’ contribution to the
process. Reflection is a key aspect of PBL and is built into the framework in the form
of continuous assessment instruments, constituting self-reflection and peer-reflec-
tion. These can be done via questionnaires, completed individually, and/or in focus
groups, tutorial groups and so on. ‘Group reflection’ helps cultivate mutual trust and
support, benefiting the overall group dynamic (Helelä and Fagerholm 2008). Where
students on PBL programmes/modules require to be graded, self-reflection can be
built into the grading of student performance (as it was in the study reported in this
paper), with the tutor evaluating the student’s ability to critically reflect on his/her
achievements and on those of others. This might, however, be seen as anathema to
PBL by PBL purists who view assessment as being under student, not tutor
ownership (see, for example, Helelä and Faberholm 2008, 22; Savin-Baden 2004).
With such emphasis placed on structure for the successful operation of PBL, it is
clearly essential to familiarise students with both the theoretical and pedagogical
framework of the approach. In the literature, this is stressed as a sine qua non: Savery
notes that students new to the approach require ‘significant instructional scaffolding’
(Savery 2006, 15; see also Helelä and Fagerholm 2008). Schwarzt, Mennin, and Webb
(2001) present PBL case studies whose success was shown in many cases to be
dependent on students appreciating the concept.

Pedagogical context of the study

Because it is effectively as much a philosophy as a pedagogical approach, PBL is
remarkably versatile: it is used at all educational levels, from elementary to third
level, including professional development and in the teaching of an ever-increasing
range of disciplines. Examples include engineering, literature, business, physics,
software development and medicine, where it originated (e.g. Barrows and Tamblyn
1980). In teacher education (TE), an area in which PBL is arguably particularly
 256 F.M. Mishan

suitable, it has become increasingly popular since the 1980s (De Simone 2008).
However, in areas such as medical education, the effectiveness of PBL has been
stringently assessed, leading, in many cases, to its whole-scale adoption at curriculum
level, while in teacher education there appears to be less evaluation of the impact
of PBL, with, typically, one instructor piloting the approach using a simple pre-/
post-test design (De Simone 2008, 181). Furthermore, one area of TE in which
PBL is conspicuously under-represented, as far as the author is aware, is language
learning materials development.
Language learning materials development is a fairly new but robust discipline
within language teaching and learning, developing as a field in its own right around
the mid-1990s. Dedicated postgraduate programmes in language learning materials
development were set up at that time (e.g. at the University of Luton and the
Metropolitan University of Leeds), with the International Graduate School of
English in Seoul’s MA ELT materials development programme a later addition in
Downloaded by [Aston University] at 20:50 30 August 2014

2002. The subject is now offered as a separate module within a number of TESOL
programmes in Britain and Ireland including the MA ELT at the University of
Limerick (UL). As has been repeatedly stressed in the literature in the field (e.g.
Mishan and Chambers 2010, i), materials development should not be considered an
‘optional extra’ in language teacher education. Materials are an intrinsic part of
language teaching and all teachers are materials developers in so far as they are
continually evaluating, adapting and/or producing materials to suit their individual
teaching contexts (Tomlinson 2003, 1). This continuum from evaluating to producing
materials, underpinned with L2 acquisition and pedagogical theory, typically forms
the basis of language learning materials development courses.
The materials development module within the MA ELT programme at UL was
first offered in autumn 2007 with its core learning outcomes as follows:

(1) To raise critical awareness of language teaching materials

(2) To encourage critical application of L2 acquisition theory to practice
(3) To promote creative engagement with language teaching materials
(4) To foster and promote reflective practice
(5) To foster ability to work collaboratively
(6) To contribute to professional development as language teachers

The module was offered to students with a minimum of three years’ teaching
experience; this was based on common practice in the field, as it is felt that
familiarity with language learning materials and teaching contexts are essential for
the informed development of learning materials.2 The module was offered for two
hours per week, for the duration of the 12 week semester. After two years of running
the module on this basis, a PBL approach was adopted, with the practical objective
of making the most of the limited time assigned to it and the pedagogical one of
better accomplishing its core learning objectives. Furthermore, as this was the first
implementation of PBL on the UL MA ELT programme, a third, broader aim was to
explore the viability of PBL on the programme in general. It was therefore decided to
integrate into the running of the module an exploratory study on the use of PBL in
this language learning materials development context. Viability was assessed in terms
of fulfilment of the learning outcomes as stated above.
 Innovation in Language Learning and Teaching 257

Methodology
The study drew chiefly on two elements. The first consisted of the instruments
that constitute an integral part of the PBL process, the assessment by students of
their own learning through PBL, in the form of (a) self- and (b) peer-reflection via
self- and peer-reflection forms (Appendices 1 and 2, respectively).3 Also included in
this element was (c) the students’ evaluation of the materials they created as the
solution to the problem via a materials evaluation form (Appendix 3). The second
element of the study was the students’ assessment of the PBL process per se, via an
anonymous online survey mechanism (Appendix 4). On to all this was overlaid a
third element, the tutor’s informal perceptions of the process.
Problem-based learning as an instructional approach is inherently flexible
it
could hardly be implemented across such a spectrum of disciplines and educational
levels if it was not
and the framework devised for this materials development
module was thus able to be tailored for the context. The model devised consisted
Downloaded by [Aston University] at 20:50 30 August 2014

of a staged transition from tutor-led to learner-led mode: this allowed for, at the
start of the course, a fairly tutor-led introduction to the theory and practice of PBL
(which was a more or less new concept to all the students), followed by staggered
and decreasing amounts of ‘tutor-input’ as students gradually took control of their
own learning and made the approach their own. Following two ‘compulsory’ tutor
inputs, therefore, on PBL itself and on ‘L2 acquisition and materials development’,
the problem (see Figure 1) was set, and a repertoire of optional tutor-inputs offered,
to be taken up if and when students felt they needed them. These included materials
evaluation, task-based learning, rubric writing and the authorial voice and use of
authentic texts.
The cohort for the 2009
2010 MA ELT materials development module consisted
of five mature students, male and female, aged between mid-20s to mid-40s, all of
whom had over three years’ ELT teaching experience in places such as Ireland, Japan
and Korea. It was decided that the group of five could comfortably work together as
a conventional PBL working group.

Within a specific English language teaching context that lacks relevant language learning
materials, you are asked to work as a group to produce a sample unit of materials.

This sample unit is to be representative of a complete coursebook for the teaching context, so
should be contextualised within the coursebook.

The sample unit needs to be backed up by (a) teachers’ notes and (b) a language learning theory
rationale.

The sample unit should be as professionally produced as possible.

The sample unit is to be presented in class at the end of the module (week 12), after which it can
be revised as necessary and hard copy submitted at end of week 15.

Figure 1. The problem.

 258 F.M. Mishan

When devising the problem, achieving a balance between not ‘leading’ students
(leaving the problem open and ‘ill-defined’) and giving them more direction in the
form of greater detail, proved to be a tricky decision. It was decided to err on the
side of openness and not specify the teaching context for the materials problem,
hence ‘Within a specific English language teaching context that lacks relevant
language learning materials’ (see Figure 1), taking a gamble that, given the variety of
the students’ backgrounds, the context which the students decided on would
inevitably be the unique one they had in common, viz. Erasmus students learning
English at UL.

Reflection and assessment

The importance of ‘learning from learning’, reflecting on the learning process, was
stressed to students as part of the preliminary introduction to PBL. This was the
Downloaded by [Aston University] at 20:50 30 August 2014

rationale for building reflection into assessment of the module. Students were
required to complete a self- and peer-assessment (of one randomly assigned peer) at
mid- and end of semester. These assessments were evaluated by the tutor on the basis
of evidence of the students’ capacity for:

Objectivity

Introspectiveness

Critical thinking (e.g. questioning, reasoning)

Professionalism

Students were also asked to assess the materials they had designed (as the ‘solution to
the problem’, see Figure 1), on the basis of a set of provided criteria (in the materials
assessment form, see Appendix 3) such as their basis in L2 acquisition and language
learning theory, their originality, and how well they matched the needs of their target
learner profile. As above, the tutor assessed the students’ ability for critical
objectivity and so on in this exercise.

Treatment of data
The data was examined, firstly, for evidence of how well the module learning
outcomes were achieved via the PBL approach. One of the core outcomes, as noted
above, is professional development as language teachers. Professional development
skills are in effect ‘unpacked’ in the other module objectives to include the ability to
work collaboratively, which includes teamwork, giving and receiving feedback and
reflective practice.
In order to assess the correspondence between the PBL approach and intended
module outcomes, therefore, these were cross-mapped to the features of PBL
characterised in Table 1, and extrapolated as a ‘strand’ (see last column in Table 2)
for ease of analysis and reference.
The findings of the pilot study are presented below. These draw on the three
assessment instruments itemised in the Methodology section above. These provided
qualitative and quantitative data, which were treated as follows. The assessment
forms were analysed on a keyword/phrase basis to extrapolate evidence to go under
 Innovation in Language Learning and Teaching 259

Table 2. Extrapolating criteria for assessment.

Characteristics of PBL Learning outcomes Strand

Group work, assigning of 01 Ability to work collaboratively TEAMWORK
roles
‘The problem’ 01 Critical awareness of language THE MATERIALS
teaching materials
01 Critical application of L2
theory to practice
01 Creative engagement with
materials
Investigation, research, 01 Critical application of L2 THE MATERIALS
learner-centredness theory to practice
Self-reflection and peer- 01 Reflective practice ASSESSMENT &
assessment 1 Ability to give and receive REFLECTION
feedback FEEDBACK
Downloaded by [Aston University] at 20:50 30 August 2014

the strands identified in Table 2. These findings were triangulated with the
quantitative data, which was calculated on the basis of ratings marked on the
5-point Likert scale on the self- and peer-assessment instruments. With the small
number of students involved in the study, however, the quantitative data does need
to be viewed with caution.

Findings
The students’ ‘solution’ to the problem was an online course book for students on
Erasmus exchange programmes staying in Ireland, this can be viewed at: [Link]
[Link]/
The findings from the data analysis are discussed below under the strands
labelled in Table 2, supported by participants’ feedback and comments.

Teamwork
The issue of teamwork was particularly acute in this study. Predictably perhaps,
students tended to rate themselves better than others in this area. In 70%4 of the
assessments, students rated themselves good at teamwork, while rating their peers
lower. On analysis of the qualitative data, it emerged that this was mainly due to the
failure to implement a vital structural aspect of PBL, team roles. As the tutor wished
to allow the process to be as learner-led as possible, she described and recommended,
but did not insist on, traditional PBL roles. In the event, the students eschewed
adopting roles formally. The lack of a discussion leader forced the self-appointment
of leaders and, interestingly, the two comments below came from two different
students:

I was a bit apprehensive at first to take a very direct leadership role within the group

however, leadership was needed and I eventually took a more central role.

I feel that I was able to act as a leader of sorts.

 260 F.M. Mishan

Lack of leadership and the assuming of the other set roles affected the ‘problem-
solving’ process. This was stated directly and in some instances repeatedly in 80% of
the assessments, as exemplified in these quotes:
Without having anyone step up as the leader was beginning to stunt our creativity.

One of the group’s key problems is that meetings have tended to finish without any
agreement on work to be done.

These outcomes, directly stated in the comment, ‘I have learned a lot about group
dynamics, the necessity of group roles’, strongly vindicate the PBL insistence on
group structure.
Group dynamics, always organic by their nature, were uneven within this
unstructured group. This was expressed in one way or another by every student in at
least one assessment:
Downloaded by [Aston University] at 20:50 30 August 2014

The group has been less than effective in our organisational approach.

It can be difficult to communicate and progress within a group.

Group dynamics did improve as the semester went on:

Our group became more comfortable with each other as the project progressed and we
were more open with our opinions and feelings.

Students recognised when the dynamic broke down, identifying the deficiencies of
other peers as team players:

B5 did not share his/her6 new-found knowledge [. . .] with the group.

C sometimes decides and implements things without consulting others, s/he could
become more of a team player.

These problems notwithstanding, by the end of the semester, four of the five students
commented positively on the evolving of the group dynamic; ‘working with peers
towards a group goal’ as one student expressed it. Going further, teamwork was
recognised as ‘a crucial tool for personal development’ (this was stated directly by
two of the five students).

Feedback
One of the mechanisms within the classic PBL framework, and particularly in this
context where students were designing language learning materials, is peer feedback.
Groups need to develop a functioning feedback-revision loop in order to arrive
at their problem’s solution. As with the development of a group dynamic, a
steep learning curve was observed in both giving and receiving feedback. This was
expressed directly in 60% of the assessments, typified by this quote:

I think there is an underlying lack of confidence both collectively and individually that
stops [us] from being critical and attempting to give feedback.
 Innovation in Language Learning and Teaching 261

This was a cause of disappointment: ‘As a result of the general reluctance to give
feedback, I didn’t learn from it as much as I’d hoped.’ It was hard for some students
to compromise:

It is complicated working in a group because some people’s ideas will always clash with
somebody else’s.

C occasionally didn’t take too kindly to suggestions.

Some, however, developed the necessary flexibility and recognised the advantage of
feedback:
I’ve learned that there is never only one way of doing things and how to combine various
approaches.

Receiving feedback has enabled me to begin to question my own approach more.

Downloaded by [Aston University] at 20:50 30 August 2014

I have given feedback on suggested activities and research done by my colleagues and
this has given me a greater insight into their approach and motivations.

This upward learning curve was reflected in the quantitative data (see Table 3 and
Table 4).

Assessment
Assessment was both a student issue (as students had to carry out self- and peer-
assessments) and one for the tutor (i.e. assessing students in order to award grades
for the module).

Self-assessment
The students’ own self-assessments were carried out mid- and end of semester as
stated above. Self-reflection was a skill discussed and practised in other modules on
this programme, so it was not thought necessary to give specific training for this
module. This proved to be an erroneous assumption. One student stated:

It seemed like the lecturer was looking for us to go through some process of reflection,
particularly for the self-assessment. It wasn’t made all that clear what the process was,
and what the lecturer wanted to see evidence of.

Countering this, the self-assessments did show a capacity for self-reflection graded by
the tutor on a range from excellent (achieved in 70% of the assessments) to poor
Table 3. Self-evaluation ratings for giving feedback.
Feedback: giving feedback to peers

Likert scale 0 1 2 3 4 5

Did I give feedback and learn from it? Semester 1 Number of scores 2 3

Semester 2 3 2
 262 F.M. Mishan

Table 4. Self-evaluation ratings for receiving feedback.

Feedback: receiving feedback from peers

Likert scale 0 1 2 3 4 5

Did I receive feedback and learn from it? Semester 1 Number of scores 1 1 3

Semester 2 1 2 2

(10%), on the basis of the criteria listed in the section Reflection and assessment above.
Among the main positive outcomes was the development of critical thinking skills:
I managed to rationalise my many ideas and edited as appropriate to the specific task.
Group discussion facilitated my critical-thinking skills and gave me a setting where
I could reason with my ideas and categorise them into different areas. I learned to think
in a more critical manner and eliminate unsuitable suggestions, something I found
Downloaded by [Aston University] at 20:50 30 August 2014

difficult at the outset.

Self-reflection was often a locus for recognising and, in most cases, attempting to
remedy, personal responses that were not felt to contribute to progress:
In group settings, where I am enthusiastic about the topic or feel excited about the
process, I tend to take a lead role. I have tried to step back from leading the group this
time and be an equal part of the whole. However I feel like I have overcompensated and
often end up frustrated if a member of the group is completely off task or not
contributing to the groups ideas/concepts.

I have many ideas but I need to rationalise them, that is, see if they really can be used in
the unit and if they facilitate learning. This is where group discussion comes in.

Peer-assessment
The students opted to be assigned which peer to assess on a random basis, with
nobody assessing the same person who was assessing him/her and a different peer
was assessed at mid-/end of semester. Students did not see the peer-assessment done
on them. There was a noticeable increase in the length, and correspondingly, amount
of detail provided in the peer-assessments from the first to the second experience of
doing these mid- and end of semester (with a jump from a good to excellent grade
awarded for 80% of the assessments). This was thought to indicate an increase in
inter-group familiarity and deepening of feelings ranging from appreciation to
resentment as the group progressed their project, as well as improved reflection skills.
Due to the nature of ‘the problem’, the development of language teaching
materials, creativity was an area pinpointed for comment in the peer-assessment
form. Both mid- and end of semester forms produced similar results, in which
students’ peer ratings and comments fell evenly along a continuum from ‘very
creative’ to ‘weak’, viz.:

Mid-semester

very creative ! a creative person ! quite creative ! would benefit from a bit more
creativity ! lacks creativity
 Innovation in Language Learning and Teaching 263

End of semester

very creative ! is creative ! some creative ðtasksÞ ! finds it hard ! think outside
the box ! X’s main weakness

Another area that drew a range of student comment from positive to negative
was the amount and type of responsibility their peers assumed within the group:

M has carried the weight of some of the weaker members and done a good job keeping
the group on task.

I would commend N on his/her ability to stay focused and guide others to do the same.

P often facilitates by getting the group back on track and focussed on our overall goals.

While Q was keen to organise meetings, I felt his/her contribution was somewhat
Downloaded by [Aston University] at 20:50 30 August 2014

limited.

R was consistently in the middle of the road throughout the project.

When S realised that others were not contributing sufficiently, s/he took it upon him/
herself to do more, which in my opinion is not the correct approach.

As can be seen from this range of comments, students rated their peers diversely, with
the most common overall rating as 3 on the Likert scale (32% of all ratings of all
factors on the peer-assessment form), followed by 4 (28.5%) and 5 (17%).

Tutor assessment
From the perspective of the researcher/tutor piloting a PBL approach in a new
discipline, the main problem area was assessment. This was not unanticipated, as
assessment appears to be a controversial concern in PBL (Savin-Baden 2004, 223; see
also Table 5). Basically, the module in which PBL was being piloted needed to be
graded, like the vast majority of university modules. The issues here were, firstly, fair
grading of the students’ contributions, viz. their self- and peer-assessments and the
‘solution’ to the problem, in this case, the teaching material produced and secondly
the proportional weighting of these. The weighting used in this pilot was as follows.
The heavy weighting given to the tutor’s assessment of the material produced (50%),
reflected the overarching aim of the module, viz. the cultivation of materials
development skills.
The self- and peer-assessments were graded according to the same criteria stated
on the students’ forms (see Appendices 1 and 2); these ranged from assessment of
contribution, to capacity for critical thinking, and grading was broken down evenly
between each criterion (i.e. approximately 10 percentage points each). The peer
assessments were judged according to each student’s capacity for insight and analysis

Table 5. Assessment breakdown.

Self-assessments 2 @10%  20%
Peer-assessments 2 @10%  20%
Materials assessment 10%
Lecturer’s materials assessment 50%
 264 F.M. Mishan

in evaluating his/her peer’s contributions. One student’s assessment of another did

not contribute to the latter’s grade.
The materials assessment instruments were the same for students and tutors (see
Appendix 3); criteria ranged from ‘student profiling and match to materials’ to
‘evidence of critical application of L2 acquisition/materials development theory and
practice’. Considering their investment in it, students understandably found it hard
to objectively evaluate their product. One criticism levelled by the tutor, the slight
unevenness of tone and structure of the sub-units comprising the full unit of
materials produced, was spotted by some students but perceived as an advantage:

The fact that the materials designed by different people [. . .] gives the material a level of
variety [. . .] (central to good lessons).

Our different styles and approaches [. . .] create an innovative, unique course unit.
Downloaded by [Aston University] at 20:50 30 August 2014

Only one student’s opinion concurred with that of the tutor: ‘It would have been nice
to see more consistency with their content and structure.’
Notwithstanding certain criticism such as the above, the materials designed in
this pilot were awarded a high grade. As this was worth 50% of the final grade, this
had the effect of inflating the students’ grades. In the subsequent running of this
module using PBL, this will be reduced and the weighting of the self- and peer-
assessments will be adjusted upwards so as to better reward student reflection.

The materials
A main concern of language learning materials development, as expressed in this
module’s learning outcomes, is that they have sound foundations in L2 acquisition
theory. With this in mind, the students’ own materials evaluation forms (which
offered their introspections on how far they had tried to apply L2 acquisition theory)
were analysed, as well as the materials themselves (the materials also contained a
section on this as requested in the problem remit).
Influenced by such theories as Schmidt’s ‘noticing hypothesis’ (1990), an
inductive approach to learning seemed to have been favoured:

We have tried, and in my opinion generally succeeded, in producing materials that allow
students discover for themselves, rather than simply presenting them with information.

We adopted a TBL approach to allow students to discover for themselves.

Schmidt’s noticing theory influenced our decision to include awareness-raising tasks in

our unit.

There was a clear influence of other classic L2 acquisition theorists, most notably,
Krashen (1985) and Gardner (1983):

The content of the materials is comprehensible while also challenging the students at the
same time. So we incorporated [‘i  1’] from Krashen’s input hypothesis.

We also made our course unit interactive by adopting a friendly voice. This friendly
voice and interactive, engaging aspect of the unit are meant to lower the students’
 Innovation in Language Learning and Teaching 265

affective filter, making them feel at ease. Krashen and other theorists have highlighted
the need to cater for students’ emotional state, not just their cognitive needs.

We also reflected upon the various learner styles and characteristics
such as visual
learners, and Multiple Intelligences.

The most favoured language learning approach was Task-Based Learning (TBL) as
exemplified by this statement:
We designed certain activities to [. . .] allow students to concentrate on a particular task
rather than concentrating too much on language accuracy. TBL is important since it has
been proven that students can retain and absorb 90% of what they do themselves,
through heuristics.

The ‘L2 acquisition rationale’ included in the teacher’s area of the materials
designed,7 supported these insights, and underpinned the materials themselves.
Downloaded by [Aston University] at 20:50 30 August 2014

These aspirations to appeal, self-directedness and discovery learning, can be

perceived in these materials; in their flexibility and non-directed aspect, in
their lively and attractive interface and in the authenticity and variety of the genres
used.

Summary of findings and analysis

The main criteria for success of the pilot were, as stated above, the degree to which
the module objectives were fulfilled. The core module objectives (in that these
incorporated the rest) were taken as encouragement of ‘creative engagement with
language teaching materials’, ‘critical application of L2 acquisition theory to
practice’, and ‘professional development as language teachers’.
The quality and originality of the materials produced, available on [Link]
[Link]/, substantiate achievement of the first core objective. Fulfilment of the
second, ability to critically apply L2 acquisition theory, was also seen as successful.
Students consciously evaluated and applied learning theories and teaching
approaches that they had learned in other programme modules as well as this one.
‘Cross-over’ from other modules was an assessment criterion in all the assessment
instruments, it was one in which students all scored themselves highly and which they
detailed in the materials assessment. Furthermore, analysis of the materials
themselves revealed their theoretical underpinnings.
Professional development skills, as ‘unpacked’ in the other module objectives,
include the ability to work collaboratively. These come particularly to the fore in
PBL, as can be seen from students’ appreciation of the opportunity to develop
teamworking skills: ‘I can say now that I have improved my collaborative skills’. A
weak area here, however, was establishing group leadership, as discussed above,
but it is surmised that this might have been avoided had PBL structures been
adhered to.
Another professional skill figuring in the module objectives is reflective practice,
and a good result was anticipated here, given the congruity with PBL aims in this
area. However, a weakness was perceived, by both students and the tutor, in reflective
skills, as noted above, with at least one student feeling s/he needed more guidance in
the reflective process.
 266 F.M. Mishan

Discussion
This pilot was in many respects strikingly successful. The students ultimately cohered
as a group and worked solidly and independently towards their problem’s ‘solution’,
in this case, the production of a unit of online language learning material. The
quality of the materials was of a substantially higher standard than in previous years
(although, it needs to be taken into account that this was a group project and in
previous years the comparable final assignment was an individual one). Students
recognised that the approach made them exercise essential professional skills, such as
teamwork, giving and accepting feedback and compromise. The pilot also brought to
light areas that need review for subsequent running of the module within a PBL
framework.
Assessment and grading are a thorny issue in PBL (see Savin-Baden 2004);
indeed, as Savin-Baden points out, the requirement of educational establishments to
grade students’ efforts on an individual basis undermines the whole ethos of PBL as
Downloaded by [Aston University] at 20:50 30 August 2014

a ‘group approach’. On the one hand, PBL encourages students to be stakeholders in

the assessment process (Savin-Baden 2004, 231
2), on the other, the rug is effectively
pulled from under them, as the tutor has final say in the awarded grade. This might
be seen as an example of trying to fit square pegs in round holes. Nor will it be
resolved until assessment in educational institutions is aligned to PBL rather than in
conflict with it. One way to reduce the impact of this in the current context is to
weight the grading more heavily towards the student self- and peer-assessments,
where they can be said to best display the skills and knowledge gained during the
PBL process. However, the ultimate responsibility for grades for these assessments
still defaults to the tutor.
In the area of self-reflection some people are naturally more introspective than
others. However, this pilot revealed the importance of integrating guidance and
structure for self-reflection, both as regards acquiring content and learning skills.
This is anticipated in the literature: ‘teachers should model reflective thinking
throughout the learning process and support the learners in reflecting on the
strategies for learning as well as what was learned’ (Savery and Duffy 2001, 6).
This pilot diverged from its PBL framework in one vital respect, the non-
imposition of the group roles. Participants perceived the impact of this on the speed
of progression of the group’s work, ‘there is little or no resolution at the end of
meetings, and as such, there is no outcome to understand and apply’ and it thus
‘stunted creativity’. It meant that a leader had to emerge organically; in the event,
leadership logically fell to the person who was building the website platform for the
materials. In the literature on PBL, the importance of group roles is an often-stressed
aspect of the PBL framework (e.g. Azer 2004; Tipping, Freeman, and Rachlis 1995).
This is confirmed by what transpired in a project where group roles were not
imposed. On the other hand, one of the attractions of PBL is that it is an intrinsically
flexible approach that can be applied successfully across educational levels and
disciplines. In some disciplines, language teaching, for example, traditional PBL roles
tend not to be used (e.g. Forrester and Chau 1999; Mathews-Aydinli 2007) or are
replaced with roles based on language proficiency level (as in Mathews-Aydinli
2007). What unifies PBL programmes whatever their incarnation is the educational
philosophy underpinning them.
Despite the small scale of the pilot described here, it is thought that its outcomes
are promising enough, particularly in its positive effects on the development of
 Innovation in Language Learning and Teaching 267

professional skills, for it to constitute a useful contribution to the research into PBL
in teacher education and its niche area, language learning materials development.
Indeed, with this in mind, larger scale research might be considered on the use of
PBL on full materials development programmes and on language teacher education
programmes in general. A start is being made on the University of Limerick MA
ELT, where, as a follow-up to this pilot, PBL is to be trialled on other modules, as
well as being re-run on the materials development module, revised according to the
findings of the study.
The last word on the project, goes, fittingly, to the students:

Over the course of 15 weeks my enthusiasm for PBL has not waned.

I think the nature of the course lends itself well to this approach.
Downloaded by [Aston University] at 20:50 30 August 2014

Notes
1. Reference should be made here to Bloom’s Taxonomy (1956), which is classically used to
categorise thinking skills on a level from one (knowledge memorisation, recall, etc.) to six
(evaluation).
2. This is currently being revised as some materials development programmes are now open
to students with less teaching experience.
3. These were adapted from self- and peer-reflection forms produced by Helelä and
Fagerholm (2008), freely accessible online at [Link]
(specifically [Link] and [Link]).
4. Percentage figures given here and below are based on analysis of the four assessment
instruments (two self- and two peer-assessments) plus the data gathered from the overall
anonymous feedback (see Appendix 4), where relevant.
5. Participants’ names have been replaced by letters on a random basis.
6. His/her, etc. used throughout to disguise gender in the interests of anonymity.
7. The L2 acquisition rationale can be seen at: [Link]
[Link]
8. This was provided on [Link] in the interests of anonymity as recommended
by the University of Limerick Faculty of Arts, Humanities and Social Sciences Ethics
Committee (FAHSSREC).

Notes on contributor
Freda Miriam Mishan is course director and lecturer on the Masters in English Language
Teaching at the University of Limerick, Ireland. She was previously an IRCHSS (Irish
Research Council for the Humanities and Social Sciences) post-doctoral scholar in the School
of Languages, Literature, Culture and Communication at the University of Limerick. Her
publications include chapters and journal articles in the areas of problem-based learning,
technology and language learning materials development, as well as two books, Designing
Authenticity into Language Learning Materials (Intellect 2005) and Perspectives on Language
Learning Materials Development (an edited co-authored volume, Peter Lang 2010). Her
current research focuses are problem-based learning, language learning materials development
and technology and language learning.

References
Azer, S. 2004. Becoming a student in a PBL course: Twelve tips for successful group
discussion. Medical Teacher 26, no. 1: 12
5.
Barrows, H.S., and R.M. Tamblyn. 1980. Problem-based learning: An approach to medical
education. New York: Springer.
 268 F.M. Mishan

Bloom, B.S. 1956. Taxonomy of educational objectives: Classification of educational goals:

Handbook I: Cognitive domain. New York: McKay Publishers.
De Simone, C. 2008. Problem-based learning: A framework for prospective teachers’
pedagogical problem solving. Teacher Development 12, no. 3: 179
91.
Dewey, J. 1938. Experience and education. New York: Macmillan.
Forrester, V., and J. Chau. 1999. Current developments in problem based learning within the
Hong Kong Institute of Education. In Implementing problem based learning project:
Proceedings of the first Asia Pacific conference on Problem Based Learning, ed. J. Marsh,
201
8. Hong Kong: The University Grants Committee of Hong Kong, Teaching
Development Project.
Gardner, H. 1983. Frames of mind: The theory of multiple intelligences. New York: Basic.
Garforth, F. 1966. John Dewey: Selected educational writings. London: Heinemann.
Helelä, M., and H. Fagerholm. 2008. Tracing the roles of the PBL tutor. A journey of learning.
Helsinki: Haaga-Helia University of Applied Sciences.
Krashen, S. 1985. The input hypothesis: Issues and implications. New York: Longman.
Mathews-Aydinli, J. 2007. Problem-Based Learning and adult English language learners.
Washington, DC: Center for Adult English Language Acquisition (CAELA). [Link]
Downloaded by [Aston University] at 20:50 30 August 2014

[Link]/caela/esl_resources/briefs/[Link] (accessed October 8, 2010).

Mishan, F. 2005. Designing authenticity into language learning materials. Bristol: Intellect
Books.
Mishan, F., and A. Chambers, eds. 2010. Perspectives on language learning materials
development. Oxford: Peter Lang.
Rieber, L. 1993. A pragmatic view of instructional technology. In The practice of
constructivism in science education, ed. K. Tobin, 193
214. London: Routledge.
Savery, J. 2006. Overview of problem-based learning: Definitions and distinctions. The
Interdisciplinary Journal of Problem-based Learning 1, no. 1: 9
20.
Savery, J., and T.M. Duffy. 2001. Problem based learning: An instructional model and its
constructivist framework. CRLT Technical Report No. 16-01. [Link]
publications/journals/[Link] (accessed June 6, 2010).
Savin-Baden, M. 2004. Understanding the impact of assessment on students in problem-based
learning. Innovations in Education and Teaching International 41, no. 2: 221
33.
Schmidt, R. 1990. The role of consciousness in second language learning. Applied Linguistics
11, no. 2: 129
58.
Schwartz, P., S. Mennin, and G. Webb, eds. 2001. Problem based learning: Case studies,
experience and practice. Abingdon: Routledge.
Tipping, J., R.F. Freeman, and A.R. Rachlis. 1995. Using faculty and student perceptions
of group dynamics to develop recommendations for PBL training. Academic Medicine 70,
no. 11: 1050
2.
Tobin, K., and D. Tippins. 1993. Constructivism as a referent for teaching and learning. In
The practice of constructivism in science education, ed. K. Tobin, 3
22. Mahwah, NJ:
Lawrence Erlbaum Associates.
Tomlinson, B., ed. 2003. Developing materials for language teaching. London: Continuum.
Weiss, R. 2003. Designing problems to promote higher order thinking skills. New Directions
for Teaching and Learning 95: 25
31.
 Innovation in Language Learning and Teaching 269

Appendix 1. LI6011 materials development for language learning

1. PBL self-assessment
The purpose of this self-assessment tool is to reflect on and raise awareness of your learning
process. Self-assessment helps you to improve your metacognitive skills and expertise. You
may also use the form to prepare for group discussions.
Theme:
Student: Date:
To assess your own performance during the module, complete and submit this form mid-
and end of semester and submit it to the tutor for feedback and assessment. You may also
complete it as many times as you wish during the semester. If relevant, attach a concept
map showing the hierarchy of the major concepts discussed.
Please focus on written comments, not only grades.
A My level of motivation and preparedness

Motivation to learn about the theme 0 1 2 3 4 5
Downloaded by [Aston University] at 20:50 30 August 2014

Exploring different sources during self-study 0 1 2 3 4 5

Comments:

B My own contribution to team knowledge construction in the discussions 0 1 2 3 4 5

Comments:

C My own contribution, drawing on other modules 0 1 2 3 4 5

Comments:

D Understanding the outcomes of the discussion and applying it to 0 1 2 3 4 5

the learning tasks and/or the project task
Comments:

E My critical-thinking skills (idea generation, questioning, 0 1 2 3 4 5

argumentation, categorisation, problem-solving and other
reasoning skills)
Comments:

F My professionalism (respect towards my peers and our team 0 1 2 3 4 5

agreement, punctuality and other relevant matters)
Comments:

G Giving and receiving feedback among peers

Did I give feedback and learn from it? 0 1 2 3 4 5

Did I receive feedback and learn from it? 0 1 2 3 4 5
Comments:
 270 F.M. Mishan

Appendix 2. LI6011 materials development for language learning

2. Peer-assessment
Group member:
Name of evaluator/s: Date:
Use this form for peer-assessment of group members at two points in the PBL process,
mid-semester and final.
If you had critical incidents in your group work, also answer the questions on page 2.

j__j Mid-semester Grade

j__j Final assessment

(0 5) Comments (qualitative assessment)

Ambition and motivation

Information search and

fieldwork
Downloaded by [Aston University] at 20:50 30 August 2014

Contribution to meetings

Contribution indicating input

from other modules

Work done on time

Work done as agreed

Reporting

Creativity

Overall grade

Contribution percentage (divide 100%

among members)

Identify and describe any critical incidents that you may have faced in your group work. Also
assess how these incidents were solved. (Provide one sheet that represents the view of the whole
group.)
 Innovation in Language Learning and Teaching 271

Appendix 3. LI6011 materials development for language learning

Materials assessment Student’s/Tutor’s

Student:................................................................Date:..............................................................

Criteria Comments Grade

Student profiling & match to materials

Evidence of critical application of L2

acquisition/materials development
theory & practice

Evidence of broad reading/cross-over from

other modules

Evidence of coursebook/materials evaluation

Downloaded by [Aston University] at 20:50 30 August 2014

Source text/s (if applicable)

Student materials

Structure

Variety of activities

Skills integration

Appeal

Originality

Teacher’s notes and L2 acquisition rationale

Overall presentation

Other comments

GRADE
 272 F.M. Mishan

Appendix 4. Problem-based learning experience evaluation8

Name: (optional) ......................................................................................................................

Please respond to the following by marking one number on the scale 1 to 5 where: 1 Strongly
disagree, 2 Disagree, 3 Neither Agree or Disagree, 4 Agree, 5 Strongly Agree

THIS PROBLEM-BASED LEARNING ACTIVITY HAS IMPROVED MY

ABILITY TO . . . Comments
Study and learn 1 2 3 4 5

Carry out research on my own 1 2 3 4 5

Think creatively 1 2 3 4 5

Reason through problems 1 2 3 4 5

Downloaded by [Aston University] at 20:50 30 August 2014

Apply my general and academic 1 2 3 4 5

knowledge

Work with others as a team 1 2 3 4 5

Contribute to team knowledge 1 2 3 4 5

construction

Carry out research with other 1 2 3 4 5

team members

Respect other students 1 2 3 4 5

Give feedback 1 2 3 4 5

Receive and learn from feedback 1 2 3 4 5

Compromise 1 2 3 4 5

Rethink the role of the lecturer 1 2 3 4 5

in class

How does your experience of learning via PBL compare to your learning via traditional
methods (lecture etc.)?