Critical thinking in PBL: Development of a bespoke tool for critical thinking

As an active learning strategy, problem based learning (PBL) puts the emphasis on students being responsible for their learning (Hmelo-Silver and Barrows 2006). Although it is student-centred, PBL facilitators can help students achieve deep learning through strategies which allow them to check their understanding and engage in critical thinking skills (Wood, 2003; Azer et al,. 2013). However enabling students to engage in critical thinking skills and assessing their learning requires the use of innovative strategies (Azer et al., 2013). This paper describes the development of a learning tool to allow students undertake their own-self assessment. It was developed on the principles of ill-structured problems which encourage the utilization of critical thinking skills (Kek and Huijser, 2011). Thus the aim of the research was to assess whether the tool would allow the students to engage in critical thinking skills such as deep understanding and application of knowledge. Results show students found the tool to be effective for their learning where they were able to apply their knowledge and test understanding. This suggested that the developed tool proved to be useful in engaging students in critical thinking skills and as a means to check student understanding.


Introduction
According to Dewey 'critical thinking arises when we are faced with perplexity, confusion or doubt' (Dewey, 1933). So from a educational perspective critical thinking provides a means to seek solutions to problems and encompasses an array of cognitive skills which include analysis, reasoning and evaluation (McPeck, 1981). These skills are situated at the high end of the educational learning objectives in Bloom's taxonomy thus signifying critical thinking as being the apex of effective learning and overall educational attainment (Bloom, 1956).
More specifically, critical thinking is essential in medical education and in empowering medical students to become competent doctors. Be it in the chaotic environment of an emergency or when undertaking complex differential diagnoses, doctors must be able to think critically. It is through drawing on critical thinking can doctors ask the right questions, order the appropriate tests and consider the most effective treatments even if they do not have all the information at hand. Thus in the context of medicine, critical thinking encompasses not only cognitive skills of analyzing, reasoning but also the ability to infer, show diligence and an overall inquisitiveness (Sharples et al., 2017). Putting this into context and in highlighting the importance of critical thinking to medical education a recent study explored the underlying causes of diagnostic errors. The findings were indicative that doctors perceived the problem to be due to lack of critical thinking rather than lack of knowledge (Crosskerry 2013).
PBL is naturally suited in helping students develop critical thinking in medical education. The study of authentic clinical cases allows students to utilize cognitive skills such as analysis and evaluation of information. A major part of PBL also involves students reaching a collective understanding of problems which draws on critical thinking skills such as knowledge application, reasoning and making inferences (Masek and Yamin, 2011). Evaluation of own current PBL format highlighted that students were lacking to effectively show or engage in critical thinking. This was based on the recognition that students were mostly reading out answers from their notes which provided little opportunity to show their understanding or ability to think critically. Thus this paper describes a study which was carried out to explore how a strategy could be developed to allow students to utilize critical thinking skills.

Literature review
A literature review was performed to extract a comprehensive understanding of the link between critical thinking and PBL. The literature was searched using the following key words 'critical thinking', and 'critical thinking AND PBL'.

Situating critical thinking in PBL
PBL is considered to be an active form of learning which engages students to exercise the different domains of Bloom's taxonomy including analysis, evaluation and synthesis of knowledge (Azher et al., 2013). Given that the origins of PBL stems from innovating medical education, it has been identified to improve critical thinking skills i.e. clinical competencies such as reasoning and analytical skills of medical students (Sharples et al., 2017). Recognising the potential of PBL as an effective means of learning, it has been adopted in other fields of education and in real-life situations. For instance it has been reported to be an effective approach for teaching managers develop skills of reasoning, analyzing and evaluation for tackling problems (Carvalho, 2015). Thus it was evident in the literature that PBL offers students scope to develop critical thinking skills, however it was quite clear that despite its implementation, using strategies which enables students to extend into the higher domains of Bloom's taxonomy i.e. application of critical thinking was often overlooked. One major factor underlying this could be that there is no general consensus on the best way to facilitate PBL for effective learning. (Savin-Baden, 2014).

Inception of critical thinking through ill-structured problems
Several innovations have been reported in the literature as a way to engage students with critical thinking. Activities such as debates can stimulate reasoning and analysis however they can be time consuming which could result in time being taken away from the core classroom time (Crenshaw, 2011). In overcoming this problem, innovations could be designed which students can engage outside classroom time. Exercise based workbooks offer students to work at their pace. The incorporation of problems within the workbooks stimulate an inquisitive approach which is thought to act as mental exercises to develop critical thinking (Crenshaw, 2011). However the extent to which such innovations help students apply the skills of critical thinking has been argued to be influenced by the design of problems. Textbook type of excercises or problems have been found to have no impact on improving students' critical thinking skills (Cotter et al., 2009). A more effective strategy could be the use of ill-structured problems to trigger students to utilize their reasoning, analytical and evaluation skills (Ge and Land, 2003). Ill-structured problems are constructed in a vague manner prompting students to consider all the components of the problem. This acts as a way to help the students make links to concepts and consider them as a whole, which is required for knowledge integration (Torre, 2013). But the important feature of such problems is that they accommodate multiple solutions thus triggering skills such as reasoning, analysis and evaluation.

515
Critical thinking in PBL: Development of a bespoke tool for critical thinking 3. Methods

Study design and data collection
This study was carried out using an action research approach which offered a collaborative approach to work with colleagues and students in better understanding the problem as well as finding solutions. Three PBL groups consisting of 34 students gave consent to be part of this study. The overall study design involved two cycles each running for two weeks. In each cycle printed matrix customized for the individual PBL week, was given to the students by the respective group's PBL tutor. Answers to the matrix were provided in the second (closing) PBL session. At the end of each cycle an online questionnaire was used to allow students to evaluate their experience of using the matrix and to find out what changes could be made to improve it.

The principles used in designing the learning matrix
From the literature review ill-structured problems were identified as being effective in developing critical thinking. However the traditional format of textbook styled problems were found to be ineffective for critical thinking which prompted research into seeking alternative format of presenting information and testing understanding. Concepts or information can be presented in a visual way using concept maps which help students visualize relationship between concepts (Torre, 2013). This principle of presenting concepts visually formed the basis on which an innovation was designed for the purpose of this research. In concept maps information is presented in a linear and hierarchic manner however a weakness in such a format is that if too much information is included, the key concepts can be masked as well as cause confusion (Gerdeman, 2013). To overcome these problems, an innovation was devised in a matrix like style to enable information to be expressed in a concise way. However like concept maps, information which convey key or critical information was presented in a minimal manner as follows. Firstly, the concepts were framed under specific headers which act as specific questions or anchors. As this reseach was interested in medical education the innovation was devised to be aligned with the objectives and content of Year 2 curriculum of a MBChB programme (School of Medicine, Keele, UK). Thus the headers were specific to clinical competencies such as scientific knowledge, recognising symptoms and undertaking differential diagnosis which were referred to as "mechanism ", "clinical features" and "diagnosis" within the matrix. Under each of these headers relevant concepts were presented within rows which are associated with different features of diseases. However to make the intervention as an ill-structured problem, the rows had been left partially incomplete thus acting as a puzzle. Represented much like a crossword, the headers act as clues as what information/concepts are needed to fill in the incomplete boxes. To make it challenging rather than a straight fill in the blank type of problem, the concepts given in the learning matrix were carefully selected to create uncertainty and 516 Nazim Ali, Russell Crawford, Margaux Horn ambiguity (Figure 1). This was aimed to trigger the students to carefully analyse what they know and use critical thinking skills to come up with the most correct solution.

Figure 1. A learning matrix to support students' learning of anaemia related topics. The right panel shows the main ILOs as set by the students for a PBL case themed on anaemia and the learning matrix devised for this is
shown on the left panel.

Results and discussion
In cycle 1 the matrix was given to the students at the beginning of the second PBL session. It was reasoned that the puzzle like nature of the matrix would be useful for students in testing their understanding. This would encourage group discussions where students could show critical thinking skills such as analysis and reasoning when offering solutions to the matrix. However it was found that very few students attempted to engage with the matrix during the session. A major reason behind this was that students felt they did not have enough time since they were busy discussing their set intended learning outcomes (ILOs) (data not shown). Students also suggested that perhaps it would be more useful to them if they had access to the matrix during the week rather than at the end i.e in the closing session.
This change was implemented in cycle 2 where students had access to the matrix after the first PBL session. Students used the matrix for two consecutive weeks after which their experience of using it for PBL was evaluated. Out of the 34 students, 13 students provided their responses on using the learning matrix. Lack of time was again cited for the low response by some of the students. Nonetheless, from the students who had responded it was clear that they found the learning matrix to be an effective and useful tool for their learning.

Critical thinking in PBL: Development of a bespoke tool for critical thinking
Analysing the free-text data for recurring theme on the usefulness of the matrix four main themes emerged as described below.

i). Interlink different concepts:
The matrix allowed students to link the different topics and form a summary of the important learning areas as evident from the following responses.
"Allowed me to connect different aspects of learning together". "Helped me summarise information from pbl". "Helped to break down large quantities of learning material into concise key points which are important for the case." The above responses provided indication that students perceived the learning matrix to have a positive impact on their learning. Using only few words the matrix design allowed the tool to be useful in summarising several key concepts. Based on the principles of concept mapping it could be argued that visualising the concepts together helped students to establish how they relate to one another and thus aid their understanding (Torre, 2013).

ii). Test understanding and apply knowledge:
The learning matrix was designed as a puzzle. It was anticipated that such a format would evoke the knowledge students had gained in tackling their ILOs and apply their knowledge to "solve" the matrix. In order words, students would be prompted to use reasoning and evaluation skills to identify the missing information of each row and thus complete the matrix. Students perceived this strategy as an effective way to test their own understanding and apply their knowledge as documented by the responses highlighted below.
"I used it at the end of the week to tie up my work and make sure I had covered everything and understood it." "Helps to test my understanding." "Helps to guide my understanding of a topic. It ensures I have covered all areas relating to a case. It also gives a baseline level where the whole PBL group should be." "Useful in pushing us to apply our knowledge." iii). Identify knowledge gap: Another positive aspect of the learning matrix reported by the students was that it allowed them to identify gaps in their learning. This is evidenced in the student responses below. This feature of the learning matrix can be attributed to its design which allows key concepts to be visualised by students. Indeed, highlighting important learning areas as concepts or keywords through concept maps have been shown to be an effective way for students to not only assess their understanding but pinpoint areas which they need to improve on (Veronese et al., 2013). Thus the matrix could be used by students as a self-diagnostic learning tool as part of their reflective process on improving their learning. vi) As a revision tool: Extending on the above point students also found the matrix to be a useful study aid which could help in their revision as exemplified by this response: "I think it will make a useful revision tool as it is a means of collecting and packaging relevant information into a succinct form. The completed matrix can be easily printed off and put up on a wall which will make it useful for glancing at during revision periods."

Conclusion
Critical thinking encompasses cognitive skills such as deep understanding, reasoning and application of knowledge. The findings here indicate that the matrix could be a potential tool to support students in developing critical thinking skills. Firstly, students identified it to be effective for their learning since they could assess their understanding and apply the knowledge they had gained. Secondly, the matrix functioned much like a puzzle owing to its innovative design thus allowing the element of gamification to be incorporated into the existing PBL format. Gamification motivates students to learn (Kapp, 2012) and so the matrix could be used to increase student engagement. Thus the matrix could be used as a tool by tutors to improve their practice especially in PBL. Checking students' understanding is an essential part of PBL (Wood, 2003) but where PBL is facilitated by tutors who are non-expert on the learning topics, this can be a challenging issue. The matrix offers a unique approach to test student understanding in a self-regulatory manner. Furthermore, it has the potential to act as an agency in ensuring tutors are able to provide guidance and make PBL effective without having to be content specialist.