Assessing Critical Thinking in a New Approach
to Computer-Mediated Communication (CMC) Transcripts

Assessing Critical Thinking in a New Approach to Computer-Mediated Communication (CMC) Transcripts

Abstract

Introduction

Critical Thinking Skills Online

Recent Tools Developed at London Metropolitan University for Measuring Students Critical Thinking within a CMC Environment

Related Work

The Study

Mapping Phases of the Practical Inquiry Model’s Cognitive Presence directly onto the Phases of the SQUAD Approach

Peter K. Oriogun

United Kingdom

Table 1

Cognitive Indicators of the SQUAD Approach (Oriogun, 2003)

Table 2

Case Study 1 -Masters Software Engineering Students (Group 1)

Table 3

Case Study 2 -Masters Software Engineering Students (Group 2)

Table 4

Case Study 3 -Masters Computing Students (Group 3)

Table 5

Matrix for mapping 5 Phases of the Practical Inquiry Model’s Cognitive Presence to Phases of the SQUAD Approach (Oriogun, Ravenscroft and Cook, 2006)

Editor’s Note: Researchers are not content to know what works, they explore the mechanisms to determine why and how cognition, processing and learning takes place. Higher levels of learning, such as critical thinking, are the source of many studies. The present research relates an experiment using Peter Oriogun’s SQUAD method to the Practical Inquiry model used by Garrison and Fahy’s Cognitive Presence model.

Critical thinking involves analysis, critique and some evaluation of the information gathered in order to make a reflective and well founded conclusion from the same. It is therefore very important to understand that critical thinking ultimately affects all forms of communication, including speaking, writing, listening and reading. Critical thinking in online communication is particularly challenging as it puts emphasis on students’ comprehension and knowledge of elements of an argument, as such, interacting with different ideas and one another. In this article, the author assesses critical thinking in a new semi-structured approach to computer-mediated communication, the SQUAD approach (Oriogun, 2003; Oriogun, Ravenscroft and Cook, 2005) using the practical inquiry (PI) model (Garrison, Anderson, & Archer, 2001) as a framework. The phases of the SQUAD approach are mapped directly (Oriogun, Ravenscroft and Cook, 2006) with the practical inquiry model’s cognitive presence phases. From such mapping, the author then compares Fahy’s (2005) latest study of the cognitive presence model with the initial pilot study by Garrison et al. (2001) in the three SQUAD case studies presented. It is argued in this article that the SQUAD approach is superior to using interrater reliability measurement of online transcripts when using the Practical Inquiry (PI) model to assess critical thinking or cognitive presence of online groups. It is further argues that there was an insufficient number of posting (24) by the four students over a period of one week, in the initial pilot study by Garrison et al. (2001) to draw any concrete conclusion from the study. The author, however, concurs with Garrison et al’s (2001) conclusion ‘that the practical inquiry model could serve as a framework for future research in a quest to better understand the cognitive nature of the teaching and learning transaction in an asynchronous text-based conferencing environment’.

Keywords: CMC; community of inquiry; cognitive presence; critical thinking; practical inquiry model; Transcript Analysis Tool (TAT); SQUAD Approach; content analysis

The challenge in online learning and distance education is for educators to develop educational strategies to meet a diversity of students needs in this modern world. New technologies afford students an opportunity to experience a number of new learning environments where they are able to communicate irrespective of time or geographical location. Some critics argue that the type of learning that occurs in distance education is insufficient to develop critical thinking, and furthermore, that learners should be empowered to critically examine and construct meaning through their own prior experiences (Garrison, 1993; Lauzon, 1992). Consequently, Lauzon (1992) suggests that distance educators should positively promote dialogue in order for learners to take ownership of the knowledge gained in such learning environment. A common learning style adopted by a number of higher education institutions offering online learning and/or distance education is the use of discussion forums to foster critical thinking skills. Unfortunately, research has shown that not all students involved in online discussion forums have the necessary latent projective skills to fully participate and interact within such teaching and learning environment.

Critical thinking involves analysis; critique and some evaluation of the information gathered in order to make a reflective and well founded conclusion. It is important to understand that critical thinking ultimately affects all forms of communication, including speaking, writing, listening and reading. According to Bullen (1998), critical thinking skills during online discussion involve four components; namely, cognitive maturity, teaching style of the lecturer/instructor, the students prior experiences and the degree of understanding of the critical thinking process. Jones (1996) summarised Meyers (1986) suggestion that critical thinking across the disciplines have features in common, namely:

1. Critical thinking is a learnable skill with teachers and peers serving as resources.

2. Problems, questions, and issues serve as source of motivation for the learner.

3. Courses are assignment-centred rather than text or lecture oriented.

4. Goals, methods, and evaluation emphasize using content rather than simply acquiring it.

5. Students collaborate to learn and enhance their thinking

Oriogun (2003) adapted Henri’s (1992) classification of critical thinking in what he called cognitive indicators, when he developed his semi-structured approach to online discourse called SQUAD. These cognitive indicators are categorized as Elementary, In-depth, Inferencing, Judgement and Application of Strategies (see Table 1)

Reasoning skills	Definitions
Elementary clarification	Observing or studying a problem, identifying its elements, and observing their linkages in order to come to a basic understanding.
In depth clarification	Analysing and understanding a problem to come to an understanding which sheds light on the values, beliefs, and assumptions which underlie the statement of the problem.
Inferencing	Induction and deduction, admitting or proposing an idea on the basis of its link with propositions already admitted as true.
Judgement	Making decisions, statements, appreciations, evaluations and criticisms. Sizing up.
Application of strategies	Proposing coordinated actions for the application of a solution, or following through on a choice or a decision.

In a more recent article, Oriogun, Ravenscroft and Cook (2006) mapped the cognitive indicators of the SQUAD approach with the method proposed by Garrison et al. (2001) for detecting triggering events, exploration, integration and resolution called Cognitive Presence. Cognitive Presence can be summarised as having four phases of critical thinking, namely, a Triggered Event deals with starting, inviting or soliciting a particular discussion; the Exploration phase is when information is exchanged between the learning participants; the Integration phase is when participant learners construct meaning and propose possible solutions; and finally, the Resolution phase is when proposed solution(s) is/are tested out (Garrison et al., 2001:11).

In this modern technology driven society, online communication is exceptionally challenging for students and educators. In recent years, we have seen the widespread adoption of computer mediated communication (CMC) in education, including extensive interest in using online communications to facilitate asynchronous dialogues, e.g. online teamwork. Consequently, recent research, for example on dialogue analysis, has attempted to explore the relationship between online dialogue features (e.g. roles, strategies, form and content) and learning (Pilkington, 2001). Such an analysis can provide useful insights into the nature of the learning processes from the perspective of, for example, a speaker's intention in a transmitted message and what the receiver perceives has been communicated by the message. However, a problem arises if we wish to investigate specific categories or variables of the learning process, e.g. participation, interaction, social, cognitive and metacognitive (Henri, 1992). It is hoped that recent tools developed at London Metropolitan University will assist educators in engaging their students online as well as aid in measurement of students critical thinking skills or what Oriogun, Ravenscroft and Cook (2005) have termed as cognitive engagement.

The Learning Technology Research Institute at London Metropolitan University recently developed a tool called ‘InterLoc’ (http://www.interloc.org/). This tool supports digital dialogue games for learning. Its structure, scaffold and support multimedia dialogues are highly engaging and foster ‘reasoned’ discourse and critical thinking through ‘live’ peer interaction. It incorporates an environment that supports a multi-phased activity (e.g. preparation, interaction and summary) and the use of particular dialogue games (e.g. critical discussion and reasoning, exploratory dialogue and creative thinking) that foster 'academic' discourse and thinking. The approach is particularly suited to groups of 4-8 students. The activities and dialogue games can be reused or adapted to address particular educational problems and contexts. The tool is highly flexible, and can support a range of pedagogical approaches, from informal student-centred learning activities to more formal course-related exercises. The key features of the tool are (http://www.interloc.org/):

§ A game design that promotes motivation, confidence and engagement

§ Integration of multimedia artefacts

§ Structured interaction through coordinating activities, dialogues, conversations and replies

§ Message openers (e.g. 'I think...', 'I disagree because...', 'Is there any evidence?...') that promote coherent dialogue, thinking and deep learning

§ Reusable and adaptable learning activity and dialogue game templates

Recently, Oriogun (2006) used content analysis of online transcripts to study quality of interaction, participation, and cognitive engagement. New tools developed by the London Metropolitan University were used to improve inter-rater reliability. One of the tools is a software prototype supporting the SQUAD approach. The SQUAD approach to CMC discourse invites students to post messages based on five given categories, namely, Suggestion, Question, Unclassified, Answer and Delivery. The approach to online discourse adopts problem-based learning (Barrows, 1996; Bridges, 1992; Oriogun et al, 2002) as an instructional method with the goal of solving real problems by (Oriogun, 2003):

§ Create an atmosphere that will motivate students to learn in a group setting online;

§ Promote group interactions and participation over the problem to be solved by the group online;

§ Help learners to build a knowledge base of relevant facts about the problem to be solved online;

§ Share newly acquired knowledge with a group online with the aim of solving the given problem collaboratively and collectively;

§ Deliver various artifacts’ leading to a solution or a number of solutions to the problem to be solved online.

In order to enhance students participation, interaction and cognitive engagement online, Oriogun, Ravenscroft and Cook (2005), suggested, that “one way of engaging learners in online collaborative learning is to create an environment in which knowledge emerges and is shared. The onus is therefore on the tutor/instructor to (1) create an environment in which knowledge emerges and is shared through the collaborative work within a group of students, and (2) facilitate sharing of information and knowledge among members of a learning team instead of controlling the delivery and pace of course content”. A methodological framework, developed by Oriogun (2003), called the SQUAD approach was used to develop their argument in the article (Oriogun, Ravenscroft and Cook, 2005) when they validated the cognitive engagement of postgraduate software engineering students at the London Metropolitan University during the two academic semesters of 2004-2005.

Existing literature at the time of the study (Oriogun, Ravenscroft and Cook, 2005) revealed that there are no tools for measuring the cognitive engagement of groups of people working on a particular task/problem online, such as a group’s course work for a module or course. There are tools available for investigating cognitive elements of individuals working online (Henri 1992; Hara, Bonk, and Angeli 2000; Fahy 2002; Garrison et al. 2001; Oriogun 2003; Oriogun and Cook 2003). In the article (Oriogun, Ravenscroft and Cook, 2005) we adopted the theoretical framework of two recently developed tools, commonly used for analyzing students’ cognitive elements online (Fahy 2002; Garrison, Anderson, and Archer 2000, 2001) at an individual level in order to validate at group level the cognitive engagement of groups of students working within the SQUAD approach.

In this article, the author will use the SQUAD statistics gather from two groups of Masters Software Engineering students and one group of Masters Computing students from 20^th June 2006 until 31^st August 2006, a total of 73 days, to measure the cognitive engagement of the students according to the mapping of the SQUAD approach to the Cognitive Presence model (Oriogun, Ravenscroft and Cook, 2006). The first group of Masters Software Engineering students was composed of 4 students. They posted a total of 23 messages over the 73 days of the study. The second group, also of Master Software Engineering student had 5 members. They posted a total of 80 messages over the 73 days of this study. The third and final group, the Masters Computing students had 5 members. Table 2 shows the SQUAD statistics for Master Software engineering Students (Group1).

These students were completing a group assignment in a module called Software Project Management, a designated or optional module on both Masters courses. This component of the module is very practical, and students were given a practical Project Management problem to solve using PRINCE 2 as a methodology, template or vehicle by which to solve the problem. If they pass the module, it will count towards the total of 6 taught modules and a dissertation, which is also worth an equivalent of 3 core or compulsory modules. Out of the 6 taught modules, 4 are core. The group assignment is 50% of the Software Project Management module; the other 50% is an open-book test, which is more theoretical in nature. These students were, at the time of the study, working from England, India, Nigeria and Pakistan. These were all full-time mature students. Table 3 shows the SQUAD statistics for Master Software engineering Students (Group2).

Student	S	Q	U	A	D	Total
S5	1	0	1	0	1	3
S6	7	2	0	7	7	23
S7	4	2	1	6	0	13
S8	5	2	1	10	21	39
S9	1	1	0	0	0	2
TOTAL	18	7	3	23	29	80

The purpose of using the SQUAD environment to facilitate these students group coursework online was because all of the students were full-time students, sharing the same designated or optional module on their Masters programmes. Another reason for getting the students to use the tool was that they have already used the SQUAD environment from September 2005 until January 2006 when they first enrolled on the module, as such they should know the way around the software tool. The final rationale for using getting the students to use the tool was to actually evaluate their collaborative group effort spent on the assignment, as well as obtaining some qualitative measure of each student’s cognitive engagement when mapped to Garrison et al’s Cognitive Presence categories. Table 4 shows the SQUAD statistics for Master Computing Students (Group3).

Student	S	Q	U	A	D	Total
S10	24	0	8	4	7	43
S11	5	0	1	0	0	6
S12	5	0	19	6	7	37
S13	32	0	0	0	0	32
S14	7	1	66	13	1	88
TOTAL	73	1	94	23	15	206

The SQUAD approach (Oriogun, 2003) to CMC discourse provides a means through which statistics compiled from students’ online discourse can be used to generate objective estimations of their degree of learning engagement. The cognitive indicators of the SQUAD approach are based on Henri’s (1992) cognitive indicators. The following section explains how we have mapped the SQUAD approach with Garrison et al’s (2001) framework. Our use of mapping in this article refers to the tools being equivalent for measurement purposes.

The SQUAD category S described above is focused on what the group has to deliver for their group coursework, and does not necessarily deal with significant personal revelation. It also encourages students to initiate, continue or acknowledge interpersonal interaction, and or “warm” and personalize the discussion by scaffolding/engaging comments connects or agree with, thank or otherwise recognize someone else, and encourage or recognize the helpfulness, ideas and comments, capabilities and experience of others. The phases of the Practical Inquiry model capable of being mapped to SQUAD category S are Triggers and Exploration (see Table 5).

The SQUAD category Q is a form of words addressed to a person in order to elicit information or evoke a response. An example of a question within the SQUAD framework is when students seeks clarification from the tutor or other students in order to make appropriate decisions relating to the group coursework (Oriogun, 2003). The phases of the Practical Inquiry model capable of being mapped to SQUAD category Q are Triggers and Exploration (see Table 5).

The SQUAD category U is normally not in the list of categories of messages stipulated by the instigator of the task at hand. This tends to happen at the start of the online postings. Students may be unsure of what the message is supposed to convey. In most cases, it falls within one of the four classified categories (Oriogun, 2003). The phase of the Practical Inquiry model capable of being mapped to SQUAD category U is other. Results of analysis of 24 message transcripts by Garrison et al’s (2001) showed that one-third (8) of the postings did not relate to any of the four phases of the critical thinking model (p.19), as such, they categorised this phase as Other (see Table 5).

Phases of the Practical Inquiry Model	Phases of the SQUAD Approach to CMC Discourse
Phases of the Practical Inquiry Model	S - Suggestion	Q - Question	U - Unclassified	A - Answer	D - Delivery
Triggers	x	x		x
Exploration	x	x
Integration					x
Resolution				x	x
Other			x

The SQUAD category A is a reply, either spoken or written, to a question, request, letter or article. Students are expected to respond to this type of message with a range of possible solutions / alternatives. Also, the SQUAD category S is the process whereby the mere presentation of an idea to a receptive individual leads to the acceptance of the idea, and, students engage with other students within their coursework groups by offering advice, a viewpoint, or an alternative viewpoint to a current one (Oriogun, 2003). The phases of the Practical Inquiry model capable of being mapped to SQUAD category A are Triggers and Resolution (see Table 5).

The SQUAD category D is the act of distribution of goods, mail etc. This is where students are expected to produce a piece of software at the end of the semester. Al the students have to participate in delivering aspects of the artifacts making up the software (Oriogun, 2003). At this point students may show their appreciations to part of the group coursework deliverable by responding with comments with real substantive meaning. The pha

Student