go top

Editor’s Note: This study is based on an online course at the University of Mauritius. It evaluates web-based learning environments from a cognitive, as compared to constructivist, point of view. The authors relate the content to learning styles, interactions, role of the teacher, ways in which learners’ process their experiences, and techniques used to measure learning. In the process, they discover that behaviourism plays a dominant role in design and implementation of educational programs.

A Cognitive Approach to Evaluating
Web-based Distance Learning Environments

Mohammad Issack Santally and Alain Senteni

Abstract

Distance education has always capitalized on the development of new communication methods and researchers have often laid emphasis on the needs for new approaches to the educational process in such a setting. In this article, our focus is based on the pedagogical evaluation of web-based learning environments from a cognitive point of view. We develop an evaluation grid based on the underlying cognitive principles and analyse how the environment fits in the profile of learners in a learning context. We briefly elaborate the guiding principles behind the design of our evaluation grid and apply it to a module that is currently being delivered in online distance education mode at the University of Mauritius. We find that although the learning environment contains essential tools and facilities to support cognitivist as well as constructivist learning, the engineering of the module content is still done in a behaviourist way. Student perceptions of these environments are also important since the term “learner-centred approach” is often associated with such environments. Finally, there is a need to bring a fundamental change in test traditions to reflect the new approaches towards teaching and learning.

Keywords: Internet, Web-based Learning, World-Wide Web, Distance Education, Distance Learning, Online Learning, e-Learning, Pedagogy, Cognitive Evaluation, Constructivist Learning, Learner-centered Instruction.

Web-Based Distance Education: New Insights

Distance education has always capitalized on the development of new communication methods. In the past, institutions promoting distance education used the postal delivery as preferred means for communication. In fact, nowadays information and communication technologies (ICT) via the Internet is set to play a very important role in distance education. The Internet linked information networks that have been set up provide an efficient infrastructure for interaction and communication with the resource persons and tutors. The flexibility and data transfer speeds of modern information networks helps in the implementation of new pedagogies such as giving support to the interaction process between the lecturer and his students and also among students themselves. Although we stress on distance education as promoting independence and autonomy of the student in the learning experience, some authors (Shale, 1991; Garison & Shale, 1990) insist on the importance of recreating and promoting the interaction between the students and teachers that used to take place in conventional educational settings. On the other hand, Marjanovic & Orlowska (2000) argue that the challenge is not to recreate the traditional classroom with all its inherent problems, but rather to create new learning environments providing unique communication patterns, changed limitations to the types of learning activities that are possible and provide a new high quality learning experience.

Distance education via the Internet addresses these concerns because apart from favouring teacher-student interactions, it provides the optimum infrastructure for the development of environments based on innovative pedagogies and contemporary theories of education such as the socio-constructivist perspective of learning. Supporting multimedia and hypermedia elements, the Internet promises to provide solutions to many problems linked to traditional distance education. One of the critiques of Guillemet (1989) on distance education was that students having a preference for group work were penalized since it was difficult to promote pedagogies supporting group and collaborative work. Virtual web-based collaborative environments such as discussion forums, chat applications, and newsgroups help these types of students to find peers and tutors online who would support them in their learning activities. Virtual group work has proved to be an effective pedagogical strategy as reported in different works (Santally, 2003; Veerman & Veldhuis-Diermanse, 2001; Hakkinen et al., 2001).

Santally (2003) describes how appropriate pedagogical activities can help students with different learning styles to contribute in a virtual discussion forum. This study took place at the University of Mauritius where students were given a discussion topic and their interactions were recorded and analysed. The study showed that online forums are useful tools to enhance the learning process for students since they encourage critical reflection by providing enough wait-time for students to reflect on when responding to postings of other peers.

Veerman & Veldhius-Diermanse (2001) have analysed learner interactions in four different collaborative environments (synchronous and asynchronous) and they found that the asynchronous learning environments favour the collective construction of knowledge (constructivist approach) while synchronous learning environments favour the development of higher cognitive skills compared to asynchronous learning environments.

The Learning Material

The CSE 1010E (Introduction to Information Technology) was initially delivered through print-based distance education mode and it became the first module to be delivered online at the University of Mauritius on a very large scale (~ 1000 students). The CSE 1010E module (Figure 1) has now been delivered without any major problems, for approximately two academic years now. The module is hosted by the University of Mauritius Virtual Campus, which provides the technological infrastructure and pedagogical tools to enhance the teaching and learning process.

Figure 1: The CSE 1010E home page

Students have an online study guide (also available in print format) where they have access to an instructional plan that helps them in their learning. They get instructions about chapters to read and exercises to carry out. The contents section provides students with a multimedia learning material arranged in a hypertext structure that helps them to understand concepts easily. Students also have access to a range of self-assessment questions for each chapter. The assignments and practical sections contain necessary information about continuous assessment and hands-on activities to be carried out in the lab. Students also have access to online discussion forums where they can discuss with peers and tutors about concepts and topics related to their module. Participation on online forums does count as part of the continuous assessment. This acts as a motivation factor for students to participate.

The Evaluation Grid/Instrument

The evaluation grid has been devised, based on the assumption that the learning environment is being evaluated from cognitive expert point of view. It is divided into two principal sections: (1) general aspects regrouping the content and technical elements; and (2) the pedagogical approaches regrouping the statements describing the characteristics of the learner based on cognitivist approaches. The students’ profiles that we retain for the evaluation grid are: the learner with a unique profile, the contextual learner and the co-active learner.

A learner with a unique profile

Every learner has a unique profile that distinguishes him from other learners. This profile depends on different variables such as cognitive, meta-cognitive and socio-affective factors that affect their learning experience. The important factors that we consider in this study are (1) learning styles, and (2) cognitive styles.

Learning Styles

A number of studies (Kolb, 1984; Santally, 2003; McLoughlin, 1999) have shown that students have different styles of learning. Individuals have different learning styles, which indicate preferences for particular learning experiences. There are different instruments to determine students’ learning styles. Two well-known learning style instruments are the Kolb (1984) Learning Style Inventory and the Honey and Mumford (1986) Learning Style Questionnaire. Kolb classifies students as assimilators, divergers, convergers and accommodators while Honey and Mumford classify students as reflectors, activists, pragmatists and theorists. For more information on learning styles, please see Santally (2003) and Ayersman & Minden (1995).

Researchers have emphasized on the importance of considering the various learning styles while designing educational materials. We need to highlight here the fact that this is a difficult task if we have to deal with thirty to forty students at a time. Teaching average students through their learning styles help them in achieving better results in the exams (Dunn et al., 1995).

Cognitive Styles

The research literature classifies cognitive styles in two major subgroups (Ayersman & Von Minden, 1995): Information Gathering and Information Organizing. Within the subgrouping of Information Gathering, there are Visual/Haptic, Visualizer/Verbalizer, and levelling/sharpening styles (Jonassen & Grabowski, 1993). Within the subgroup of Information organizing, there are serialist/holist and analytical/relational styles. Barber & Milone (1980) developed a simpler instrument to measure cognitive styles (VAK Instrument), which simply classifies learners as being visual, auditory and kinaesthetic or a combination of these. Cognitive styles are recognized as describing learner traits and are mostly related to their preferred way to process information.

A Contextual Learner

Many students face problems to use and apply knowledge and abilities acquired via traditional learning situations to real-life and everyday contexts (Carraher et al., 1985). This problem mainly stems from decontextualised formal learning experiences, which means learning of facts that are isolated from the contexts in which they derive meaning (Choi & Hannafin, 1995). In such cases, students may pass exams but be unable to apply the same knowledge in everyday circumstances. Contextual learning therefore has several implications for the design of learning environments. Such environments need to emphasize on higher-order thinking skills over memorization of factual information (Choi & Hannafin, 1995). Duffy & Jonassen (1991) stress on the importance of the paradigm shift in the role of the teachers to facilitate contextual learning. The teacher needs no longer be a knowledge transmitter but he needs to play the role of a coach or facilitator of students’ understanding. Contextual learning also requires a need to re-engineer existing assessment methods. Since contextual learning environments focus on the individual’s cognitive processes and transfer of knowledge, assessment needs to be dynamic, and reflect ever-merging samples of the learner’s progress (McLellan, 1993).

A Coactive Learner

A coactive learner is always in constant interaction with peers, the physical learning environment and the tools that are accessible to him that will help in the realisation of learning tasks required. The coactive learner has control over the navigation sequence and his interaction with the other constituents of the environment. He is therefore engaged in an active process of knowledge construction in an interactive environment. Such an environment needs to be conceived in such a way that it promotes interactivity between learners and tutors as well as providing the appropriate tools for communication and collaboration among the actors involved in the process. This scenario can be represented by the framework offered by Cultural-historical Activity Theory (Vygotsky, 1978) and its later application to organisational learning called expansive learning (Engestrom, 1987, 2001) which is increasingly being used to inform the design of better teaching materials and e-Learning environments.

The basic concept of activity theory is that learning is a human activity that is socially situated and artefact-mediated. In essence, tools mediate the processes between the subject and object; rules mediate the processes between subject and community; and division of labour mediates the processes between community and object (figure 2).

Instruments

In a learning situation, the subject is the student and the object would be a learning activity. The aim of the student is to carry out the activity and he has access to a network allowing him to interact and communicate with the other components of this environment (instruments and tools, community of peers and teachers, rules to preserve good functioning of the system as well as promote collaboration and cooperation) to attain his goal (the outcome) which would be a sound learning experience.

Content Evaluation

		1	2	3	4	5
1.	Presentation of learning outcomes and course plan				■
2.	Organisation and clarity of information presented			■
3.	Usefulness of examples and illustrations used in the course			■
4.	Coherence of contents/activities with established learning outcomes				■
5.	Usefulness and clarity of guidelines given for the course				■
6.	General Evaluation of the contents				■

Technical Evaluation

1	Navigational structure and ease of navigation	■
2.	Cognitive ergonomics of the user interface	■
3.	Accessibility to the environment		■
4.	Ease of downloading of documents		■
5.	Technical help and support to use the environment	■
6.	Ease of use of communication and collaboration tools	■
7.	General evaluation of technical aspects	■

A Unique Learner Profile

1	Environment favours the use of different learning strategies	■
2.	Multimedia elements caters for different cognitive styles		■
3.	Activities are coherent with students learning styles		■
4.	Activities included help students to complete Kolb Learning cycle		■
5.	Environment favours a differentiated pedagogical approach adapted to individual student needs.	■

Contextual Learning

		1	2	3	4	5
1	Learning activities and tasks are related to authentic situations				x
2.	Activities urge students to use higher levels cognitive abilities				x
3.	A variety of evaluation methods are used (e.g. Portfolios, projects, continuous assessment, group work….)				x
4.	Inclusion of examples, templates, previous students work and models of expected answers to facilitate putting the students in context			x
5.	Appropriateness of metaphor used to better situate the context			x

A Co-Active Learner

1	Activities included incite students to work in groups and collaborate online		x
2.	Activities favour frequent interactions between learner and tutor		x
3.	The environment provides student with a panoply of interactive tools and resources	x
4.	The environment favours pedagogical approaches based on knowledge construction	x

Analysis of the CSE 1010E Learning Environment

The analysis shows that the environment is average from a cognitive perspective especially from a pedagogical approach. The best aspects of the environment are the good quality of the course contents and the technical aspects while it is average on the pedagogical side. The first thing to point out here is that the learning environment is two-fold: firstly, it contains the course itself, that is the module contents, assignments and self-assessment questions; and secondly it contains the virtual campus platform that hosts the course contents and provides access to communication and collaboration tools as well as other resources to help the student in his learning endeavour.

It is obvious, from the grid that the virtual campus has been conceived to promote a cognitive and constructivist approach to learning but the CSE1010E course itself has been mainly developed on a behaviourist perspective of learning. CSE1010E is a classical module with its contents structured in chapters that the students need to master to take part in the exams. The module actually being delivered on the virtual campus is basically just an electronic replica of the traditional version. To read contents online and to learn it “by heart” to succeed in exams is a concept completely out of phase with the cognitivist perspective and does not satisfy the conditions necessary to offer a learning experience tailored to the needs of a learner with a unique profile, coactive and in context.

On the other hand, we notice that the virtual campus is multilingual, ergonomic, easy to learn and very usable. The environment uses the space metaphor to represent virtual classrooms, conference halls, library and the café etc. The use of such appropriate metaphors helps reduce the cognitive load on the student to understand and use the functionalities of the system.

It is to be noted, that in the current virtual campus setting, the same version and type of content is proposed to the students. One of the critiques addressed to e-learning environments is the lack of personalisation in the courses (Cristea, 2003; Rumetshofer & Wob, 2003; McLouglin, 1999; Ayersman & Minden, 1995). This is a problem with the current environment since it fails to cater for individual learning styles and cognitive preferences. A recent study (Santally, 2003) on learning/cognitive styles of students and their perceptions of web-based learning showed that students find that the course does not meet their individual preferences and that they prefer to print the contents rather than read the whole bunch of HTML text online.

However, on the other hand, the environment favours the student’s use of metacognitive strategies to better manage their learning. With the “bookmarking” facility offered by the system, students are able to save important links so that they may come back again to these very easily for example when they need to revise. In this way, the student can also keep a trace of his learning paths. Furthermore, the course plan offered by the environment help the students to better plan their learning. Here it implies that the environment provides the facility to incite students to make use of metacognitive strategies.

The environment allows students to work in asynchronous collaboration to exchange messages and to participate in online virtual discussion forums. The forum tool is very much used at the University of Mauritius by academics to carry out pedagogical activities whose practise in traditional classrooms is limited. However, the decision to use the forum tool depends on the tutors and there are many tutors who do not use this facility at all in the online environment. It is therefore difficult to precisely evaluate the effectiveness of forums embedded in the current context.

Discussion and Conclusion

In this study, our analysis mainly focused on two principal aspects of the environment: the module and the virtual campus. An important thing to point out after the analysis is that the tools by themselves do not promote any particular approach, method or pedagogy. It is the way we use and incite the students to use these tools for learning purposes which is more important. For an environment to meet the cognitivist criteria the nature of the tools and the instructions do not merely suffice; this depends also on how much the students and tutors adapt and master the learning environment. Therefore the way that pedagogical experts conceive the activities play a determining role in the promotion of a particular approach to learning.

We have seen in the analysis that the module has been conceived on behaviourist principles while the virtual campus has all the necessary ingredients to promote a cognitivist approach. The role of the teacher is therefore essential in the process since to conceive a particular pedagogy and to apply it in authentic situations is a different thing.

The education system in Mauritius is mainly based on behaviorism just like the majority of educational systems throughout the world. It is therefore obvious that teachers and even instructional designers continue to work on these principles. Therefore we recommend that appropriate training (or re-training) be given to these persons in the cognitive sciences education field. This is however, a time-consuming process before such changes can be visible in any education system. The reform of such educational practices need not only be on the pedagogical design level but depends also on the educational policies of countries and nations.

There is also a need to change the way a student’s progress is assessed. Assessment and evaluation form an integral part in the educational process. For instance, the normal weightage for examinations and continuous assessment in most classic educational systems are 70% and 30% respectively. The weightage for continuous assessment where students are given authentic tasks such as mini-projects could be increased. Reliable alternatives should also be found to gradually replace traditional written exams where many students do not work at their best under pressure.

There is also a need to find out about student perceptions of such environments and pedagogical approaches. We have been emphasizing on the need for learner-centred approaches to education and on the new role of the teachers in the process. Although the majority of students found web-based learning interesting and stimulating (Santally, 2004), there are students who still prefer the classic approach to education and prefer spend 3 hours in a lecture and to take an exam fifteen weeks later rather than to actively participate each week in knowledge construction activities. They find that these new approaches need much more involvement on their part. This is mainly a problem encountered with mature learners who have other professional and social obligations than with young university students.

References

Ayersman, D.J. & Minden, A. (1995). Individual differences, computers and instruction. Computers in human behaviour, 11(3-4), p 371-390.

Carraher, T.N, Carraher, D. W. & Schliemann, A. D (1985). Mathematics in the streets and in schools. British Journal of development psychology, 3, 21-29.

Choi, J. I., Hannafin, M. (1995). Situated Cognition and Learning Environments: Roles, Structures and Implications for Design. Educational Technology Research and Development, 42(3), 53-69

Cristea, A. (2003). Adaptive Patterns in Authoring of Educational Adaptive Hypermedia. Educational Technology & Society, 6(4), 1-5, Available at http://ifets.ieee.org/periodical/6_4/1.pdf

Dunn, R., Griggs, S.A., Olson, J., Gorman, B., & Beasley, M. (1995). A Meta Analytic validation of the Dunn and Dunn learning styles model. Journal of Educational Research, 88(6), p 353-361.

Duffy, T.M, & Jonassen, D.H. (1991). Constructivism: New implications for instructional technology? Educational Technology, 31(5), 7-12

Engestrom, Y. (1987). Learning by expanding: An Activity-theoretical approach to developmental research. Helsinki. Orienta-Konsultit

Engestrom, Y. (2001). Expansive Learning at Work: toward an activity theoretical reconceptualisation. Journal of Education and work. 14(1).

Guillemet, P. (1989). The Distance Education Problem. in Distance Education Today, reference document, Sainte Foy, TeleUniversity: Quebec

Garrison, D. & Shale, D. (1990). Education At a distance: From issues to practice, p. 123-134

Hakkinen, P., Jarvela, S., & Byman, A. (2001). Sharing and making perspectives in web-based conferencing, in proceedings CSCL 2001, 22-24 mars, Maastricht, Pays-Bas, pp. 285-292. Url: http://www.mmi.unimaas.nl/euro-cscl/Papers/64.doc

Honey P., Mumford A (1986). Using your learning styles. Maidenhead. Honey Publications

Jonassen, D. H., & Grabowski, B. (1993). Individual differences and instruction. New York:Allen & Bacon.

Marjanovic, O. & Orlowska, E. (2000). Making Flexible learning more flexible, IWALT 2000 (International Workshop on Advanced Learning Technologies)

McLellan, H. (1993). Evaluation in a situated learning environment. Educational Technology, 33(3), 39-45

McLoughlin, C. (1999). The implications of research literature on learning styles for the design of instructional material. Australian Journal of Educational Technology, 15(3), p 222-241. [online]. available at http://www.ascilite.org.au/ajet/ajet15/mcloughlin.htm

Rumetshofer, H. & Wob, W. (2003). XML-Based Adaptation Framework for Psychological-driven E-Learning Systems. Educational Technology & Society, 6(4), 18-29, [online]. Available at http://ifets.ieee.org/periodical/6_3/4.pdf

Santally, M. (2003). Students Learning Styles & Computer Conferencing as a pedagogical tool to enhance and support the teaching and learning process. World Conference on E-Learning in Corp., Govt., Health., & Higher Ed. 2003(1), 1165-1168. [Online]. Available: http://dl.aace.org/13880

Santally, M. (2004). Students Perceptions of Web-Based Learning. (in press) Academic Exchange Quarterly, Vol 8(1).

Shale (1991). Towards a new conceptualization of distance education. The American Journal of Distance Education, 2, (3), p. 25-35

Veerman, A., & Veldhuis-Diermanse, E. (2001). Collaborative learning through computer mediated communication in academic education, in proceedings CSCL 2001, 22-24 mars, Maastricht, Pays-Bas, pp. 625-632. url : http://www.mmi.unimaas.nl/euro-cscl/Papers/166.doc

Vygotsky, L. (1978). Mind in society: the development of higher psychological processes. Cambridge. Harvard University Press

About the Authors

Mohammad Issack Santally has been working as Instructional Designer in the Virtual Centre for Innovative Learning Technologies (http://vcampus.uom.ac.mu) at the University of Mauritius for more than 2 years. He is in charge of the Online Courses Development Section and research in the e-learning/technology in education field under the supervision of the Director of the Center. m.santally@uom.ac.mu

Alain Senteni is a Professor in Computer Science and is currently the Director of the Virtual Centre for Innovative learning technologies in Mauritius. For the last fifteen years, Alain Senteni’s teaching and research have been related to the uses of technology in education, including computer-mediated communication, multimedia, technology-based training and pedagogical engineering. senteni@uom.ac.mu

A Cognitive Approach to Evaluating Web-based Distance Learning Environments