June 2005 Index
 

Home Page


 Editor’s Note: Formulative evaluation enables the instructor to assess the success of learning strategies and make mid course corrections to optimize results. The Midcourse Feedback procedure applied here was taken from The Online Teaching Guide by Ken White and Bob Weight.

Implementing a Midcourse Feedback Procedure
in the Online Learning Environment


Kelly Bruning

Abstract

The onset of the information age has ushered in a new revolution of learning. This learning takes the form of asynchronous communication between learner and educator using a technical platform that allows global access. This teaching process brings with it new assessment tools in evaluating student learning and the educational objectives of an online course. Student assessment and feedback is critical to the online environment and developing the learning community. Research indicates that formative evaluation tools serve as a justifiable assessment tool that allows for honest student feedback and an opportunity for the instructor to redirect the class based upon that feedback while also nurturing the learning.

Keywords: Online assessment, online learning, online student feedback, community.

Growth of Online Distance Education

The growth of the Internet and its impact on society has often been compared to the impact of the industrial revolution. The Internet's effect on online education is now being considered by academia as a reputable mode of educational delivery with global reach. Online instruction is the fastest growing field in distance education, making use of the Internet and other communication technologies to reach wide-spread audiences of distance learners (Scagnoli, 2002). Universities and colleges worldwide are taking advantage of this trend. Scagnoli (2002), states, "Universities embrace this new trend and adopt this medium to offer their traditional distance courses or to promote new programs or courses" (p. 19).

Evaluation Concepts

Course and student progress evaluation generally take two forms: formative and summative evaluation. Formative evaluation is an ongoing process that can occur at any point throughout the course; it can surface gaps in course material or in the learners' ability to grasp the material. Formative evaluation gives instructors a way to shift the focus of the course if the course is not proceeding as the instructor expected. Evaluations conducted to assess feedback prior to the course end in order to further develop the class are know as formative evaluation because they influence how the learning community develops (Preece, 2000).

Evaluations also play a validation or summative role. Summative evaluation assess the completed course and is the model most often used for evaluation in academic institutions (Palloff & Pratt, 1999). Stephen Brookfield (1995) as cited by Palloff & Pratt, 1999, in commenting on the reliance of summative evaluation, states this form of evaluation is really a measure of student satisfaction with the course and the instructor not a measure of student learning.

Many academic institutions require the use of a course evaluation format that is standardized across the organization. These evaluation forms rarely address whether or not a class has supported students in achieving their learning objectives. Instead, they tend to evaluate whether the student liked the instructor or the course (Brookfield, 1995). According to Palloff & Pratt (1999), if instructors are truly establishing a collaborative transformative process, the formative as well as summative evaluation must be used. Formative evaluation helps determine to what extent instructors are successfully facilitating reflection on the course material under study, reflection on the means of learning, and reflection on self as a learner as the course progresses. Summative evaluation helps to assess how well the instructor and students achieved the goals and learning outcomes established in the course outline.
 

Using Midcourse Feedback to Improve Online Teaching and Learning

The concept and procedure for the midcourse feedback evaluation tool is outlined in the text entitled The Online Teaching Guide A Handbook of Attitudes, Strategies, and Techniques for the Virtual Classroom written by Ken White and Bob Weight who are members of the University of Phoenix faculty. White and Weight (2000), state,

"In order for online teaching to be effective it needs to include the feedback of student opinions….it is really the online students who are in a better position to comment on classroom teaching skills, course difficulty, and online instructor-student interaction" (p. 175). Pioneered for traditional onsite courses as a means of evaluating teaching effectiveness, online education can be greatly enhanced by the use of midcourse feedback (White & Weight, 2000).

The purpose of the midcourse feedback procedure project was to gain an honest assessment by students of the course at midpoint in order for the instructor to redirect the class. Fostering an open and nurturing learning environment will aid in soliciting honest student feedback. Many times, participants are unwilling to be completely honest about their evaluation of a course or an instructor due to fear of repercussions (Brookfield, 1995). Consequently, the willingness of participants to be honest will be related to one that includes the instructor as an equal member (Palloff & Pratt, 1999).

The Midcourse Feedback Process for BUS105

The first step in the process is to introduce the idea of the Midcourse Feedback Procedure as early in the course as possible, preferably in the syllabus. The second process is to secure a volunteer from the course who would be willing to collect the individual e-mails and cut and paste them into a word document to protect student anonymity. In this case, a freshman level Business Mathematics Course was selected for the process by the instructor who created and piloted the course during the summer (taught one time prior to the fall class providing the assessment feedback).


Tue, Oct 22, 2002 -- Midcourse Feedback Procedure

I am taking an online course myself entitled "Online Teaching and Training." I would like to introduce the concept of a Midcourse Feedback Procedure. The procedure begins with a call for a student volunteer. The student volunteer is responsible for receiving e-mails directly to their e-mail address from fellow learners in the class. The volunteer then compiles the responses to the below question without attaching any names to the comments and forwards it to the instructor.

The two questions are:

(1) List three areas that are working well in this course

(2) List three ways to improve the class

The volunteer will then combine all of the messages-verbatim and unedited- (you could cut and paste) and send them to the instructor. The information will be used to make changes for the remaining of the course and for future courses.

I would like responses collected through this Sunday, December 27th, and have the volunteer combine them, and forward them to me by next Wednesday, December 30.

Any volunteers?
 


Figure 1. Blackboard Posting to the Class From Instructor (step 1)

 


Wed, Oct 23, 2002 -- Midcourse Feedback Procedure

Hello to all!

Please take a moment to read the Midcourse Feedback Procedure below and submit your response to James Shepard by Sunday. I appreciate your participation in this evaluation process.

I am taking an online course myself entitled "Online Teaching and Training." I would like to introduce the concept of a Midcourse Feedback Procedure. The procedure begins with a call for a student volunteer. The student volunteer is responsible for receiving e-mails directly to their e-mail address from fellow learners in the class. The volunteer then compiles the responses to the below question without attaching any names to the comments and forwards it to the instructor.

The two questions are:

(1) List three areas that are working well in this course

(2) List three ways to improve the class

The volunteer will then combine all of the messages-verbatim and unedited- (you could cut and paste) and send them to the instructor. The information will be used to make changes for the remaining of the course and for future courses.

I would like responses collected through this Sunday, have the volunteer combine them, and forward them to me by next Wednesday, December 30.

James Shepard was the first to volunteer for this project. His e-mail is jsheps44@hotmail.com Please e-mail your response on the two questions by this coming Sunday, December 27th. Thank you.


Figure 2. Blackboard Posting to Class from Instructor
after volunteer secured (step 2)

 

Midcourse Feedback-BUS105 December 30, 2002 

Thank you all for participating in the midcourse feedback. Below are your comments and in italics are my responses. 

1. List three things that are working for you in this class: 

2. List three ways to improve the course:
 


Figure 3. Blackboard posting of Responses (step 3) by Instructor

 

1. List three things that are working for you in this class:
 

Student Responses:

"I really enjoy Kelly's class. She helps out so much. She is always available. Now if we could talk about some other internet classes.

I wouldn't change anything."

"I like that we can work ahead if we want to.

I like the prompt responses from the teacher.

I like the CP interaction with my classmates."

"Ability to access site.
Great communications link
Instructors aid with problems prompt.

No thoughts on how to improve procedures. Structure of class is excellent."

"Three things that are working well are the discussion boards, the ease of the website, the layout and easy access to information (grades, assignments, so forth) I also like that our teacher communicates with us often."

"The student/teacher interaction in this course is unmatched. Great job!

The discussion forums for HW help and the whole Create-a-Problems concept is very good.

I think the deadline dates and times are right where they need to be." 

Instructor Reply:

I am glad that most of you enjoy the format of the class. It has taken a long time to develop. I believe the CP problems add to student understanding and communication among learners. I make it a priority to be in contact with you on a regular basis, so I am glad to learn I am doing a good job! J
 

2. List three ways to improve the course:

Student Response:

"I think more time is needed in the quiz portion. I run right to the wire on most of the answers trying to do the calculations. Also as far as tests go I wish we could use our handouts or sample tests. It is hard to keep everything straight. Also the CP are good but some of the questions are confusing or misleading they are a good learning tool but I will use myself as an example I have a hard enough time without confusing others too."

Instructor Response

The quizzes are timed 30 minutes the same as the “live class.” That group also takes the same quizzes and the time limit is adhered to. Those that aren’t finished (which are only a few) turn it when the time limit is up. The quiz is timed at an interval to measure comprehension and understanding at an acceptable level (meaning that 98% -100%of the class is done by that time).

Student Response:

"I think that it is difficult (for some) to get to campus for the exams.

Instructor Response

It is the Instructor’s prerogative to request exams to be taken on campus. In the introductory material sent to all learners and in orientation it was stated that this was a requirement. To protect the integrity of the exams, I take the option of on campus testing and make it a point to inform students as early as possible (and it’s in the course description about 4 exam weeks as well). I feel I’ve allowed flexibility in the exam week leaving students plenty of options on dates and times. I do not plan on having exams online and feel that I’ve done a good job in incorporating test integrity with on-campus exams in a flexible time-frame for students.

Student Response:

I think that it would be easier if things were due on the same day instead of some things this day and some things another day.

Instructor Response:

The create-a-problem exercises do not allow everything to be due on the same day since your partner needs time to respond. Also, students need time to review the homework sheet with the correct answers before taking their quiz. I felt that the Wed./Sunday due dates were well spaced out and easy to remember while also incorporating time for students to learn from reworking homework problems and from the Create-a-Problems (that’s why I don’t have everything due on Wed). To have HW, CP, Quiz, and CP responses all due on Wed. I think would be too much too soon without allowing time for things to “sink in.”

Student Response:

I cannot think of a third, all in all this class runs pretty smoothly."

Instructor Response:

Good ….. that makes me happy!

Student Response:

"I feel a time limit on quizzes is little too much. The purpose of the quiz is to help you do better it seems some instructors are so geared on punishment and not exactly helping you. Homework assignments that need to be handed in at about 10:00.

Instructor Response:

I’m not really sure on what this comment means. The quizzes are timed just as in a regular course. As well, deadlines are in place just as they are in a regular course. The homework assignments need to be passed in by 11:00 p.m. on Wed. This deadline shouldn’t be a problem unless students work last minute. The 10:00 p.m. deadline was for questions to the homework on the discussion link, not the homework to the drop box (you still have that extra hour). This is because I will reply to HW questions to the discussion link up until 10:00 p.m. on Wednesday nights. This resulted when a HW question was posted at 10:55 p.m. on a Wed. with an expectation for an immediate response from the instructor. If students want a response, then courtesy needs to play a role as well. I’ll check up until 10:00 p.m. and answer.

The purpose of the quiz is as a learning tool. That’s why they are only weighted at 10% and that’s also why you get an immediate response to the questions you had incorrect. I believe by allowing answers to appear helps the learner understand errors and students can also e-mail me any questions regarding the quiz as some of you have. The time limit is in place as in the regular classes.

Some students have an expectation that Internet courses don’t have due dates or time limits and that they are self-paced. That myth is dispelled in my introductory letter sent to students before orientation and also on the Announcement page. This posting and direct mail letter to students informs students before orientation and class start that there are due dates and time limits and exam weeks so the student can make an informed decision on if the platform of the course meets their expectations.

I hope this has served as a useful learning tool for all of us. Again, thank you for your participation in the midcourse feedback process.

Sincerely,

Kelly Littlefield


 

Fri, Nov 01, 2002 -- Quiz-Building Speed
Hello!

I forwarded the Mid-term Feedback results to my boss, Gordon Neimi, who teaches Accounting courses here at NMC and also has taught BUS105 in the past. Below is his recommendation and I thought you may be interested in it.

One suggestion for the students that need more time on the quizzes is to do more problems from the book to improve understanding and speed. The students get faster with speed and the learning curve theory backs that up. Also, in a business setting time is money so in a business class it is important for students to not only demonstrate knowledge but speed and accuracy as well.

Sincerely,

Kelly Littlefield
 

Figure 4. Blackboard posting from Instructor

 

Learning Outcome of Midcourse Feedback Process

The learning outcome from this process is that the instructor was able to assess the development of the course. The instructor did not implement change, since the majority of the comments provided positive feedback. It reinforced to the instructor that the learning platform, navigation of the course content, and assignments were all contributing to the success of the course.

A personal reflection by the instructor was that the feedback procedure reinforced the concept of the Business Math Create-A-Problems which was a learning tool developed by the instructor to form a learning community. The students interact with one another using the discussion thread and create mathematical word problems based on course concepts that a fellow learner needs to solve. The partner solves the problems and sends them back to the author via the discussion thread. The author then must correct any mistakes in detail and/or provide positive feedback to the partner. Meanwhile, the author also receives problems from another learner which he/she is responsible to solve and the student that created those problems must provide feedback in a positive tone.

The Midcourse Feedback Procedure provided additional feedback to the Discipline Chair on the student assessment of this class. As indicated in Figure 4, the Midcourse Feedback Procedure was copied and sent to the Discipline Chair for his review and comments. The procedure also nurtured the development of the learning community for the course. By posting the student comments verbatim as well as the instructor's response to the course room, it allowed other learners to read the comments from their peers. The outcome has been more interaction by the learners in the discussion thread. As well, more positive e-mails were received by the instructor regarding student issues. According to White & Weight (2000) the midcourse feedback is especially useful in online classrooms because it is more personal than individual student ratings and allows student to read what other students are thinking (p. 181).

Benefits and Limitations of Midcourse Feedback

The process of the midcourse feedback builds the online learning community. Its timing allows instructors to make changes during the same class, and its feedback contains specific suggestions on how to make those changes. The procedure is a tool that emphasizes to students tat they have a role in reshaping the method of instruction and subsequent learning outcomes (White & Weight, 2000). The process allows the instructor to gain insight from the perception of the student on how the class is going before it is too late to restructure the class. Continual evaluation is essential and faculty must be willing to adjust their course materials to accommodate what students are telling them while still achieving the learning outcomes (Bauman, 2002).

"Maintaining the midcourse feedback as a formative process-voluntary, anonymous and confidential helps to encourage open communication and a meaningful online student feedback" (White & Weight, 2000, p. 181). If the feedback tool is forced upon students or the students feel threatened with an instructor's recourse and/or anonymity is breeched, the feedback process will not produce honest, reliable student assessment of the class.

Conclusion

The Midcourse Feedback Procedure provided the opportunity to receive feedback from online students at a point mid-way through the course enabling the instructor to gauge the learning methods in the course and redirect if necessary. The formative evaluation tool was outlined in the course text entitled The Online Teaching Guide written by Ken White and Bob Weight, both faculty members from the University of Phoenix.

The Midcourse Feedback Procedure is a formative evaluation tool that allows the instructor to solicit student assessment of the online class in a non-threatening way. The process is voluntary by students, the students' comments remain anonymous and confidentiality by the volunteer that assimilates the comments is guaranteed. This creates a secure environment in which students feel safe in sharing their true assessment of the course.

The Midcourse Feedback Procedure starts when the instructor explains to students as early as possible the purpose of the feedback tool and stresses the importance that their feedback will help shape the course for the remainder of the semester. An e-mail message goes out to the students asking for a volunteer. The first person that responds is usually the volunteer. The instructor composes two questions that comprise the evaluation tool. They consists of (1) List three things that are working for your in the course, and (2) List three things to improve the course. The volunteer collects the e-mail and copies and paste them to word to protect student anonymity. The instructor receives the student responses unedited by the volunteer, in one document, with no names. The instructor can choose to assimilate and reflect upon the feedback and not post a response to the students. However, it is strongly recommended by White and Weight that the instructor constructively responds to each and every comment and posts it to the public platform entity of the learning community. This provided an opportunity for other learners to read the comments from their peers along with the instructor's response, which leads to developing the learning community.

The outcome of the Midcourse Feedback Process provided the instructor with feedback to make changes to the course if warranted. In this case, it reinforced receptivity by students with the Create-A-Problem interaction and the way that the course was designed into four modules allowing for easy navigation to find assignments, quizzes, homework, and helpful hints on specific chapters. It also allowed the instructor to share the feedback with the Discipline Chair who also suggested a way of improving student time on quizzes that was subsequently posted to the course room for students to read.

Another outcome of this Midcourse Feedback Process was that the instructor went on to use the concept in all classes (meaning face-to-face classes) and provided the same type of feedback to students in an oral discussion after the comments were reviewed. This too, has nurtured a more positive classroom environment since student feel that their input makes a difference in how the rest of the class is conducted.

The Midcourse Feedback Process is a formative evaluation tool that can change the direction of the course well in advance of the end of the semester. The instructor gains honest student assessment feedback in a non-threatening way and the students gain a sense of input on their learning outcomes while also developing either a learning community or a more positive classroom camaraderie. The Midcourse Feedback Process outlined by White and Weight in The Online Teaching Guide is an excellent formative student assessment tool when put into practice in both online classes and face- to-face traditional classes.

References

Bauman, P. (2002). Student retention: what you can control, and how.
Distance Education Report, 6,
p. 8.

Brookfield, S.D. (1995). Becoming a critically reflective teacher. SanFrancisco: Jossey-Bass, Inc.

Palloff, R., and Pratt, K., (1999). Building learning communities in cyberspace: effective strategies for the online classroom. San Francisco: Jossey-Bass, Inc.

Preece, J. (2000). Online communities designing usability, supporting sociability.
New York: Hohn Wiley & Sons, LTD.

Scagnoli, Norma, I., (2001). Student orientation for online programs. Journal of Research on Technology in Education, 34, 19-27.

White, K., and Weight, B. (2000). The online teaching guide: a handbook of attitudes, strategies and techniques for the virtual classroom. Needham Heights: Allyn & Bacon.

 

About the Author

Dr. Kelly Bruning

Dr. Kelly Bruning has been working in business, marketing, and education for over twenty years. She is now a fulltime instructor at Northwestern Michigan College. Her background consists of a broad range of business knowledge including organizational behavior, economics, marketing, finance, human relations, and information technologies.

She completed her Doctorate Degree in Business Management and Organization with a specialty in Information Technologies Management in November of 2003. She earned her MBA from Lake Superior State University.

Dr. Bruning has been active in online teaching and training since its introduction to the academic arena. She has designed content for online course and now teaches online courses at both the graduate and undergraduate level. She earned a graduate certificate in online teaching and training while writing her dissertation.

She can be reached at kbruning@nmc.edu.

 

go top

 June 2005 Index

Home Page