13th Annual ICTCM
Paper Presenter: Dr. Jacci White
408 Sunset Dr. S.
St. Petersburg, FL 33707
Assistant Professor of Mathematics
Saint Leo University
(352)588-8338
fax (509)692-4210
jacci.white@saintleo.eduTitle: Online testing: The dog sat on my keyboard.
Abstract: This session will highlight some advantages and disadvantages of several online models for student assessment. These models will include: live exams, multiple choice tests, essay exams, and student projects. In addition, real student responses and "problems" will be used as prompts to improve models of authentic online assessment in mathematics.
Online course models and enrollment are growing exponentially. As a natural result, student assessment problems are growing at an equal rate. One skill that a majority of students learn in the online classroom is creativity, students have become increasingly creative, and bold, in their attempts to improve grades. Email does not register as a permanent and legally binding document to most students. Hence, a bold new student is cropping up that will put together their own incriminating documentation in an effort to save a failing grade. Administration is on a mission to increase student enrollment for obvious reasons, in response; faculty must be on a mission to improve quality. Quality control appears to stop at student assessment, the area where control is most important in order to uphold academic standards for documenting student ability. Effort rather than ability often determine online course grades, especially when that effort is directed towards manipulating the testing systems. This paper will highlight four main styles of online student assessment by comparing and contrasting the advantages and disadvantages of each. Many of the issues will be supported by original emails from online students.
Live exams are popular in online courses that are limited to students who reside in specific geographic regions. Institutions who utilize live exams in online curriculum typically limit their service area to the physical college vicinity or have partnerships with institutions in other areas that allow them to expand their student pool to include the regions within an easy access distance of the other campus sites. By limiting the geographic region of service, instructors can require students to take live exams at a campus location during regularly scheduled testing center hours. Two advantages to this system is the ability to control the amount of time allotted for testing as well as checking the identity of the person taking the exam. Two disadvantages include the limited service area of the online medium and the inability to prevent students from discussing test questions with others who may be taking the test at a different time. Using multiple versions of the same exam may reduce the latter disadvantage.
Multiple choice or true and false online exams are very popular at institutions with enrollment that does not follow any geographic boundaries. The advantages are a shorter list when using this model. First, the exams are available online so the students can access them anywhere at any time. Second, the pool of questions on the exams can be randomized to reduce the possibility of sharing answers. Third, the exams can be automatically scored online with the scores then available to the student and the instructor, saving the instructor many hours of grading. Fourth, students can get immediate feedback on their work when the exams are graded online. The disadvantages are numerous, but can be categorized as "technical difficulties". The primary disadvantage from an instructional standpoint is the poor ability to accurately reflect student learning using these sorts of test questions. Once you move on from the instructional problem, you run into a seemingly infinite list of possible technical problems. Students can become very creative when their grade is at stake. For example: "I dropped my calculator on the keyboard and the test submitted" (shall I ask the student how the calculator happened to bounce on the enter key since you must select enter to begin the submit process and again to choose the yes command for submitting), "something must be wrong with my server because it says that my quizzes were submitted but I did not take them" (interesting how the server not only submitted the quizzes, but filled in answers as well), "I cannot access my quiz to submit the answers but here they are" (amazing how you were unable to access your quiz yet it was submitted so all of the correct answers were returned to you with your grade of zero. Could it be that you were able to write all of the correct answers down after you submitted your quiz blank and the answers were sent back to you with your score of 0%)… Needless to say the list goes on.
If we move on to essay exams, the advantages are numerous. However, the main disadvantage is enough to steer most instructors to another assessment strategy. The disadvantage is the time it takes to thoroughly grade each question for each student. This is a disadvantage that should not be an issue in education, but the nature of online courses has been to increase class enrollment to double, triple, or even ten times that of traditional class sizes. While we count the high enrollments, keep in mind the extensive time commitment for answering all student questions while prompting students to interact with each other on the message board. Of course we do not want to forget the hours spent in the chat room with small groups of students since the atmosphere is not conducive for the entire class to attend at the same time. I almost forgot the extra time it takes to type answers instead of verbalizing different concepts. Now we can begin to see why the time that is required to grade essay exams can be overwhelming for online faculty.
Lastly we have student projects. This assessment strategy would tend to belong to the same class as essay exams. However, essay exams can be taken and submitted automatically online. Hence, a disadvantage of projects is finding a common way to submit the projects to the instructor. Typically a combination of technologies must be available in order for all students to have the opportunity to complete their work and have the instructor able to open it completely. Some students can find common software while others must fax their final project. Of course a project is a good opportunity for group work, so the time required to grade a project could be reduced when several students work together for the same grade. Then again, what will stop a student from slacking and taking the group grade someone else earned? I guess some problems are the same between traditional and online teaching environments.
Given many different assessment methods, I find myself returning to the question: Do online students learn as much as traditional students? According to Dutton, Dutton, & Perry, (1999) from North Carolina State University, online students can perform at least as well as traditional students. However, their results show that not any student, randomly selected, can do as well in an online class. Undoubtedly self-selection means that students with greater computer skills and/or greater maturity are more likely to opt for an online course. Goldberg (1998) compliments this idea with the statement "Jack Wilson, faculty dean at Rensselear Polytechnic Institute, where almost one-third of graduate students are taking courses at a distance, said the off-campus students perform just as well as their on-campus counterparts in the same courses." We can assume that graduate students would have a greater maturity than typical undergraduates.
Some studies (Mock, 2000) have shown in-class students scoring higher than web-based students on unit tests, in each case the difference between these two groups was not significant. This idea is supported by Ryan (2000) where the final grades for online and lecture participants were not significantly different for either course offering. On the other hand, a study by Redding (2000) showed that an online group is the most successful at cognitive learning as measured by the end of course examinations. The results of the study do provide strong support for the conclusion that online instruction for individuals entering the insurance field is highly effective, and can be more effective then traditional classroom delivered instruction.
Bibliography:
Dutton, J., Dutton, M., & Perry, J. (1999). Do online students perform as well as traditional students? Manuscript submitted for publication.
Goldberg, D. (1998, April 5) Teaching Online - Education Review. Washington Post
Mock, R. L. (2000). Comparison of online coursework to traditional instruction Unpublished master’s Thesis. Michigan State University, Lansing, MI.
Redding, T. R. & Rotzien, J. (2000) Comparative analysis of SDL online training with traditional classroom instruction. Paper presented at the 14th International Symposium on Self-Directed Learning.
Ryan, R. C. (2000, January) Student assessment comparison of lecture and online construction equipment and methods classes. The Journal, 27(6).