The Process
Are you thinking about designing an online assessment for a Flipped classroom?
This article examines four steps I undertook when teaching a first year undergraduate accounting course.
First, consider whether the tool you are thinking about is the right tool for what you want to assess.
Second, does this online assessment align with the rest of the course curriculum.
Third, think about what strategies you can use to engage students with the online assessment.
Lastly, both during the semester and at the end of semester consider how will you evaluate this online assessment, including the online tool itself.
1. Think about matching the right online tool to the assessment
I designed an assessment for a first year undergraduate financial accounting course.
The assessment takes the form of a team quiz to be undertaken during a flipped classroom tutorial by students using the technology Socrative. The Socrative website was selected as it has a team quiz function which includes a team performance monitoring tool, the graphs of which are quirky and fun—rockets racing across the screen! The instructor loads up the multiple choice questions into the Socrative website in the instructor view before the tutorial starts. During the tutorial, students work in teams to complete pre-assigned questions and when all teams have completed the questions the instructor starts the team quiz in Socrative.
Students login to the Socrative website and then complete the quiz as a team. The students are asked to complete the quiz on a single device. This will require the team members to discuss and share ideas about the questions and then select the team’s choice of answer. The use of multiple choice questions (MCQs) is established as being suitable in the accounting discipline and students perform similarly in written and MCQ accounting questions (see Bible et al., 2008). In this course the speed of automatically marking MCQs, which Socrative helps to provide, is highly beneficial in returning feedback to students on their performance in a timely way. This is especially useful in classes where student numbers are over 1,000.
As students complete the team quiz they find out whether their answer is correct or incorrect in real time. Students, within their group, can immediately reflect on this feedback and this presents a ‘peer-learning’ opportunity.
This is consistent with the team-learning model designed by Michaelsen (1998) (see also Michaelsen et al. 2002) and used by other academics (e.g. Lancaster and Strand, 2001; Brink, 2013; Opdecam and Everaert, 2012).
2. How does online assessment align with the rest of the course curriculum?
The MCQs in the quiz are based on the tutorial questions and require students to apply principles to facts. I preferto use meta-questions, where they indicate a previous tutorial question, e.g. ‘Refer to Q2, now consider…’. The facts of the question are modified or supplemented such that the students need to apply the principles and analytical approach of accounting in order to solve these MCQs.
In this way students become aware that in the business world the facts or evidence can change, over time or with discovery of information, requiring a re-analysis. This prepares students for a dynamic information environment as they questions have dynamic elements introducing new information or altering existing information (Barsky et al., 2003). The role of accountant requires an ability to add new information into analysis as it becomes available. Also, accountants are increasingly being required to prepare ‘what-if’ analysis for managers.
Furthermore, the team quizzes can help students to develop soft-skills, such as communication and interpersonal skills which are in demand by employers of accounting graduates (de Villiers, 2010).
3. Consider strategies that engage students with the online assessment
My strategy to engage students with this assessment is that the Socrative system itself is highly engaging and easy to use. This is particularly the case with the team quiz mode.
Students were introduced to and prepared for using Socrative through a guiding set of slides presented to individual tutor groups. I also provided an opportunity for students to familiarize themselves with the Socrative system in the first tutorial week in which they undertook a team quiz for ‘practice’, i.e. no marks. It is important to provide an opportunity for students to get comfortable with the technology in a low-risk type situation (Brown and Knight, 1994). In the first week of tutorials I felt that it was important that the team quiz did not have marks attached as students can be challenged by technology that they are not familiar with and need a ‘learning space’ to get comfortable without the potential consequence losing marks while students’ were learning the technology (Ehrmann, 2010).
By the second tutorial week students were comfortable with the Socrative system and marks for correct answers were introduced. Within the design of the assessment a degree of flexibility was allowed, to only count the ‘best 4 out of 5 team quizzes’ for the overall mark. This way, each of the quizzes would be worth 5%. This ‘best of’ approach can reduce anxiety of students about an assessment as they have a greater chance of obtaining higher marks. Also this ‘best of’ can be useful in case of technological problems with a system or device.
Having a team quiz also helps reducing technical problems, as each team has six team members and they are likely to each have one device within the group that works rather than a single student with one device.
4. Make a plan for evaluating the online assessment
In evaluating the quiz I receive feedback through multiple perspectives both during the semester and at the end of the semester after the final results are released; I survey students, tutors and lecturers in the first year accounting course. This is crucial in hearing their opinion and experience on the Socrative system and whether using it for team quizzes meet learning objectives. I specifically ask tutors whether they believe the Socrative system helps to facilitate the outcomes of ‘peer-learning’ and team learning. Investigating whether the technology of Socrative was appropriate for the team MCQs is another concern, to be constantly re-evaluated. During the semester an academic with the School of Accounting who teaches on another course will be invited to peer review the team quiz and the Socrative tool. The student’s engagement with the tool during the semester will be also evaluated using the data collected through Socrative usage.
Socrative was a tool I discovered through tried and tested experiences of other staff members. I personally tested the features of the system using other academics as ‘test students’ to see how the system worked and whether it was suitable. I always keep in mind the continual development of online tools and programes for educational assistance, and while Socrative is useful in my class now, it may not be in the future. It is important to always use the correct tool for the task at hand, rather than adapt the lesson to the program (although a certain amount of this is expected).
References:
- Barsky, N. P., Catanach, A. H. and Kozlowki, B. M. (2003) ‘Creating strategic responses to critical risks facing accounting education’, Issues in Accounting Education, 18(4), 445-462.
- Bible, L., Simkin, M. G. and Kuechler, W. L. (2008) ‘Using multiple-choice tests to evaluate students’ understanding of accounting,’ Accounting Education, 17(1), 55-68.
- Brink, A. G. (2013) ‘The impact of pre- and post-lecture quizzes on performance in intermediate accounting II’, Issues in Accounting Education, 28(3), 461-485.
- Brown, S. and Knight, P. (1994) Assessing Learners in Higher Education, RoutledgeFalmer, Taylor & Francis Group, Abingdon, Oxon.
- de Villiers, R. (2010) The incorporation of soft skills into accounting curricula: Preparing accounting graduates for this unpredictable futures, Meditari Accountancy Research, 18(2), 1-22.
- Ehrmann, S. C. (2010) ‘Asking the right questions: What does research tell us about technology and higher learning?’, Change: The Magazine of Higher Learning, 27(2), 20-27.
- Lancaster, K. A. S. and Strand, C. A. (2001) ‘Using the team-learning model in a managerial accounting class: An experiment in cooperative learning’, Issues in Accounting Education, 16(4), 549-567.
- Marriott, P. (2009) ‘Students’ evaluation of the use of online summative assessment on an undergraduate financial accounting module’, British Journal of Educational Technology, 40(2), 237-254.
- Marriott, P. and Lau, A. (2008) T’he use of on-line summative assessment in an undergraduate financial accounting course,’ Journal of Accounting Education, 26(2), 73-90.
- Michaelsen, L. (1998) ‘A workshop on using small groups effectively,’ AAA Annual Meeting, New Orleans, LS, Continuing Professional Education.
- Michaelsen, L. K., Knight, A. B. and Fink, L. D. (2002) Team-based learning: A Transformative Use of Small Groups, Praeger Publishers, Westport, CT.
- Opdecam, E. and Everaert, P. (2012) ‘Improving student satisfaction in a first-year undergraduate accounting course by team learning,’ Issues in Accounting Education, 27(1), 53-82.
Feature image courtesy of Flickr, DavidGuthrie.