Breaking Down Barriers in Blended Learning

ETHOS Issue 23, October 2021

BLENDED LEARNING—A BEHAVIOURAL CHALLENGE

While online modes of learning have become much more prevalent in the field of corporate training, most businesses and training institutions have not abandoned traditional face-to-face modes in favour of online learning. Instead, blended learning—bringing the best of both online and in-person modes of learning together—has emerged as a popular approach. However, implementing blended learning for adult learners is not straightforward. Even though it enables learning to take place outside the walls of training rooms and grants learners autonomy and flexibility to learn at their own pace and time, blended learning comes with its own challenges.

The Civil Service College (CSC) in Singapore has shifted towards more blended learning in recent years, as part of its efforts to redefine learning. The flipped classroom model is a common blended learning approach used at CSC. This model ‘flips’ the traditional learning experience by first introducing pre-course learning materials (e.g., videos, articles) for learners to engage with in their own time, before they attend in-class sessions to discuss and apply this learning.

Similar to working adults who enrol in corporate training, CSC participants sign up for these courses on their own or are nominated by their supervisors to attend. The challenges arising from blended learning are typically associated with pre-course learning. Compared with in-class sessions, pre-course learning demands more from adult learners, because it requires them to:

• be able to learn independently without much guidance from the trainer; and

• be motivated to learn independently, and to set aside time to complete their pre-course learning.

Blended learning also assumes that all learners will easily adapt to this way of learning and that they have the opportunity (time) to complete their pre-course learning while managing work and personal commitments. From our observations, CSC course participants have been struggling to complete their pre-course learning, with completion rates as low as 11% in some cases. This is an important issue to address, given that pre-course learning can affect learning effectiveness during in-class sessions.

A Behavioural Insights (BI) approach offers a useful lens through which to understand learners’ behaviours and why they sometimes fall short of completing their pre-course learning. A team of officers from CSC’s Institute of Governance and Policy (IGP) and Learning Futures Group (LFG) conducted a study in 2021 to identify behavioural barriers and enablers affecting pre-course learning. Their findings suggest ways to design interventions to nudge¹ more learners to complete their pre-course learning.

Compared with in-class sessions, pre-course learning demands more from adult learners.

UNCOVERING DIFFERENT PRE-COURSE LEARNING BEHAVIOURS

A total of 291 participants² from CSC’s programmes responded to a poll from 17 May 2021 to 16 July 2021, which asked about their ability, motivation and opportunity to complete pre-course learning.³

Respondents indicated “agree”, “disagree”, “pass” or “undecided” on 27 statements (e.g., “I usually complete my pre-course learning”). Respondents could also contribute their own statements for others to vote on.

Using the COM-B model for behavioural change, each poll statement was designed to uncover one of the three factors (Capability, Opportunity and Motivation) of Behavioural change (see Figure 1):⁴, ⁵

The study identified three main opinion groups based on how likely they were to complete their pre-course learning. The Most Likely Group was the largest (51% of all respondents), followed by the Moderately Likely Group (29%) and lastly, the Least Likely Group (19%) (see Figure 2). Unsurprisingly, the Least Likely Group struggled with the most barriers from all three components of behavioural change. In contrast, the Moderately Likely Group faced Motivation and Opportunity barriers while the Most Likely Group only faced Opportunity barriers.

Figure 1. Overview of the COM-B Model for Understanding Behaviour
Source: Figure adapted from S. Michie, M. M. van Stralen, and R. West⁶

Figure 2. Opinion Groups Based on Respondents’ Likelihood of Completing their Pre-Course Learning

What's Stopping Learners from Completing Their Pre-Course Learning?

CAPABILITY BARRIER

1. Least Likely Group faced difficulty in learning independently.

Around half of the Least Likely Group agreed that they had difficulty in learning independently, whereas majority of the Most Likely Group and Moderately Likely Group disagreed with that statement. It is thus important to identify learners who need greater support for pre-course learning.

OPPORTUNITY BARRIERS

2. Busyness at work, and intention-action gap are key barriers for many learners.

59% of all respondents, including most of the Least Likely Group and Moderately Likely Group, agreed that they were too busy at work to complete their pre-course learning. This problem could be further compounded if adult learners think pre-course learning should be completed during work.

Nearly half of all respondents—including 80% of the Least Likely Group—forgot to start their pre-course learning despite intending to complete it. This finding suggests learners need help to narrow their intention-action gap.

3. Least Likely Group thought not completing pre-course learning was a norm.

69% of the Least Likely Group agreed that there was no point in finishing their pre-course learning because many participants would not finish it. This perceived social norm could be damaging if trainers forgo discussion time to go through the pre-course learning again during in-class sessions.

MOTIVATION BARRIERS

4. Least Likely Group believed there were no consequences of not completing their pre-course learning.

71% of the Least Likely Group indicated that it did not matter if they complete their pre-course learning because they believed the trainer would go through the same content during the in-class session. This perpetuates the notion that pre-course learning doesn’t matter.

5. Least Likely Group and Moderately Likely Group perceived the benefits of pre-course learning to be low and the cost of completing pre-course learning to be high.

Only a minority of the Least Likely Group and Moderately Likely Group felt that they could remember what they had learnt through their pre-course learning at CSC. This perceived lack of benefit could reinforce negative perceptions of pre-course learning and lower learners’ motivations.

While 70% of all respondents reported doing their pre-course learning outside working hours, a majority of the Least Likely Group and Moderately Likely Group felt that they should not do so. These findings suggest a gap between organisational expectations, which assumes learners are willing to learn in their private time, and learner’s expectations, in which such learning is regarded as a personal cost.

Overall, most learners faced some forms of barriers even for the Most Likely Group (see Figure 1). Unsurprisingly, the Least Likely Group struggled with the most barriers and it was the only group with the Capability barrier.

Figure 1. Overview of Barriers to Completing Pre-Course Learning by Each Group
Source: C. Lim, M. Loi, and C. Wan

Note

1. T. C. Lim, M. Loi, and C. Wan, “Pre-Course Learning Study: Findings on Barriers and Enablers” (Institute of Governance and Policy and Learning Futures Group, Civil Service College, 2021).

CLOSING THE INTENTION-ACTION GAP

The CSC study found that closing the intention-action gap was a common challenge for many learners: nearly half of all respondents said that they forgot to complete their pre-course learning despite having intentions to do so. This was further compounded by busyness at work and reluctance of both the Least Likely Group and Moderately Likely Group to spend their personal time on pre-course learning.

What can be done to address these barriers to learning? The following interventions include a combination of what different teams in CSC have tried and possible nudges to change behaviour.

1. Set deadlines for the same week to nudge learners to act “now” instead of “later”

Tu and Soman have found that people tend to think about time in categories (e.g., week, month, year) instead of thinking about it continuously.⁷ As a result, people are more likely to complete a task with a deadline set in the same week as compared to a deadline set next week, despite having a shorter time to complete it. A CSC study in 2020 found that setting the pre-course learning deadline on the Friday of the same week as the notification email was sent out contributed to increasing the completion rate of pre-course learning.

Researchers at CSC conducted an email experiment in 2019 to study if behavioural nudges could lower participants’ barriers to completing their online pre-course learning component.

Nudge to Learn:1 An Email Experiment to Improve Online Learning Completion Rate for Blended Workshops2

Researchers at CSC conducted an email experiment in 2019 to study if behavioural nudges could lower participants’ barriers to completing their online pre-course learning component. The experiment was carried out for participants of two different courses. They divided participants into two groups, with one (Control) group receiving a notification email with instructions for the online learning component. The other (Treatment) group received a modified notification email that included the following nudges aimed at increasing the likelihood of them completing the task:

• A “[For Your Action]” call to action was inserted in the email subject line to increase saliency that participants act after reading the email. This was especially tailored to public officers who are familiar with calls to action in email subject lines.

• The deadline was set on the Friday of the same week as when the participant received the notification email.³ In contrast, the Control group’s deadline was set to the next week, giving them more time (i.e., around two weeks) to complete their online learning.

• Only the duration for each online module was indicated, without showing the total amount of time needed to complete all modules. This was to lower the perceived time cost of completing the task.

The experiment found that around 23% more participants in the Treatment group completed their online learning before the given deadline compared to the Control group (44% versus 21%). Furthermore, 25% more participants in the Treatment group than Control group participants completed the online learning before the workshop (88% versus 63%).

Figure 1. Example of Email Sent Out to Treatment Group⁴
Source: T. Cheng, S. Kim, and K. Koh

Notes

1. LEARN is the Civil Service College’s digital learning app for public officers in Singapore.

2. C. Lim, M. Loi, and H. V. K. Do, “Nudging a Higher Completion of Pre-Course Online Assignments for Blended Learning” (Institute of Governance and Policy, Civil Service College, 2020).

3. To minimise possible negative sentiments from participants, we sent them an email one week before the treatment email to only inform that they will receive an email the following week with pre-course learning instructions.

4. See Note 2.

2. Send email reminders but keep them as simple as possible

It was evident from the 2021 CSC study that learners needed help to narrow their intention-action gap. Furthermore, 59% of all respondents indicated that they needed regular reminders to complete their pre-course learning. Sending simple regular email reminders could help participants narrow this intention-action gap.

3. Set aside time during the workshop for pre-course learning

Not every learner has the privilege to learn in their own time even if they want to. Trainers could set aside protected time for adult learners to complete their pre-course learning during working hours, especially if the learning is primarily for work purposes. Learners would still have some autonomy to learn at their desired pace within the time provided during the workshop, without using their personal time to do so. In the 2021 CSC study, 73% of all respondents agreed that there should be protected time for pre-course learning.

When several CSC programmes incorporated this approach, participants were observed to put in more effort in completing their pre-course learning compared to having them finish the pre-course learning in their own time.

An experiment conducted by Iryna Nikolayeva et al. in a blended university course setting tested the effectiveness of sending weekly email reminders to help students overcome procrastination and complete their quizzes.

Simple Email Reminders Work Best

An experiment conducted by Iryna Nikolayeva et al. in a blended university course setting tested the effectiveness of sending weekly email reminders to help students overcome procrastination and complete their quizzes. Students were randomly assigned to the control condition (no email) or one of the treatment conditions, with different email content (such as the level of personalisation).

The study found simple non-personalised email reminders (“Hello, as a reminder, the next deadline for quizzes is on the DD/MM/YY! You can view them here. All the best for future work!”) to be effective in helping students complete more quizzes throughout the course.

Furthermore, the simple non-personalised reminders were more effective than complex personalised reminders (which included the student’s name, reminder, a summary of the student’s recent results, advice to seek support from peers and teachers, and a reflection exercise). The researchers hypothesised that providing students with too much information demotivates them from taking action.¹

Note

1. I. Nikolayeva, A. Yessad, B. Laforge, and V. Luengo, “Does an E-mail Reminder Intervention with Learning Analytics Reduce Procrastination in a Blended University Course?” Addressing Global Challenges and Quality Education. EC-TEL 2020. Lecture Notes in Computer Science 12315 (2020): 60–73, https://doi.org/10.1007/978-3-030-57717-9\_5.

People are more likely to complete a task with a deadline set in the same week as compared to a deadline set next week, even if they have a shorter time to complete it.

HELPING LEAST LIKELY GROUP COMPLETE THEIR PRE-COURSE LEARNING

Unlike the other groups of learners, the Least Likely Group faced additional barriers. A number of behavioural interventions offer possible ways to address these.

1. Use social norms

Social norm nudges could be used to encourage the Least Likely Group to complete their pre-course learning, since they tend to be influenced by how they think others behave.

A published 2021 study on a leading Massive Online Open Courses (MOOCs) platform in China sought to determine the effectiveness of call-to-action messages on MOOCs users’ completion of course assignments before the due date.

Social Norms Are More Effective than Simple Call-To-Action Messages

A published 2021 study on a leading Massive Online Open Courses (MOOCs) platform in China sought to determine the effectiveness of call-to-action messages on MOOCs users’ completion of course assignments before the due date. Four types of call-to-action messages were tested:

1. Simple reminder.

Students were notified about the start of their assignment and encouraged to submit their assignment as soon as possible.

2. Deadline reminder.

On top of the simple reminder, students were also informed of the assignment’s due date.

3. Social norm.

On top of the simple reminder, the message also mentioned the proportion of the student’s peers who had completed the assignment thus far.

4. Financial incentive.

On top of the simple reminder, students were notified that they would be entered into a lottery to stand a chance of winning an unspecified monetary gift if they complete the assignment on time.

The social norm nudge significantly increased the probability of participants completing the assignments before the deadline compared to the simple reminder.¹

Note

1. N. Huang, J. Zhang, G. Burtch, X. Li, and P. Y. Chen, “Combating Procrastination on MOOCs via Optimal Calls-to-Action”, Information Systems Research 32, no. 2 (2021): 301–317, https://doi.org/10.1287/isre.2020.0974.

2. Highlight the consequences of not completing pre-course learning

A majority of the Least Likely Group indicated that they did not remember much of what they learnt through CSC’s pre-course learning and believed there were no consequences for not completing it. More learners may complete their pre-course learning if we highlight the consequences of not doing so (e.g., “without finishing your pre-course learning, you will find it difficult to understand what is taught in the course”) instead of the gains (e.g., “completing the pre-course learning will help you understand what is taught in the course”).

Research in other educational settings has shown that highlighting losses (consequences) is effective in improving learning outcomes. This is due to a behavioural bias called loss aversion, which refers to a tendency for people to react more strongly to losses over gains of the same amount.⁸

Between 2015 and 2016, Associate Professor Ben Smith and his colleagues conducted an experiment in a US university involving 217 students to determine if loss aversion can be used to improve students’ performance.

Loss Aversion Increases the Performance of University Students

Between 2015 and 2016, Associate Professor Ben Smith and his colleagues conducted an experiment in a US university involving 217 students to determine if loss aversion can be used to improve students’ performance.

Students in the control group began the term with 0 points and were awarded points after completing assignments or exams. Students assigned to the treatment group began the term with the maximum possible points, with points deducted for each error.

After accounting for factors such as their gender, age, and overall Grade Point Average, students in the treatment group saw significant improvement in their final score compared to control group peers.¹

Note

1. B. O. Smith, R. Shrader, D. R. White, J. Wooten, J. Dogbey, S. Nath, M. O'Hara, N. Xu, and R. Rosenman, “Improving Student Performance through Loss Aversion”, Scholarship of Teaching and Learning in Psychology 5, no. 4 (2019): 278–288, https://doi.org/10.1037/stl0000149.

3. Improve self-regulation

Research has shown that learners with higher levels of self-regulation (e.g., recognising that learning is their personal responsibility) learn more effectively in a flipped classroom.⁹, ¹⁰

Of all the barriers faced by the Least Likely Group, their challenge of learning independently is most worrying. They may find it difficult to self-regulate and make adjustments to their learning process to achieve learning goals.¹¹ Not all learners are homogeneous; we need a better understanding of this group’s challenges to implement pre-course learning better.

A study published in 2018 by Sun et al. sought to determine the self-regulatory factors affecting learning achievement in a flipped undergraduate Mathematics course of a US university.

Self-Efficacy and Help-Seeking Strategies Affect Self-Regulation

A study published in 2018 by Sun et al. sought to determine the self-regulatory factors affecting learning achievement in a flipped undergraduate Mathematics course of a US university.¹

Students with stronger belief in their ability (self-efficacy) in mathematics had better metacognitive (strategies to process information during learning) and environmental strategies (strategies to learn in a conducive environment). Highly self-efficacious students were also more likely to obtain higher achievements in their pre-class and in-class learning.

The study also found that help-seeking strategies had a positive effect: students who sought more help from others obtained higher grades for their pre-class learning. For better learning outcomes, the researchers suggested providing a platform for students to seek help from peers and trainers, and encouraging a culture where students were more comfortable seeking help from each other.

Building a student’s confidence in the subject area, as well as providing positive feedback on their progress and letting students observe how others solve the problems, were suggested as ways to increase student self-efficacy.

Note

1. Z. Sun, K. Xie, and L. H. Anderman, “The Role of Self-Regulated Learning in Students’ Success in Flipped Undergraduate Math Courses”, The Internet and Higher Education 36 (2018): 41–53, https://doi.org/10.1016/j.iheduc.2017.09.003.

CONCLUSION

Strategies to improve blended learning have been traditionally focused on designing the learning experience, curating the content of pre-course learning and paying attention to how it complements in-class sessions for blended learning. What actually goes into the implementation of pre-course learning has been given less attention. This is sometimes due to our assumptions of what pre-course learning should be (e.g., participants learning flexibly in their own time) and assumptions we have about adult learners (e.g., they can adapt easily to be independent learners).

The 2021 CSC study prompts us to rethink these assumptions and question if current approaches of blended learning are set up for the best outcomes, given the expectation of learning outside of work and the difficulties of balancing work and personal commitments. We should be aware that some adult learners need more help to learn independently: solving this problem is not only about their motivation but their capability as well. Even those with high motivation need help to follow through with their intentions.

For better outcomes, pre-course learning needs to be implemented in a behaviourally compatible way. It is only when we break down the behavioural barriers to precourse learning and find ways to improve the implementation, that working adult learners can fully benefit from the blended learning model. In this regard, behavioural interventions could make a difference.

ABOUT THE AUTHORS

Charmaine Lim is Lead Researcher at the Institute of Governance and Policy, Civil Service College. Her research interests include evidence-based policymaking and behavioural economics.

Mervin Loi is Researcher at the Institute of Governance and Policy, Civil Service College. His research interests include evidence-based policymaking and behavioural economics.

Do Hoang Van Khanh is Lead Researcher at the Institute of Governance and Policy, Civil Service College. Her research interests include evidence-based policymaking and behavioural economics.

NOTES

1. Nudges are ways to design the context or choice environment to influence people’s behaviours in a predictable way, while preserving their freedom of choice.

2. 95% of the respondents were Singapore public officers and 5% were overseas participants.

3. This poll was set up on OPPi, a platform that uses Artificial Intelligence to analyse the results. OPPi uses machine learning and advanced statistical techniques to identify and cluster opinion groups based on how similar or different respondents vote. More information on their methodology can be found on their website: https://www.oppi.live/faqs.

4. S. Michie, M. M. van Stralen, and R. West, “The Behaviour Change Wheel: A New Method for Characterising and Designing Behaviour Change Interventions”, Implementation Science 6, no. 42 (2011), https://doi.org/10.1186/1748-5908-6-42.

5. The Behavioural Insights Team, “Barrier Identification Tool”, https://www.bitbarriertool.com, n.d., accessed August 4, 2021.

6. See Note 4.

7. Y. Tu, and D. Soman, “The Categorization of Time and Its Impact on Task Initiation”, Journal of Consumer Research 41, no. 3 (2014): 810–822, https://doi.org/10.1086/677840.

8. Aurora Harley, “Prospect Theory and Loss Aversion: How Users Make Decisions”, Nielsen Norman Group, June 19, 2016, accessed September 21, 2021, https://www.nngroup.com/articles/prospect-theory/.

9. C. L. Lai and G. J. Hwang, “A Self-Regulated Flipped Classroom Approach to Improving Students' Learning Performance in a Mathematics Course”, Computers and Education 100 (2016): 126–140, https://doi.org/10.1016/j.compedu.2016.05.006.

10. G. S. Mason, T. R. Shuman, and K. E. Cook, “Comparing the Effectiveness of an Inverted Classroom to a Traditional Classroom in an Upper-Division Engineering Course”, IEEE Transactions on Education 56, no. 4 (2013): 430–435, https://doi.org/10.1109/TE.2013.2249066.

11. P. R. Pintrich, “The Role of Goal Orientation in Self-Regulated Learning”, in Handbook of Self-Regulation, eds. M. Boekaerts, P. R. Pintrich, and M. Zeidner (Academic Press, 2000), 451–502, https://doi.org/10.1016/B978-012109890- 2/50043-3.

Back to ETHOS Issue page