Journal of Public Pedagogies
Alison Anderson Holland
University of Minnesota
In June of 2015, I began my position as an Academic Technologist for University of Minnesota Extension’s Technology Department in the United States of America. Extension is the outreach arm of the University, working to bring the University’s research and resources to the people of its state. In other words, Extension connects communities with the latest knowledge the University has on important issues, and connects faculty members with communities to further develop their understanding of critical issues through action research. Topics include, but are not limited to, food safety and access, youth and community development, sustainable farming and environmental practices, and parenting and financial decisions for families (University of Minnesota Extension, About section). In my role, I was tasked with promoting and supporting academic technologies for teaching by Extension faculty and staff. Nearly everyone I spoke to in the organization in my first year was troubled by the low completion rates in their online courses.
In the literature there is a great deal of agreement that online learner engagement and course completion rates are often low, especially in the non-credit setting (Clow, 2013; Rovai, 2003; Tyler-Smith, 2006; Walji et al, 2016). While there is a breadth of research on how to engage online learners in traditional online courses (Carr, 2000; Parker, 1999; Patterson and McFadden, 2009), the literature does not adequately describe the Internet at large as a means of content delivery that can lead to learning. It also fails to sufficiently describe the way people engage with different types of website designs to learn online informally in daily life (Sefton-Green, 2004). There are inherent challenges in assuming online courses are the primary way to provide online educational content to audiences who are not seeking credit or certification.
Discovering the field of Public Pedagogy, and as a result thinking more broadly about what ‘out of school’ learning might entail, was extremely useful in reframing this issue and potential solutions. Similarly, in their review of the field, Sandlin, O’Malley, and Burdick (2011) suggest one problem in understanding and defining public pedagogies is our existing cultural frameworks for understanding what pedagogy is, or should be. Furthermore, Sandlin O’Malley, and Burdick (2011) suggest a need for ‘heightened attention to empiricism in the inquiry process’ (p. 359). Therefore, utilizing the spiral of inquiry framework (see Figure 1) in an effort to break free from existing perceptions of pedagogy and to frame an empirical inquiry, this paper shares a developing concept of public pedagogy for the Web (Timperley, Kaser, and Halbert, 2014).
In 2012, Coursera co-founder Daphne Koller spoke at TED about the extensive reach of their Massive Open Online Courses (MOOCs) with enrollees across the world. She spoke about learning from watching students learn, and fail, in patterns. Instructors were able to modify their strategies to more effectively teach the content. But what she did not speak to is the miniscule completion rates in MOOCs. Hill (2013) elegantly illustrates the alarming portion of “no-shows,” as well as the disheartening trajectory of participation from “observers,” “drop-ins,” “passive participants,” and “active participants” in his line graph (see Figure 2). This closely aligns with our understanding of what is happening in Extension’s online courses as well. Though Extension’s courses are not MOOCs, they share many commonalities, namely for both, learners often have few or no external motivators to complete them.
Figure 1. From Timperley, H, Kaser, L, & Halbert, J 2014, A framework for transforming learning in schools: Innovation and the spiral of inquiry (Vol. 234), Centre for Strategic Education, Melbourne, p. 5
Figure 2. Emerging student patterns in Coursera-style MOOCs by Phil Hill.
I thought for a while about the fact that perhaps we were measuring the wrong things. Maybe completion did not matter. After all, it is not a measure of learning. Many of our students come to our content without the need to complete anything. There is no significant reason for them to finish, other than intrinsic motivation. Perhaps students come to get what they need, such as a resource or a skill, find it and leave. That is a success, right? But that is not really the purpose of a course, and the structure is not well designed to allow for this ebb and flow without hindering other students’ experiences. If we better understood how people learn online informally, maybe we could redesign our pedagogical approach to sharing content for learning online and redefine success in Extension’s outreach education.
Today, in a time of doing more with less, Extension’s educators and program staff often try to transfer workshops and courses, that had been delivered in-person or via postal mail, to an online learning management system (LMS) like Moodle, sell them to organizations and learners, and then ask why people do not complete their courses. I see the need for Extension Educators to move past that question. Instead, I would like to focus on the mission or the “why” behind even holding the course. Extension wants to connect, share, and collaborate with community members and professionals who are seeking credible information and opportunities to learn that will help them improve their lives, professions, and communities. What are other ways to achieve that and better serve Extension’s potential audience, the people of the state? How can Extension best satisfy its mission to “Make a difference by connecting community needs and University resources to address critical issues in Minnesota” (University of Minnesota Extension, About section) in the Web 2.0 era?
I believe the answer lies in our ability to conceptualize and design public websites pedagogically. User-centered design can be an effective tool to increase engagement and learning; it can “enhance the effectiveness of communication, increasing the capacity of the recipient to engage with the information and learn from the communication” (Kelly, 2015, p. 393). Like museums do with physical spaces, online outreach education must not only consider the design of the educational content they create and share, but also the structural and functional design of the website in order to enable the activities that will allow for the most effective learning environment possible.
Researchers have a very limited understanding of what, how, and how well adults learn informally, with the bulk of related case study work occurring in the 1970s (Livingstone, 2007). Today is a very different time. With Web 2.0 in most of our pockets, “information is readily and easily accessible” and “change is so rapid that traditional methods of training and education are totally inadequate” (Hase and Kenyon, 2000, para. 3). Existing literature fails to sufficiently describe the way people learn online informally in daily life (Sefton-Green, 2004). To identify what is currently known about informal online learning and the recent trends in this research, I completed a metasynthesis review of the literature (Holland, 2019). With this approach, I sought to examine and interpret the findings of a range of qualitative or mixed method studies to produce findings that are more substantive than those in individual investigations (Finfgeld, 2003). I took existing understandings from discrete studies to develop a more thorough body of knowledge and develop a theory around effective principles of informal online learning. In bringing 22 studies together, I identified that informal online learning is supported when the content is segmented into learning objects that are titled and tagged appropriately; interaction opportunities support knowledge construction and learner empowerment; and flexibility and choice are paramount to the design (Holland, 2019). A connecting thread in these principles is the philosophy of constructivism and the theory of heutagogy. These established frameworks, therefore, may be particularly useful lenses through which to draw on for future research on informal online learning.
Coincidentally, an informal set of focus groups conducted as formative evaluation with an Extension team working on nutrition education had similar conclusions. Focus group participants shared decisively that they were not interested in taking courses with prescribed lessons. They wanted information they could trust, delivered through small engaging segments of content that were easy to search for and find precisely what and when they wanted. They also had an interest in sharing and interacting with peers. This vision calls for an online framework that is quite distinct from the LMS framework to which we are accustomed.
Abandoning the course-like structure for their e-learning initiative, the nutrition education team referenced in the focus group discussion above is currently working to develop a public website to share the educational content present in their in-person courses. For this, we have segmented the learning content into short videos and GIFs, and have been planning opportunities for interaction. The key focus for this team now is to develop a website infrastructure that will allow them to provide open educational resources, while still tracking data required by their funder and that will allow for evaluation of the efficacy of the “intervention,” also known as an educational website.
One of the primary challenges in academia when you set learning content free for access and utilization outside the LMS is finding effective means of measuring its use and learner outcomes. When you leave the LMS, you also leave pre-developed systems for easy learner-specific data gathering behind. However, there are other, perhaps more useful, modes of outcome evaluation. For example, the RE-AIM framework, which measures a resource or intervention’s reach, effectiveness, adoption, implementation, and maintenance, can be employed to understand the influence of a resource or intervention, and is particularly helpful in the developmental evaluation phase (Cronin, Hendrickson, and Croymans, 2018). Similarly, Guion and Free (2010) suggest a conceptual framework for infusing behavior change theories into program design, delivery, and evaluation utilizing a combination of the transtheoretical model of behavior change, diffusion of innovations theory, and an ecological approach (See Table 1). This practical illustration of the application of well-established theories in modern online spaces is of critical importance to moving this work forward. Tobey, Koenig, Brown, and Manore (2016) took a related approach to measuring the outcomes of an educational social marketing campaign. While not primarily an online intervention itself, their evaluation method is applicable to educational websites. Tobey et al. (2016) conducted interviews, recording on a five-point scale how strongly participants agreed with belief statements before and after the launch of the campaign. The measure of change in one’s belief is an arguably more clear measure of actual learning outcomes and community impacts than simple fact-based quizzes commonly used to measure learning in online courses.
Table 1. Guion and Free’s (2010) Framework for Designing, Delivering, and Evaluating Behavior Change Focused Educational Strategies
I propose public pedagogy for the Web requires a non-linear approach and an integrated design. The intended learning outcomes should be supported by learning activities, learning resources, and learning supports, the three of which sometimes intersect (See Figure 3). This is similar to Oliver’s (1999) model for illustrating learning design sequences in a temporal format (see Figure 4), but necessarily deviates significantly in its illustration of the learning setting. Instead of a structured semester with sequential activities with prescribed resources and pre-planned supports described by Oliver (1999), the self-directed learner in an informal online space does not take a linear path through materials nor interactions. Therefore, the illustration of the learning experience may look more like a child’s scribbles than a neatly diagramed semester (see Figure 5). However, that does not make it less valid. This theorized pedagogy of the Web is limited primarily by the lack of evaluation data on its implementation. Future studies should map actual learning paths and outcomes for learners who utilize websites designed with the pedagogy described.
Figure 3. Adapted from Oliver, R. (1999). ‘Exploring strategies for online teaching and learning’. Distance Education, 20(2), p. 240-254.
Figure 4. Adapted from Oliver, R. (1999). ‘Exploring strategies for online teaching and learning’. Distance Education, 20(2), p. 254.
Figure 5. An illustration of a theoretical self-directed informal online learning experience that plays out over time with multiple return visits to an educational website.
Carr, S 2000, ‘As distance education comes of age, the challenge is keeping the students’, Chronicle of Higher Education, Vol. 46, Iss. 23, pp.A39-A41.
Clow, D 2013, ‘MOOCs and the funnel of participation’, in Suthers, D., Verbert, K., Duval, E., X. Ocha (ed), Proceedings of the Third International Conference on Learning Analytics and Knowledge, ACM, Belgium, pp. 185-189.
Cronin, S, Hendrickson, L, & Croymans, S 2018, ‘Using the RE-AIM framework to evaluate disaster recovery videos’, Journal of Human Sciences and Extension, Vol. 6 Iss. 1, pp.36-57.
Finfgeld, DL 2003, ‘Metasynthesis: The state of the art—so far’, Qualitative Health Research, 13, pp.893-904
Guion, LA, & Free, TR 2010, ‘A conceptual framework for infusing behavior change theories into program design, delivery and evaluation: A financial education example’, The Forum for Family and Consumer Issues, Vol. 15 Iss. 1, [e-journal]. Retrieved from: https://www.theforumjournal.org/2010/03/03/a-conceptual-framework-for-infusing-behavior-change-theories-into-program-design-delivery-and-evaluation-a-financial-education-example/
Hase, S and Keyton, C 2000, ‘From andragogy to heutagogy’, viewed 21 February 2018 , <https://epubs.scu.edu.au/gcm_pubs/99/> .
Hill, P 2013, ‘Insight on MOOC student types from ELI Focus Session’, viewed 23 February 2017, <https://mfeldstein.com/insight-on-mooc-student-types-from-eli-focus-session/>.
Holland, AA 2019, ‘Effective principles of informal online learning design: A theory-building metasynthesis of qualitative research’, Computers & Education,128, 214-226. doi:10.1016/j.compedu.2018.09.026
Kelly, M 2015, ‘Visual Communication Design as a form of public pedagogy’, Australian Journal of Adult Learning, Vol. 55, Iss. 53, pp.390-407
Koller, D 2012, ‘What we’re learning from online research’, viewed 26 September 2015, <https://www.ted.com/talks/daphne_koller_what_we_re_learning_from_online_education?language=en>.
Oliver, R 1999, ‘Exploring strategies for on-line teaching and learning’, Distance Education, Vol. 20, Iss. 2, pp.240-254
Livingstone, DW 2007, ‘Informal learning: conceptual distinctions and preliminary findings’, in Bekerman, N.C., Silberman-Keller, D. (ed), Learning in places: The informal education reader, 1st ed., Peter Lang, United States, pp.203-227
Parker, A 1999, ‘A study of variables that predict dropout from distance education’, International Journal of Educational Technology, Vol. 1, Iss. 2, pp.1-10
Patterson, B, McFadden, C 2009, ‘Attrition in online and campus degree programs’, Online Journal of Distance Learning Administration, Vol. 12, Iss. 2, pp.1-11
Rovai, AP 2003, ‘In search of higher persistence rates in distance education online programs’, Internet and Higher Education, Vol. 6, Iss. 1, pp.1-16
Sandlin, JA, O’Malley, MP, and Burdick, J 2011. ‘Mapping the complexity of public pedagogy in scholarship: 1894-2010’, Review of Educational Research, Vol. 81. Iss. 3, pp.338-375
Sefton-Green, J 2004, Informal learning with technology outside school, 1st ed., Futurelab, England
Timperley, H, Kaser, L and Halbert, J 2014, A framework for transforming learning in schools: Innovation and the spiral of inquiry (Vol. 234), Centre for Strategic Education, Melbourne.
Tobey, LN, Koenig, HF, Brown, NA and Manore, MM 2016, ‘Reaching Low-Income Mothers to Improve Family Fruit and Vegetable Intake: Food Hero Social Marketing Campaign—Research Steps, Development and Testing’, Nutrients, Vol. 8, Iss. 9, p.562
Tyler-Smith, K 2006, ‘Early attrition among first time eLearners: A review of factors that contribute to drop-out, withdrawal and non-completion rates of adult learners undertaking eLearning programmes’, Journal of Online learning and Teaching, Vol. 2, Iss. 2, pp. 73-85
University of Minnesota Extension, (n.d.). About section, viewed 23 February 2017, <http://extension.umn.edu/about>.
Walji, S, Deacon, A, Small, J and Czerniewicz, L, 2016. ‘Learning through engagement: MOOCs as an emergent form of provision’, Distance Education, Vol. 37, Iss. 2, pp.208-223