Connected Courses MOOC (#ccourses) and #oclmooc: Assessing Connected Learning Outcomes

Mimi Ito, Vera Michalchik, and Bill Penuel take us into the wonderfully intriguing deep end of the Connected Learning swimming pool in the latest Connected Courses MOOC (#ccourses) session, as anyone who attended the live version or catches it through the archived recording can confirm. And that’s a great thing since the deep end of any body of water is often where we find the interesting signs of life.

Continuing the massive open online course’s current two-week exploration of what drives the learning process (“Why We Need a Why”), the three educators interact with their online learners by exploring connections between learning (particularly connected learning), learning assessments, and learning outcomes.

This is far from the usual review of how well learners do within the confines of an explicitly defined learning experience—a semester-long course or, by extension, the sort of workplace learning and performance (staff training) offering that so many of us as trainer-teacher-learners provide and then review through a formal assessment process. It’s a connected-learning session that takes us into the outer reaches of models like Donald Kirkpatrick’s four levels of learning, where the gold standard is to ask what impact the learning eventually has not only on the learner, but on the community the learner ultimately serves. And it encourages us to take the learner’s point of view into account rather than focusing solely on the learning facilitator’s or learning organization’s vantage point.

“When you look from the perspective of the learner…the learning ecosystem looks very different than when you’re looking from the perspective of the class,” Ito reminds us during the session; it requires that we are as cognizant of what the learners bring to the learning space as we are of our own approach to facilitating the learning process—thinking about the learners “and how our practices in the classroom connect back out.”

“We want to take a long view of the outcomes,” Penuel suggests. “Do kids get to where they want to go next?…Where did this class lead them?”—questions we should just as firmly be asking within any training-teaching-learning setting.

All three presenters lead us into the deeper, far more significant question: Do we follow up with our learners to see “whether learning made a difference in their lives?”

“We have so few tools and practices that enable us to know what happened to students over time…a lot of these outcomes play out over time…you don’t really know what happens a few years down the line because nobody actually studies that,” Ito says.

But the picture is not completely bleak, she adds. A Gallup-Purdue University study released in May 2014 and designed to document post-graduation levels of workplace engagement and overall “well-being” provides some guidance for learning facilitators seeking ways to provide long-term positive benefits through their efforts. Learning that was project-based and that provided meaningful connections between the learners and those facilitating their learning led to significantly higher levels of workplace engagement and overall chances that the learners would thrive “in all areas of well-being.”

“It’s not just what kids got out of the course…but what happens next, “Ito reiterated.

Much of this, of course, leads us back to the goal of better understanding what connected-learning practices (fostering learner-centric approaches, finding ways to “harness the advances and innovations of our connected age to serve learning,” and nurturing deeper learning and understanding) might provide for learners of all ages, in a variety of training-teaching-learning settings.

“Interest is the beginning point, but the idea of cultivating an interest is that you don’t know where it is going to go,” Penuel notes. “Our interests are really a starting point, and we really do need experiences of various kinds that allow us to learn in order to deepen them—so develop knowledge related to our interests, to engage with others in relation to our interests—to find out what others are interested in and to perhaps do something with them. I think interests evolve, and I think new interests emerge from this deep engagement.”

All three presenters, as the session draws to an end, remind us that we are not lacking resources if we want to join them in their explorations. The Connected Learning Research Network, for example, engages in learning research, design, and practice. And the National Survey of Student Engagement is currently looking into how much time and effort students in higher education put into their learning—as well as how our colleagues in higher education are “deploying resources” to foster engagement among learners.

As we reflect upon what #ccourses and another current connectivist MOOC—the Open  and Connected Learning MOOC (#oclmooc)—offer trainer-teacher-learners in terms of guidance and inspiration, we can’t help but be encouraged. They remind us to reflect upon what our own learning produces. They also consistently and continually serve as examples of how the connections these connected-learning opportunities contribute to our own growth, productivity, and satisfaction within the extended communities of learning we create, nurture, and sustain over a very long period of time.

N.B.: This is the fourth in a series of posts documenting learning through #ccourses and #oclmooc.

ALA Annual Conference 2012: When Learners Create Learning Objects

Put a group of trainer-teacher-learners into a room, and you’ll quickly see barriers dissolve and information flow, as happened yesterday during an ALA Learning Round Table “Nuts and Bolts of Staff Training” discussion here in Anaheim at the American Library Association (ALA) Annual Conference.

Facilitators Maurice Coleman and Sandra Smith, who serve on the Learning Round Table board of directors, facilitated a 90-minute session that informally took participants through a start-to-finish tour of problems and solutions in workplace learning and performance (staff training) programs. And most of the solutions came from participants themselves as experienced colleagues shared ideas and resources with those new to the profession—and those relatively new to the profession quickly learned that they had plenty to contribute through the questions that they raised and the suggestions they themselves contributed.

The session also served as a good example of facilitated and experiential learning. Participants initially identified key challenges they face in their workplace learning and performance programs. That exercise helped establish the start-to-finish overview: how to successfully manage programs with a one- or two-member training department; identify and respond to the needs of different learners (including those with diverse cultural backgrounds); choose the tech tools that allow us to manage course offerings, registration, course content, and feedback through evaluations; make learning accessible to learners; deliver effective learning opportunities; and decide how to effectively manage the evaluation process.

Attempting to tweet the responses provided a learning opportunity in and of itself: how to create a learning object from learners’ class discussions as documented through a Twitter feed in TweetChat. By capturing comments in 140-character summaries, we were able to produce the Twitter feed (available at @trainersleaders for June 25, 2012) that participants can review, and I’ve also written this article in the hope that it can alone as a useable lesson/summary of best practices cited by active trainer-teacher learners.

Several samples from the twitter feed, edited and expanded since we are not constrained by the 140-character limit in this posting, are offered here:

• To be an effective trainer-teacher-learner, strive to play a leadership role within your organization.
• Reach learners who are new to tech tools by using peers as instructor/facilitators rather than always relying on those seen as “techies,” e.g., members of the organization’s IT staff.
• Connect learners with learning opportunities by making information about training sessions clear and accessible.
• Be sure that training sessions support organizational goals and objects so learners are effectively served by the learning opportunities they accept.
• Provide clear, concise, and measurable learning objectives so managers and learners know what to expect and so that we have the framework to conduct successful and meaningful evaluations after learners return to their worksites and begin using what they learn.
• Recognize that learners best absorb new information in relatively brief chunks—generally no more than 10 minutes in duration, although there is quite a bit of disagreement among trainer-teacher-learners on this topic—and offer learners frequent opportunities to apply what they are learning.
• Incorporate playfulness into learning to decrease stress (which limits a learner’s ability to absorb new information) and to make the learning experience memorable, e.g., offer “sit and play” sessions where new learners become comfortable by actually using the tech devices they are going to use in their workplace.
• Create online sandboxes for learners—spaces where they can find tools and resources they want to try and master.
• To be sure learners use what they learn, create clear tools and avenues for accountability.
• Use evaluation models including Donald Kirkpatrick’s four levels of learning evaluation and Jack Phillip’s model for Return on Investment in Training and Performance Improvement Programs.

There was, of course, much more to the session than can be captured in a relatively brief summary—including the idea that some of the best learning occurring yesterday came from the realization that people from small training units are far from alone when they turn to their own communities of learning, including the ALA Learning Round Table.

Training Trends, Learning Outcomes, and Setting More Productive Goals

When we look at trends and predictions for workplace learning and performance (training) in pieces such as Training Industry, Inc. CEO and Founder Doug Harward’s recent article posted on TrainingIndustry.com, we find an intriguing combination of potentially positive changes and misdirected attention.

The positive elements include predictions that “total spending for training services” will increase by seven to nine percent in 2011; “the role of the learning leader” in organizations is changing for the better; “learning technologies are becoming social, collaborative, and virtual”; and “learning content will be transformed for easier consumption”—situations many of us have already been seeing or can, without too much thought, accept as likely.

Sources including the ASTD (American Society for Training & Development) “2010 State of the Industry Report” confirm that training remains a well funded industry in some ways even though many of us note and lament reductions in training budgets: “U.S. organizations spent $125.88 billion on employee learning and development  in 2009” (p. 5)—the year during which the data in the 2010 report was gathered. The eLearning Guild’s “Getting Started with e-Learning 2.0” and co-writers Tony Bingham and Marcia Conner, in their book The New Social Learning: A Guide to Transforming Organizations Through Social Media, provide support for the idea that social media tools are already making a positive difference in fostering learner-centric training. And interviews that Lori Reed and I conducted for our forthcoming book on trainers as leaders (Workplace Learning & Leadership) document the increasingly important roles workplace learning and performance professionals are assuming in developing, delivering, and evaluating effective learning opportunities.

One particularly interesting assertion among Harward’s predictions is that learning leaders are becoming solutions architects or learning architects—“someone who designs innovative approaches for employees to access knowledge, when they need it, in relevant chunks, no matter where they are.” This, he suggests, moves them/us closer to the role of consultant—a role which trainer-consultants including Peter Block (Flawless Consulting) and the late Gordon and Ronald Lippitt (The Consulting Process in Action—particularly Chapter 6) have abundantly described in their own work when they write of internal and external consultants (long-term employees as opposed to those hired for well defined projects with specific beginning and end points).

As was the case with Training Industry, Inc.’s report on “How to Promote the Value of Online Training Within Your Organization,” however, there is a bit of myopia among the predictions. The proposal that “metrics for learning will be based on content access, view, involvement, and downloads” rather than “how many students attended a program” doesn’t appear to provide a significant and positive change; furthermore, it ignores the larger issue to be addressed: is all this workplace learning leading to positive change for learners, organizations, and the customers and clients they serve? The unfortunate answer, as documented elsewhere, is an emphatic “no.”

More importantly, this proposed shift in focus misses the larger mark because it still makes no attempt to engage in the levels of assessment suggested in Donald Kirkpatrick’s Four Levels™ Evaluation Model, Robert Binkerhoff’s Telling Training’s Story: Evaluation Made Simple, Credible, and Effective, or Calhoun Wick, Roy Pollock, Andrew Jefferson, and Richard Flanagan’s The Six Disciplines of Breakthrough Learning: How to Turn Training and Development Into Business Results—those measurements of workplace learning and performance’s real results in terms of positive change.

There is much to admire in what Hayward writes. There is also obviously much room for seeking trends that, in his words, “will reshape the training industry” in a significant and sustainable way. All we have to do is keep our attention on the learners and those they serve. And set even more productive, measurable goals.