Learning Through Digital Media
Experiments in Technology and Pedagogy

Jonah Bossewitch and Michael D. Preston

Educators are becoming all too accustomed to the “glazing over” effect: students thinking they understand the material presented in videos shown in class or assigned for home viewing, but subsequent discussions often revealing minimal or superficial comprehension. This chapter is concerned with the development of tools and activities to help students attend to video-based materials with increased focus and heightened awareness of their own intellectual project. Specifically, we are interested in facilitating deeper and more critical explorations of video and encouraging students to marshal video-based primary sources as evidence to support their thinking. We hope to demonstrate that when video is introduced into a curriculum, these activities produce a deeper level of engagement, better understanding of the content, or even an improvement in students’ cognitive capacities for learning from video.

In our experience at a large research university, video is used increasingly across a wide variety of learning contexts, and is no longer limited to film and media studies. Video-based materials appear in the curricula of the humanities, natural and social sciences, and fine arts. Many pre-professional programs, from social work to teacher education, also rely heavily on video to support case-based learning: students are assigned to watch and critique scenarios in order to improve their skills of observation, interpretation, reasoning, and judgment. Schools of medicine, business, social work, journalism, and teaching have all embraced video-based learning materials, drawing on specially produced educational videos, library holdings, public broadcast archives, television and film. Furthermore, as production costs continue to plummet, we are also beginning to see more self-reflective learning activities in which students capture their own original video for subsequent analysis.

In this chapter, we explore a range of learning activities constructed around video annotations and multimedia composition. First, we describe some of the different forms that critical video compositions can take, and the importance of reinforcing the literacies involved in these new forms of expression and scholarship. We focus on the pedagogical motivations for introducing “video annotation” (which includes “clipping,” or the selection and naming of specific segments of video from longer videos) as an essential device for teaching with video: as a strategy that can help students learn to discriminate what matters from what does not and as a means for citing video content to support their ideas as part of a discursive essay, discussion, presentation, or other forms of knowledge production and sharing. Next, we explore tools that can be mobilized to support these pedagogies, including gratis web services and open-source alternatives we have developed ourselves to streamline students’ learning experiences. Finally, we discuss a variety of lesson plans and assignments created based on our experiences as staff of the Columbia Center for New Media Teaching and Learning (CCNMTL) collaborating with faculty at Columbia University. We describe educational activities and the supporting open-source web applications we have developed around gathering and organizing video annotations (clips), and around composing critical multimedia essays incorporating these clips.

With the support of Columbia University, the National Science Foundation, and the Institute of Museum and Library Services, we have developed software designed specifically to support these pedagogical strategies and have released them under open-source licenses.[1] Currently, we are only able to offer hosted solutions to the Columbia University community, so the installation and maintenance of our open-source applications requires a hosting provider and system administration skills. Please contact the authors if you are interested in collaborating around these solutions.

Pedagogical Shifts: New Approaches to Teaching with Video

For thousands of years, critical and scholarly discourse around text has revolved around citation and reference. The “critical essay” is a foundational genre in education and scholarship, predicated on establishing the provenance of sources that support an argument. Yet, twenty years after the invention of HTML, there is still little consensus around what this kind of discourse should look like for multimedia sources. The practice of studying and collecting multimedia sources to support these presentations is similarly in flux. As video becomes an increasingly common and important primary source, it is essential to continue experimenting with techniques for critically engaging these sources, conducting research and communicating these findings.

Citation in film studies is also evolving. Scholars in this field traditionally devoted a significant portion of their writing to describing the scenes they wanted to analyze, because they could not assume that their readership would have seen the works cited. In fact, the authors themselves often had limited access to their objects of study, and relied heavily on their memory in their analysis. But once VHS became popular, libraries and commercial vendors began to provide ready access to film and video works, and authors could finally rely upon access to the material they were analyzing. They studied the films with greater attention through repeated viewings and no longer needed to describe their sources in as much detail. Analog archives and rights-encumbered digital sources still remain a serious barrier to access, but a seismic shift is once again occurring around the study and presentation of digital video.

These practices converge for educators and students in today’s classroom. New presentation formats and methods for communicating with video have changed the landscape of what is possible, but these new possibilities amplify rather than minimize the need for educators to teach and model effective practices for careful analysis and conscientious citation.

Just as the study of text typically involves close reading and re-reading, highlighting and underlining, note taking and review, educators need to learn how to encourage analogous practices around video to foster a culture of serious scholarship through deep concentration and focus. Both online and offline digital video environments provide mechanisms for viewers to select and clip segments of the video they are watching in order to recontextualize and repurpose them. Video can be treated as a manipulative object, as raw material to be controlled, segmented, reorganized, reviewed, discussed and debated as part of an active learning experience, and the instructor or students can develop the narrative. While instructors may have to adapt their teaching practices to learn how to teach use of video as an object of analysis and as the basis of class discussion, they can also use the opportunity to model the technique of close viewing for students, and demonstrate how viewing can inform their own expert thinking about the material, which varies according to discipline.

Instructors must also help students develop sufficient curiosity and an orientation toward evidence that will motivate them to seek out information in the video. Assignments designed to force students to look closely and repeatedly at video interrupt the natural viewing flow, disrupting the passive acceptance of the video’s governing narrative and editorial intent. By contrast, the close viewing process requires students’ attention and cultivates the observational skills required to identify significant, interpretation-worthy moments. Thus, students develop the motivation to watch closely out of a sense that there is something worth discovering in the content. Furthermore, the burden of proof—making a convincing argument to a peer or instructor—requires careful selection of content and application within a written form.

These various forms of inquiry should be founded on healthy study habits that support the close reading of primary sources, evidence-based reasoning, and conscientious attribution. Whether students are producing video segments that juxtapose primary sources in the style of Jon Stewart’s Daily Show political analyses, or embedding media in a written essay in the style popularized by bloggers, educators need to model and teach their students the appropriate methods for engaging deeply with primary sources. Similarly, new rhetorical forms should not sacrifice the culture of rigorous and scholarly citation. Significant and ongoing work in multimedia environments with attention to these priorities raises awareness among students of provenance, rights, and access in a digital environment. In other words, close attention to these details will improve their media literacy.

Cultivating New Skills: Students in the Cutting Room

What we call a “video annotation” comprises identifying specific time codes in a video as well as metadata, tags, and working notes associated with a particular selection, or clip, of video. Offline video editing environments can support these operations through the direct manipulation of the source video material, and web-based video streaming services increasingly support direct referencing, embedding, and sharing of sub-selections of video. Standards committees are close to finalizing conventions for specifying so-called “time-based,” “isochronic” or “fine-grained” metadata for web-based video resources. However, these standards do not specify the design of user interfaces, workflows, and pedagogies leveraging these standards. In fact, the design of the student experience around video may largely proceed independently of the finalizing of these standards, so the iterative design of these analysis environments ought to inform their completion.

In addition to enabling the association of metadata and note-taking, the emerging standards also support and suggest new forms of composition and presentation. By specifying a particular sub-selection of a video, students can embed video clips in addition to creating a traditional bibliographic citation. This style of composition allows students to “quote” their video directly, instead of describing it in the text, or referring to the time codes corresponding to their analysis. Students might then produce a PowerPoint presentation, “film essay,” interactive media environment, or hyperlinked text with embedded multimedia. Some educators are even challenging the traditional rhetorical mode of the linear critical essay, and are experimenting with alternative rhetorical modes that suggest possibilities instead of trying to convince readers of a conclusion. The narrative form itself is being interrogated as archives, databases and collections operate as independent modes of participatory communication. Iconic examples of each of these styles of discourse are emerging, but students need clarity and guidance to master these forms of expression and persuasion.

The current generation of web-based video services is capable of supporting these learning activities, albeit with some curation and coaxing. When the videos being studied are hosted on YouTube, students can be instructed to utilize services like Splicd or TubeChop to clip specific selections, and then to compose their responses, incorporating their selections, in any multimedia-authoring environment, such as their learning management system, a course blog, or a Wiki. But while these tools can support the activities described above, the lack of integration between them can make for a cumbersome and clunky experience. For this reason, CCNMTL has been actively developing integrated web applications that streamline these educational workflows.

Learning with Video Annotation: Methodologies and Social Practices

 By heavily leveraging video annotation capabilities, we have designed a variety of learning engagements around video that emphasize the thoughtful separation of study—research, gathering, collection, and organization; composition—a dedicated workspace where students work with the clips they have collected to compose a critical essay; and assessment—where faculty can review and evaluate students’ compositions and provide them with feedback. In our curricular engagements, we have also explored variations in access models corresponding to the modalities enabled by social media. We support faculty who are committed to traditional solo assignments, faculty who blend online and in-class discussions, and faculty who are interested in exploring the possibilities around distributed research, collaborative authorship, and course-wide or even public sharing and publication.

The immediate goal of implementing these methods is to increase the likelihood that students will gather their own evidence of understanding of video based materials and to encourage them, through repeated viewing and manual interaction with the video content, to be deliberate in validating what they see and in explaining the connections between their evidence and claims about learning. A long-range goal of these methods is to help students develop a more contingent notion of truth that encourages them to entertain alternative hypotheses and promotes further inquiry as well as new ideas for teaching. This process represents a recursive, iterative response to new events; gathering more information creates the need for new evidence.

One important general finding is that videos do not need to be lengthy to be compelling. In fact, shorter segments may place greater emphasis on close viewing and resultant comprehension. We have found that when students are encouraged to view a clip repeatedly, and their viewing is scaffolded through prompts and questions about the content, they learn to look with increasing granularity. Withholding evidence by stopping the clip at significant moments also helps students develop an awareness of their own uncertainty, forcing them to make their best assessment using only the information that is immediately available to them.

Lessons Planned, Lessons Learned

The following multimedia-based assignments are a sampling of some of the approaches we have developed with Columbia faculty:

• Guided lessons: Instructors preselect video clips and organize them into a specific sequence to be viewed by students, who must answer questions associated with each video segment. Guided lessons have been successfully used for teaching skills, such as clinical interviewing, by simulating the process of responding to an interviewee, prompting the student to interpret what they saw, propose the next question, or critique the interviewer’s technique in the video.
• Lecture comprehension: Students are assigned to view a recorded interview or lecture and then select three segments and comment on them. Students are then instructed that comments should be in their own words and to avoid repeating the words of the source. The first comment should be one that they think is a novel notion. The second should be something they do not understand, a difficult idea, or something they want to understand better. The third is a segment that they think is related to the current classroom dialogue.
• Close object analysis with targeted comparisons: Students work with a curated collection of multimedia learning objects, and select two objects to closely compare and contrast. They work individually to write comparison essays, embedding specific annotations from within the object to illustrate and support their claims. Next, students are asked to study the comparison projects of other students in the class, leave comments and questions about their findings, and keep notes about additions they might make about their own project.
• Communal hunting and gathering, with in-class synthesis: Students are introduced to a curated collection of sources, but are also encouraged to explore pertinent cultural representations available on the open Web. In this environment, the annotations that students create are shared across the class, and an explicit learning objective is the transference of a “judicial selection” of source material from faculty to students. Students gather objects and then compare their selections during in-class discussion. Finally, students compose final projects that incorporate these annotations.
• Collective analysis across semesters of a core set of resources: Students explore an archive of a serialized work, such as a digitized newspaper, to investigate patterns that emerge over time but might not otherwise be detected by the typical consumer of the source material, and whose focus might be less critical or not longitudinal. These findings are collected and shared in a class investigation of a particular resource.
• Reflection on self-evaluations/performances: Students videotape their own performances as pre-service teachers, therapists, doctors, etc. and then write an analysis, self-critique or reflection, embedding clips from their performances to illustrate points raised, according to criteria established by the instructor. Students learn to recognize successful and unsuccessful behaviors they can correct and to utilize self-reflection as a tool for ongoing improvement as a professional.

Conclusion

As the means of accessing and using video for education rapidly expand, we want to reinforce the notion that pedagogy matters. The mere availability of video alone is not sufficient to improve educational outcomes; pedagogical approaches to video that encourage close reading through annotation and composition may help do so. These methods encourage students to treat video sources critically as raw material for discourse and analysis. In our world, complexity is inherent, and one goal of educating students should be to help them embrace this complexity and develop a propensity to examine it more closely and to interpret it with appropriate intellectual rigor.

The pedagogical strategies suggested in this chapter have the added benefit of making video that is available on the open Web more relevant to education. They provide students with a way to bridge their media experiences outside the classroom with serious scholarship and research. The practices of participatory education and research extend beyond the classroom into activism, advocacy, journalism and government. Recent advances in video production, editing, and communication technology make once technically challenging activities into a fairly commonplace enterprise, and students who learn to think critically about video can also learn to use that video as a medium for persuasive expression and dialogue with others.

In this chapter, we have purposely avoided focusing on our own tools in favor of a more generalized approach to using video for teaching and learning. While our tools do facilitate smoother workflows and group permissions, the pedagogical principles and examples we have provided are not necessarily dependent on specific tools. We hope that readers will conclude this chapter with ideas for how they might purposefully utilize video in their own teaching and adopt new methods for engaging their students.

---

[1] To download and learn more about our web-based multimedia annotation tools, VITAL and MediaThread, please visit <http://ccnmtl.columbia.edu/our_services/tools/vital/> and <http://ccnmtl.columbia.edu/mediathread>, respectively.